Visual Augmentation of Deictic Gestures in MOOC Videos

We present an eye-tracking study to compare different modalities for visual augmentations of the teacher’s explicit deictic gestures on a video lecture. We compared three visualizations: 1) hand gestures with a pointer, 2) gaze overlay, and 3) no-augmentation baseline. We investigate the teacher-student pair in a video-based learning context as an abstraction of an expert-novice pair where the goal is to attain a high level of shared understanding. The key phase of having a shared understanding is to have a common ground between the pair. Previous studies showed that explicit deixis plays a major role in initiating and maintaining common ground. This led us to hypothesize that augmenting videos with teacher’s deictic gestures might help students perform better. We found that augmenting the video with teacher’s gaze results in higher learning gain than no visualization. Moreover, gaze visualization also helped students in maintaining longer attention spans than hand gestures.