Let Your Body Speak: Communicative Cue Extraction on Natural Interaction Using RGBD Data

Employment interviews are relevant scenarios for the study of social interaction. In this setting, social skills play an important role, even though the interactions between potential employers and candidates are often limited. One fundamental aspect of social interaction is the use of nonverbal communication, which affects how we are socially perceived. We present a method to automatically extract body communicative cues from one-on-one conversations recorded with Kinect devices. First, we find the three-dimensional position of hands and head of the subject, and, aided by training data, we infer the upper body pose. Then, we use the inferred poses to perform action recognition and build person-specific activity descriptors. We evaluate our system with both domain-specific and public, generic datasets, and show competitive performance.