Crossmodal prosodic and gestural contribution to the perception of contrastive focus

Authors.
Pilar Prieto, Cecilia Pugliesi, Joan Borràs-Comes, Ernesto Arroyo, and Josep Blat .
Information and Communication Technologies Department, Universitat Pompeu Fabra

Published in:
12th Annual Conference of the International Speech Communication Association, Interspeech 2011

Abstract:
Speech prosody has traditionally been analyzed in terms of acoustic features. Even though visual features and gestures have been shown to help and enhance linguistic processing, the conventional view is that facial and body gesture information in oral (non-sign) languages tends to be redundant and has the role of helping the hearer recover the meaning of an utterance. We conducted two perception experiments with a 3D animated character showing conflicting auditory and visual information to investigate two related questions regarding the importance of gestures in conveying prosodic meaning: (a) how important are facial cues with respect to auditory cues for the perception of contrastive focus?; and (b) what is the relevance of the different gestural movements (i.e., head nod and eyebrow raising) for the perception of this type of focus? Our findings lend support to the notion that the visual component is crucial in the semantic interpretation of contrastive focus.

Download:
- Publisher
- Author’s version

Citation:
Prieto., Pugliesi C., Borràs-Comes J., Arroyo E., Blat J. Crossmodal prosodic and gestural contribution to theperception of contrastive focus to the perception of contrastive focus. In 12th Annual Conference of the International Speech Communication Association, Florence, Italy, 2011.