Prosody-Based Adaptive Metaphoric Head and Arm Gestures Synthesis in Human Robot Interaction - ENSTA Paris - École nationale supérieure de techniques avancées Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2013

Prosody-Based Adaptive Metaphoric Head and Arm Gestures Synthesis in Human Robot Interaction

Amir Aly
Connectez-vous pour contacter l'auteur

Résumé

In human-human interaction, the process of communication can be established through three modalities: verbal, non-verbal (i.e., gestures), and/or para-verbal (i.e., prosody). The linguistic literature shows that the para-verbal and non-verbal cues are naturally aligned and synchronized, however the natural mechanism of this synchronization is still unexplored. The difficulty encountered during the coordination between prosody and metaphoric head-arm gestures concerns the conveyed meaning , the way of performing gestures with respect to prosodic characteristics, their relative temporal arrangement, and their coordinated organization in the phrasal structure of utterance. In this research, we focus on the mechanism of mapping between head-arm gestures and speech prosodic characteristics in order to generate an adaptive robot behavior to the interacting human's emotional state. Prosody patterns and the motion curves of head-arm gestures are aligned separately into parallel Hidden Markov Models (HMM). The mapping between speech and head-arm gestures is based on the Coupled Hidden Markov Models (CHMM), which could be seen as a multi-stream collection of HMM, characterizing the segmented prosody and head-arm gestures' data. An emotional state based audio-video database has been created for the validation of this study. The obtained results show the effectiveness of the proposed methodology.
Fichier principal
Vignette du fichier
Aly-ICAR2013.pdf (1019.44 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01180239 , version 1 (24-07-2015)

Licence

Domaine public

Identifiants

Citer

Amir Aly, Adriana Tapus. Prosody-Based Adaptive Metaphoric Head and Arm Gestures Synthesis in Human Robot Interaction. The 16th IEEE International Conference on Advanced Robotics (ICAR), Nov 2013, Montevideo, Uruguay. ⟨10.1109/ICAR.2013.6766507⟩. ⟨hal-01180239⟩

Collections

ENSTA ENSTA_U2IS
45 Consultations
169 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More