[go: up one dir, main page]

Skip to main content

Multimodal Sensing, Interpretation and Copying of Movements by a Virtual Agent

  • Conference paper
Perception and Interactive Technologies (PIT 2006)

Abstract

We present a scenario whereby an agent senses, interprets and copies a range of facial and gesture expression from a person in the real-world. Input is obtained via a video camera and processed initially using computer vision techniques. It is then processed further in a framework for agent perception, planning and behaviour generation in order to perceive, interpret and copy a number of gestures and facial expressions corresponding to those made by the human. By perceive, we mean that the copied behaviour may not be an exact duplicate of the behaviour made by the human and sensed by the agent, but may rather be based on some level of interpretation of the behaviour. Thus, the copied behaviour may be altered and need not share all of the characteristics of the original made by the human.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Martin, J.C., Abrilian, S., Devillers, L., Lamolle, M., Mancini, M., Pelachaud, C.: Levels of representation in the annotation of emotion for the specification of expressivity in ecas. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS, vol. 3661, pp. 405–417. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  2. Peters, C.: Direction of attention perception for conversation initiation in virtual environments. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS, vol. 3661, pp. 215–228. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  3. Scherer, K., Ekman, P.: Approaches to Emotion. Lawrence Erlbaum Associates, Mahwah (1984)

    Google Scholar 

  4. Ekman, P., Friesen, W.: The Facial Action Coding System. Consulting Psychologists Press, San Francisco (1978)

    Google Scholar 

  5. Tian, Y., Kanade, T., Cohn, J.: Facial expression analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of face recognition (2003)

    Google Scholar 

  6. Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573 (2005)

    Google Scholar 

  7. Morency, L.P., Sidner, C., Lee, C., Darrell, T.: Contextual recognition of head gestures. In: Proceedings of ICMI 2005, Trento, Italy (2005)

    Google Scholar 

  8. Gratch, J., Marsella, S., Maatman, M., Gratch, J., Marsella, S.: Natural behavior of a listening agent. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661. Springer, Heidelberg (2005)

    Google Scholar 

  9. Donato, G., Bartlett, M., Hager, J., Ekman, P., Sejnowski, T.: Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999)

    Google Scholar 

  10. Wu, Y., Huang, T.: Hand modeling, analysis, and recognition for vision-based human computer interaction. IEEE Signal Processing Magazine 18, 51–60 (2001)

    Article  Google Scholar 

  11. Wexelblat, A.: An approach to natural gesture in virtual environments. ACM Transactions on Computer-Human Interaction 2, 179–200 (1995)

    Article  Google Scholar 

  12. Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for effort and shape. In: ACM SIGGRAPH 2000, New Orleans, LA, pp. 173–182 (2000)

    Google Scholar 

  13. Byun, M., Badler, N.: FacEMOTE: Qualitative parametric modifiers for facial animations. In: Symposium on Computer Animation, San Antonio, TX (2002)

    Google Scholar 

  14. Kopp, S., Sowa, T., Wachsmuth, I.: Imitation games with an artificial agent: From mimicking to understanding shape-related iconic gestures. In: Gesture Workshop, pp. 436–447 (2003)

    Google Scholar 

  15. Rapantzikos, K., Avrithis, Y.: An enhanced spatiotemporal visual attention model for sports video analysis. In: International Workshop on content-based Multimedia indexing (CBMI), Riga, Latvia (2005)

    Google Scholar 

  16. Pelachaud, C., Bilvi, M.: Computational model of believable conversational agents. In: Huget, M.-P. (ed.) Communication in Multiagent Systems. LNCS (LNAI), vol. 2650, pp. 300–317. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  17. Hartmann, B., Mancini, M., Pelachaud, C.: Implementing expressive gesture synthesis for embodied conversational agents. In: Gesture Workshop, Vannes (2005)

    Google Scholar 

  18. Johansson, G.: Visual perception of biological motion adn a model for its analysis. Perception and Psychophysics 14, 201–211 (1973)

    Article  Google Scholar 

  19. Wallbott, H.G., Scherer, K.R.: Cues and channels in emotion recognition. Journal of Personality and Social Psychology 51, 690–699 (1986)

    Article  Google Scholar 

  20. Gallaher, P.E.: Individual differences in nonverbal behavior: Dimensions of style. Journal of Personality and Social Psychology 63, 133–145 (1992)

    Article  Google Scholar 

  21. Ball, G., Breese, J.: Emotion and personality in a conversational agent. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Characters. MITpress, Cambridge (2000)

    Google Scholar 

  22. Pollick, F.E.: The features people use to recognize human movement style. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 10–19. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  23. Hartmann, B., Mancini, M., Buisine, S., Pelachaud, C.: Design and evaluation of expressive gesture synthesis for embodied conversational agents. In: Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Utretch (2005)

    Google Scholar 

  24. Tekalp, A., Ostermann, J.: Face and 2-d mesh animation in MPEG-4. Signal Processing: Image Communication 15, 387–421 (2000)

    Article  Google Scholar 

  25. Raouzaiou, A., Tsapatsoulis, N., Karpouzis, K., Kollias, S.: Parameterized facial expression synthesis based on mpeg-4. EURASIP Journal on Applied Signal Processing 2002, 1021–1038 (2002)

    Article  MATH  Google Scholar 

  26. Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)

    Article  Google Scholar 

  27. Harrigan, J.A.: Listener’s body movements and speaking turns. Communication Research 12, 233–250 (1985)

    Article  Google Scholar 

  28. Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis. In: Computer Animation 2002, Geneva, Switzerland. IEEE Computer Society Press, Los Alamitos (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bevacqua, E. et al. (2006). Multimodal Sensing, Interpretation and Copying of Movements by a Virtual Agent. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Weber, M. (eds) Perception and Interactive Technologies. PIT 2006. Lecture Notes in Computer Science(), vol 4021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11768029_16

Download citation

  • DOI: https://doi.org/10.1007/11768029_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34743-9

  • Online ISBN: 978-3-540-34744-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics