Abstract
This paper addresses the static and dynamic recognition of basic facial expressions. It has two main contributions. First, we introduce a view- and texture-independent scheme that exploits facial action parameters estimated by an appearance-based 3D face tracker. We represent the learned facial actions associated with different facial expressions by time series. Second, we compare this dynamic scheme with a static one based on analyzing individual snapshots and show that the former performs better than the latter. We provide evaluations of performance using three subspace learning techniques: linear discriminant analysis, non-parametric discriminant analysis and support vector machines.
Similar content being viewed by others
Notes
Several parameters relative to K-NN and SVM have been tested, but only the best results are shown.
References
Abboud, B., Davoine, F.: Facial expression recognition and synthesis based on appearance model. Signal Process. Image Commun. 19(8), 723–740 (2004)
Ahlberg, J.: Model-based coding: extraction, coding and evaluation of face model parameters. Ph.D. Thesis, Department of Electrical Engineering, Linköping University, Sweden (2002)
Ambadar, Z., Schooler, J., Cohn, J.: Deciphering the enigmatic face: the importance of facial dynamics to interpreting subtle facial expressions. Psychol. Sci. 16(5), 403–410 (2005)
Bartlett, M., Littlewort, G., Lainscsek, C., Fasel, I., Movellan, J.: Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, vol. I, pp. 592–597. The Hague, The Netherlands (2004)
Black, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Comp. Vis. 25(1), 23–48 (1997)
Breazeal, C.: Robot in society: friend or appliance? In: Proceedings of Workshop on Emotion-Based Agent Architectures. Seattle (1999)
Breazeal, C.: Sociable machines: expressive social exchange between humans and robots. PhD Thesis, Department of Electrical Engineering and Computer Science, MIT, Cambridge, USA (2000)
Cañamero, L., Gaussier, P.: Emotion understanding: robots as tools and models. In: Nadel, J., Muir, D. (eds.) Emotional Development: Recent Research Advances, pp. 235–258. Oxford Univesity Press, Oxford (2005)
Cristianini, N., Shawe-Taylor, J.: Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Dornaika, F., Davoine, F.: View and texture-independent facial expression recognition in videos using dynamic programming. In: Proceedings of IEEE International Conference on Image Processing, Genova, Italy (2005)
Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Trans. Circuits Syst. Video Technol. 16(9), 1107–1124 (2006)
Dornaika, F., Raducanu, B.: Inferring facial expressions from videos: tool and application. Signal Process. Image Commun. 22(9), 769–784 (2007)
Ekman, P.: Facial expressions of emotions: an old controversy and new findings. Philos. Trans. Royal Soc. Lond. B335, 63–69 (1992)
Ekman, P.: Facial expression and emotion. Am. Psychol. 48(4), 384–392 (1993)
Ekman, P., Davidson, R.: The Nature of Emotion: Fundamental Questions. Oxford University Press, New York (1994)
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, Boston (1990)
Huang, X., Zhao, G., Pietikäinen, M., Zheng, W.: Expression recognition in videos using a weighted component-based feature descriptor. In: Heyden, A., Kahl, F. (eds.) Image Analysis. Springer, Berlin (2011)
Kanade, T., Cohn, J., Tian, Y.L.: Comprehensive database for facial expression analysis. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53. Grenoble, France (2000)
Kim, Y., Lee, S., Kim, S., Park, G.: A fully automatic system recognizing human facial expressions. In: Negoita, M., Howlett, R.J., Jain, L.C. (eds.) Knowledge-Based Intelligent Information and Engineering Systems. Springer, Berlin (2004)
Meng, H., Romera-Paredes, B., Bianchi-Berthouze, N.: Emotion recognition by two view SVM_2K classifier on dynamic facial expression features. In: Proceedings of IEEE International Conference on Face and Gesture Recognition—Workshop on Facial Expression Recognition and Analysis Challenge, Santa Barbara, 25 March 2011
Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Comp. Vis. Image Underst. 115, 541–558 (2011)
Moore, S., Ong, E., Bowden, R.: Facial expression recognition using spatiotemporal boosted discriminator classifiers. In: Campilho A., Kamel, M. (eds.) International Conference on Image Analysis and Recognition, Springer, Berlin (2010)
Müller, M.: Information Retrieval for Music and Motion. Springer, Berlin (2007)
Pantic, M.: Affective Computing. In: Pagani, M. (ed.) Encyclopedia of Multimedia Technology and Networking vol. I, pp. 8–14. Idea Group Publishing, USA (2005)
Picard, R.W., Vyzas, E., Healy, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)
Robin, T., Bierlairey, M., Cruz, J.: Dynamic facial expression recognition with a discrete choice model. J. Choice Model. 2(1), 95–148 (2011)
Shan, C., Gong, S., McOwan, P.W.: Robust facial expression recognition using local binary patterns. In: Proceedings of International Conference on Image Processing, vol. II, pp. 370–373. Genoa, Italy (2005)
Shan, C., Gong, S., McOwan, P.W.: Dynamic facial expression recognition using a bayesian temporal manifold model. In: Proceedings of British Machine Vision Conference, vol. I, pp. 297–306. Edinburgh, UK (2006)
Sung, J., Lee, S., Kim D.: A real-time facial expression recognition using the STAAM. In: Proceedings of International Conference on Pattern Recognition, vol. I, pp. 275–278. Hong Kong, PR China (2006)
Tian, Y.: Evaluation of face resolution for expression analysis. In: Proceedings of IEEE Workshop on Face Processing in Video. Washington, DC, USA (2004)
Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23, 97–115 (2001)
Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Scherer, K.: The first facial expression recognition and analysis challenge. In: Proceedings of IEEE International Conference on Face and Gesture Recognition—Workshop on Facial Expression Recognition and Analysis Challenge, Santa Barbara, 25 March 2011
Wu, T., Bartlett, M.S., Movellan, J.R.: Facial expression recognition using gabor motion energy filters. In: Computer Vision and Pattern Recognition Workshops (CVPRW). San Francisco, USA (2010)
Xiang, T., Leung, M.K.H., Cho, S.Y.: Expression recognition using fuzzy spatio-temporal modeling. Pattern Recogn. 41(1), 204–216 (2008)
Yeasin, M., Bullot, B., Sharma, R.: Recognition of facial expressions and measurement of levels of interest from video. IEEE Trans. Multimed. 8(3), 500–508 (2006)
Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 699–714 (2005)
Zhang, S., Zhao, X., Lei, B.: Robust facial expression recognition via compressive sensing. Sensors 12, 3747–3761 (2012)
Acknowledgments
B. Raducanu is supported by the projects TIN2009-14404-C02 and CONSOLIDER-INGENIO 2010 (CSD2007-00018), Ministerio de Ciencia e Innovacion, Spain. This work is supported in part by the Spanish Government under the project TIN2010-18856.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Raducanu, B., Dornaika, F. Texture-independent recognition of facial expressions in image snapshots and videos. Machine Vision and Applications 24, 811–820 (2013). https://doi.org/10.1007/s00138-012-0447-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-012-0447-z