• University: 1Institute for Neural Computation, University of California, San Diego 2Intelligent Robotics and Communication Laboratory, ATR, Kyoto Japan.
  • Authors: G.C. Littlewort1, M.S. Bartlett1, I.R. Fasel1,2, J. Chenu1,2, T. Kanda1,2, H. Ishiguro1,2, and J.R. Movellan1,2


Computer animated agents and robots bring a social dimension to human computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a time scale of less than a second. In this paper we present progress on a perceptual primitive to automatically detect frontal faces in the video stream and code them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder employs a cascade of feature detectors trained with boosting techniques [13, 2]. The expression recognizer employs a combination of AdaBoost and SVM’s. The generalization performance to new subjects for a 7-way forced choice was over 90% correct on two publicly available datasets. The outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system was deployed and evaluated for measuring spontaneous facial expressions in the field in an application for automatic assessment of human-robot interaction.

Download Publication