What makes a smiling face look happy? Visual saliency, distinctiveness, and affect

Abstract: We investigated the relative contribution of (a) perceptual (eyes and mouth visual saliency), (b) conceptual or categorical (eye expression distinctiveness), and (c) affective (rated valence and arousal) factors, and (d) specific morphological facial features (Action Units; AUs), to the recognition of facial happiness. The face stimuli conveyed truly happy expressions with a smiling mouth and happy eyes, or blended expressions with a smile but non-happy eyes (neutral, sad, fearful, disgusted, surprised, or angry). Saliency, distinctiveness, affect, and AUs served as predictors; the probability of judging a face as happy was the criterion. Both for truly happy and for blended expressions, the probability of perceiving happiness increased mainly as a function of positive valence of the facial configuration. In addition, for blended expressions, the probability of being (wrongly) perceived as happy increased as a function of (a) delayed saliency and (b) reduced distinctiveness of the non-happy eyes, and (c) enhanced AU 6 (cheek raiser) or (d) reduced AUs 4, 5, and 9 (brow lowerer, upper lid raiser, and nose wrinkler, respectively). Importantly, the later the eyes become visually salient relative to the smiling mouth, the more likely it is that faces will look happy.