The Facial Action Coding System (FACS) refers to a set of facial muscle movements that correspond to a displayed emotion. Originally created by Carl-Herman Hjortsjö with 23 facial motion units in 1970, it was subsequently developed further by Paul Ekman, and Wallace Friesen. The FACS as we know it today was first published in 1978, but was substantially updated in 2002.

Using FACS, we are able to determine the displayed emotion of a participant. This is currently the only available technique for assessing emotions in real-time. Other measures, such as interviews and psychometric tests, must be completed after a stimulus has been presented. This delay ultimately adds another barrier to measuring how a participant truly feels in direct response to a stimulus.

For a long time, and still in some cases today, researchers have been limited to manually coding video recordings of participants according to the action units described by the FACS. This is now much more likely to be carried out automatically, using software such as iMotions. This saves vast amounts of time and money, as scoring no longer requires analysis of each frame by a trained researcher – the software simply does the work for you.

Below we have listed the major action units that are used to determine emotions. Roll your mouse over the image to start the movement! Below the action units we have also listed the combinations of action units that are determined as a specific emotion.

Main Action Units

Action Unit Description Facial Muscle Example (Hover to Play)
 1 Inner Brow Raiser Frontalis, pars medialis AU1 FACS
 2 Outer Brow Raiser (unilateral, right side) Frontalis, pars lateralis AU2 right only FACS
 4 Brow Lowerer Relaxation of Levator palpebrae superioris; Orbicularis oculi, pars palpebralis au4 brow lowerer
 5 Upper Lid Raiser Levator palpebrae superioris AU5 FACS
 6 Cheek Raiser Orbicularis oculi, pars orbitalis AU6 cheek raiser
 7 Lid Tightener Orbicularis oculi, pars palpebralis AU7 lid tightener
 9 (also shows slight AU4 and AU10) Nose Wrinkler  Levator labii superioris alaquae nasi AU9 with 4+10
 10 (also shows slight AU25) Upper Lip Raiser Levator labii superioris  AU10 with 25
 11 Nasolabial Deepener Zygomaticus minor  AU11 - nasolabial deepener
 12 Lip Corner Puller Zygomaticus major AU12
 13 Cheek Puffer Levator anguli oris (Caninus) AU13 cheek puffer
 14 Dimpler Buccinator AU14 dimpler
 15 Lip Corner Depressor Depressor anguli oris (Triangularis) AU15 FACS
 16 (with AU25) Lower Lip Depressor Depressor labii inferioris AU16 with 25
 17 Chin Raiser Mentalis AU17 FACS guide
 18 (with slight AU22 and AU25) Lip Puckerer Incisivii labii superioris and Incisivii labii inferioris AU18 with 22A and 25A
 20 Lip stretcher Risorius  AU20 lip stretcher
 22 (with AU25) Lip Funneler Orbicularis oris AU22 with 25 FACS
 23 Lip Tightener Orbicularis oris AU23 lip tightener
 24 Lip Pressor Orbicularis oris AU24 image FACS guide
 25 Lips part Depressor labii inferioris, or Mentalis, or Orbicularis oris AU25 lips part
 26 (with AU25) Jaw Drop Masseter, Temporalis and internal Pterygoid AU26 with 25 FACS affectiva
 27 Mouth Stretch Pterygoids, Digastric AU27 mouth stretcher
 28 (with AU26) Lip Suck Orbicularis oris AU28 with 26 FACS affectiva
 41 Lid droop Levator palpebrae superioris AU41 lid droop
42 Slit Orbicularis oculi AU42 slit
 43 Eyes Closed Levator palpebrae superioris; Orbicularis oculi, pars palpebralis AU43 eyes closed
 44 Squint Orbicularis oculi, pars palpebralis  AU44 squint
 45 Blink Levator palpebrae superioris; Orbicularis oculi, pars palpebralis  AU45 blink
 46 Wink Levator palpebrae superioris; Orbicularis oculi, pars palpebralis AU46 wink

 

Head Movement Action Units

Action Unit Description Example (Hover to Play)
 51 Head Turn Left AU51 head turn left
 52 Head Turn Right AU52 head turn right
 53 Head Up AU53 head up
 54 Head Down AU54 head down
 55 Head Tilt Left AU55 head tilt left
 56 Head Tilt Right AU56 head tilt right
 57 Head Forward AU57 head forward
 58 Head Back  AU58 head back

 

Eye Movement Action Units

Action Unit Description Example (Hover to Play)
 51 Eyes Turn Left AU61-eyes-turn-left
 51 Eyes Turn Right AU62 eyes right
 51 Eyes Up AU63 eyes up
 51 Eyes Down AU64 eyes down

Emotions and Action Units

The Action Units described above show the different movements of facial muscles. Certain combined movements of these facial muscles pertain to a displayed emotion. Emotion recognition is completed in iMotions using Affectiva or Emotient, which uses the collection of certain action units to provide information about which emotion is being displayed. For example, happiness is calculated from the combination of action unit 6 (cheek raiser) and 12 (lip corner puller). A complete list of these combinations and the emotion that they relate to is shown below. The gifs on the right are shown in the same order that the action units listed.

Emotion Action Units Description Examples (Hover to Play)
Happiness / Joy 6 + 12 Cheek Raiser, Lip Corner Puller AU6 cheek raiser

AU12 lip corner puller

Sadness 1 + 4 + 15 Inner Brow Raiser, Brow Lowerer, Lip Corner Depressor AU1 inner brow raiser

au4 brow lowerer

AU15 lip corner depressor

Surprise 1 + 2 + 5 + 26 Inner Brow Raiser, Outer Brow Raiser, Upper Lid Raiser, Jaw Drop AU1 inner brow raiser

au2 outer brow raiser

au5 upper lid raiser

AU26 jaw drop

Fear 1 + 2 + 4 + 5 + 7 + 20 + 26 Inner Brow Raiser, Outer Brow Raiser, Brow Lowerer, Upper Lid Raiser, Lid Tightener, Lip Stretcher, Jaw Drop AU1 inner brow raiser

au2 outer brow raiser

au4 brow lowerer

au5 upper lid raiser

AU7 lid tightener

AU20 lip stretcher

AU26 jaw drop

Anger 4 + 5 + 7 + 23 Brow Lowerer, Upper Lid Raiser, Lid Tightener, Lip Tightener au4 brow lowerer

au5 upper lid raiser

AU7 lid tightener

AU23 lip tightener

Disgust 9 + 15 + 16 Nose Wrinkler, Lip Corner Depressor, Lower Lip Depressor AU9 nose wrinkler

AU15 lip corner depressor

AU16 lower lip depressor

Contempt 12 + 14 (on one side of the face) Lip Corner Puller, Dimpler AU12 lip corner puller

AU14 dimpler

 

Putting it all together

When measuring facial expressions within iMotions, the stimuli are paired automatically to the FACS analysis, allowing you to pinpoint the exact moment that the stimulus triggered a certain emotion. The FACS is also graded on a scale of intensity, which gives a measure of how strongly the emotion is displayed. These measurements can also be synchronized with recordings of galvanic skin response, which provides a measure of arousal. With this information combined, it’s possible to start drawing conclusions about how strongly an individual felt, and what those emotions consisted of, in response to a set stimulus.

The screenshot below shows how the facial expression data is displayed while a participant watches an advertisement.

 imotions action units affdex

If we zoom in, we can see the intensity of the displayed emotion. There are five emotions displayed in the image below, however iMotions provides a measure of the seven central emotions (shown in the table above), alongside, and in conjunction with measurements of action units.

 imotions action units facs

I hope this explanation of action units and FACS has been helpful, and informative. If you’d like to learn more about facial expressions, then we also have a free pocket guide that you can download below, that has even more information available!

 

“facial