Abstract: Facial pain expression is an important modality for assessing pain, especially when a patient’s verbal ability to communicate is impaired. A set of eight facial muscle-based action units (AUs), which are defined by the Facial Action Coding System (FACS), have been widely studied and are highly reliable means for detecting pain through facial expressions. Unfortunately, using FACS is a very time consuming task that makes its clinical use prohibitive. An automated facial expression recognition system (AFER) reliably detecting pain-related AUs would be highly beneficial for efficient and practical pain monitoring. On the other hand, automated pain detection under clinical settings is based on spontaneous facial expressions with limited knowledge about ground truth and can be viewed as a weakly supervised problem, which limits the application of general AFER system that trained on well labeled data. Existing pain oriented AFER research either focus on the individual pain-related AU recognition or bypassing the AU detection procedure by training a binary pain classifier from pain intensity data. In this paper, we decouple pain detection into two consecutive tasks: the AFER based AU labeling at video frame level and a probabilistic measure of pain at sequence level from AU combination scores, which naturally imitates the strategies of human coders in clinical settings. Our work is distinguished in the following aspects, 1) State of the art AFER tools Emotient is applied on pain oriented data sets for single AU labeling. 2) Two different data structures are proposed to encode AU combinations from single AU scores, which forms low-dimensional feature vectors for the learning framework. 3) Two weakly supervised learning frameworks namely multiple instance learning (MIL)[22] and multiple clustered instance learning (MCIL)[24] are employed corresponding to each feature structure to learn pain from video sequences. The results shows a 87% pain recognition accuracy with 0:94 AUC (Area Under Curve) on UNBC-McMaster Shoulder Pain Expression dataset. Tests on long videos in Wilkie’s lung cancer patient video dataset suggests the potential value of the proposed system for pain monitoring task under clinical settings.
Keywords:
- FACS
- Action Unit Combinations
- Pain
- MIL
