International Conference on Auditory-Visual Speech Processing 2008

Tangalooma Wild Dolphin Resort, Moreton Island, Queensland, Australia
September 26-29, 2008

Improving Pain Recognition Through Better Utilisation of Temporal Information

Patrick Lucey (1), Jessica Howlett (1), Jeffrey F. Cohn (2), Simon Lucey (2), Sridha Sridharan (1), Zara Ambadar (3)

(1) Speech, Audio, Image and Video Technology Laboratory, Queensland University of Technology, Brisbane, QLD, Australia
(2) Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
(3) Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA.

Automatically recognizing pain from video is a very useful application as it has the potential to alert carers to patients that are in discomfort who would otherwise not be able to communicate such emotion (i.e young children, patients in postoperative care etc.). In previous work [1], a "pain-no pain" system was developed which used an AAM-SVM approach to good effect. However, as with any task involving a large amount of video data, there are memory constraints that need to be adhered to and in the previous work this was compressing the temporal signal using K-means clustering in the training phase. In visual speech recognition, it is well known that the dynamics of the signal play a vital role in recognition. As pain recognition is very similar to the task of visual speech recognition (i.e. recognising visual facial actions), it is our belief that compressing the temporal signal reduces the likelihood of accurately recognising pain. In this paper, we show that by compressing the spatial signal instead of the temporal signal, we achieve better pain recognition. Our results show the importance of the temporal signal in recognizing pain, however, we do highlight some problems associated with doing this due to the randomness of a patientís facial actions.


  1. Ashraf, A., Lucey, S., Cohn, J., Chen, T., Ambadar, Z., Prkachin, K., Solomon, P., & Theobald, B., "The painful face: Pain expression using active appearance models", In Proceedings of the International Conference on Multimodal Interfaces - ICMI, 9-14, 2007

Full Paper

Bibliographic reference.  Lucey, Patrick / Howlett, Jessica / Cohn, Jeffrey F. / Lucey, Simon / Sridharan, Sridha / Ambadar, Zara (2008): "Improving pain recognition through better utilisation of temporal information", In AVSP-2008, 167-172.