Action unit detection with segment-based SVMs
Tomas Simon, Minh Hoai Nguyen, Fernando De la Torre et al. · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2010
Automatic facial action unit (AU) detection from video is a long-standing problem in computer vision. Two main approaches have been pursued: (1) static modeling - typically posed as a discriminative classification problem in which each video frame is evaluated independently; (2) temporal modeling - frames are segmented into sequences and typically modeled with a variant of dynamic Bayesian networks. We propose a segment-based approach, kSeg-SVM, that incorporates benefits of both approaches and avoids their limitations. kSeg-SVM is a temporal extension of the spatial bag-of-words. kSeg-SVM is trained within a structured output SVM framework that formulates AU detection as a problem of detecting temporal events in a time series of visual features. Each segment is modeled by a variant of the BoW representation with soft assignment of the words based on similarity. Our framework has several benefits for AU detection: (1) both dependencies between features and the length of action units are modeled; (2) all possible segments of the video may be used for training; and (3) no assumptions are required about the underlying structure of the action unit events (e.g., i.i.d.). Our algorithm finds the best k-or-fewer segments that maximize the SVM score. Experimental results suggest that the proposed method outperforms state-of-the-art static methods for AU detection.
Cited by 81 works. doi:10.1109/cvpr.2010.5539998
Authors
References
- Regression Shrinkage and Selection Via the Lasso
- Object recognition from local scale-invariant features
- Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
- Video Google: a text retrieval approach to object matching in videos
- Active appearance models
- Max-Margin Markov Networks
- Large Margin Methods for Structured and Interdependent Output Variables
- Active Appearance Models Revisited
- A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
- Automatic analysis of facial expressions: the state of the art