Efficient ocular expression analysis for synthetic reproduction

• Published in: IEEE transactions on multimedia Vol. 8; no. 1; pp. 70 - 80
• Main Authors: del Valle, A.C.A., Dugelay, J.-L.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.02.2006; Institute of Electrical and Electronic Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Animation; Applied sciences; Artificial intelligence; Broadcasting. Videocommunications. Audiovisual; Cameras; Cloning; communications systems; Computer science; control theory; systems; Exact sciences and technology; Eyebrows; Facial features; Frames; Illumination; image motion analysis; image processing; Kalman filtering; Lighting; machine vision; Markers; Mathematical models; Multimedia; Pattern recognition. Digital image processing. Computational geometry; Predictive models; real-time systems; Representations; Reproduction; Robustness; Telecommunications; Telecommunications and information theory; teleconferencing; Usability; Video sequences; Videocommunications (networks and services); Computer vision; Motion estimation; Image processing; Teleconference; Kalman filter; communications systems; Computer animation; Real time system; teleconferencing; image motion analysis; machine vision; Eye; Image analysis; Three dimensional model; Image sequence; real-time systems; Animation; Kalman filtering; Motion analysis
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2005.861290

This paper presents an original framework that analyzes the complete ocular (eye + eyebrow) expression on video sequences to reproduce it later on synthetic three-dimensional (3-D) models in real-time. We propose a two step process to develop robust techniques for facial feature analysis aimed at working without special illumination conditions or physical constraints on the user (markers, fixed frontal pose, etc.). First, simple and efficient image-processing methods based on motion models are designed over a frontal point of view of the face. Natural and realistic intra-feature and inter-feature constraints are applied to improve the analysis results. Then, the usability of these algorithms is extended to enable the analysis regardless of the person's pose in front of the camera. This is achieved by redefining the motion models involved over the speaker's highly realistic synthetic representation (clone), by using a suitable observation model, and by predicting the head pose in 3-D, frame by frame.