A Phase‐Based Approach for Animating Images Using Video Examples

• Published in: Computer graphics forum Vol. 36; no. 6; pp. 303 - 311
• Main Authors: Prashnani, Ekta, Noorkami, Maneli, Vaquero, Daniel, Sen, Pradeep
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.09.2017
• Subjects: Brittleness; Candles; I.3.7 [Computer Graphics]: Three‐Dimensional Graphics and Realism—Animation; motion transfer; Object motion; Optical flow (image analysis); Phase transitions; phase‐based motion processing; video‐driven image animation
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.12940

We present a novel approach for animating static images that contain objects that move in a subtle, stochastic fashion (e.g. rippling water, swaying trees, or flickering candles). To do this, our algorithm leverages example videos of similar objects, supplied by the user. Unlike previous approaches which estimate motion fields in the example video to transfer motion into the image, a process which is brittle and produces artefacts, we propose an Eulerian phase‐based approach which uses the phase information from the sample video to animate the static image. As is well known, phase variations in a signal relate naturally to the displacement of the signal via the Fourier Shift Theorem. To enable local and spatially varying motion analysis, we analyse phase changes in a complex steerable pyramid of the example video. These phase changes are then transferred to the corresponding spatial sub‐bands of the input image to animate it. We demonstrate that this simple, phase‐based approach for transferring small motion is more effective at animating still images than methods which rely on optical flow. We present a novel approach for animating static images that contain objects that move in a subtle, stochastic fashion (e.g. rippling water, swaying trees, or flickering candles). To do this, our algorithm leverages example videos of similar objects, supplied by the user. Unlike previous approaches which estimate motion fields in the example video to transfer motion into the image, a process which is brittle and produces artefacts, we propose an Eulerian phase‐based approach which uses the phase information from the sample video to animate the static image. As is well known, phase variations in a signal relate naturally to the displacement of the signal via the Fourier Shift Theorem. To enable local and spatially varying motion analysis, we analyse phase changes in a complex steerable pyramid of the example video.