Video Segmentation by Tracking Many Figure-Ground Segments

• Published in: 2013 IEEE International Conference on Computer Vision pp. 2192 - 2199
• Main Authors: Fuxin Li, Taeyoung Kim, Humayun, Ahmad, Tsai, David, Rehg, James M.
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.12.2013
• Subjects: Algorithms; appearance model; composite statistical inference; Computer vision; CPMC; CSI; Exact solutions; Image segmentation; Mathematical models; Motion segmentation; Online; Predictive models; Proposals; Segmentation; Segments; Target tracking; Tracking; tracking segments; Training; Video Segmentation
• ISSN: 1550-5499
• DOI: 10.1109/ICCV.2013.273

We propose an unsupervised video segmentation approach by simultaneously tracking multiple holistic figure-ground segments. Segment tracks are initialized from a pool of segment proposals generated from a figure-ground segmentation algorithm. Then, online non-local appearance models are trained incrementally for each track using a multi-output regularized least squares formulation. By using the same set of training examples for all segment tracks, a computational trick allows us to track hundreds of segment tracks efficiently, as well as perform optimal online updates in closed-form. Besides, a new composite statistical inference approach is proposed for refining the obtained segment tracks, which breaks down the initial segment proposals and recombines for better ones by utilizing high-order statistic estimates from the appearance model and enforcing temporal consistency. For evaluating the algorithm, a dataset, SegTrack v2, is collected with about 1,000 frames with pixel-level annotations. The proposed framework outperforms state-of-the-art approaches in the dataset, showing its efficiency and robustness to challenges in different video sequences.