Multiscale video sequence matching for near-duplicate detection and retrieval

• Published in: Multimedia tools and applications Vol. 78; no. 1; pp. 311 - 336
• Main Authors: Yang, Yuanyuan, Tian, Yonghong, Huang, Tiejun
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2019; Springer Nature B.V
• Subjects: Algorithms; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Matching; Multimedia Information Systems; Multiscale analysis; Queries; Reproduction (copying); Retrieval; Segments; Special Purpose and Application-Based Systems; Video compression; Near-duplicate video detection; Multiscale matching; Video sequence matching; Bi-directional scanning; Constrained longest ascending matching subsequence
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-018-5862-3

As one of key technologies in content-based near-duplicate detection and video retrieval, video sequence matching can be used to judge whether two videos exist duplicate or near-duplicate segments or not. Despite a lot of research efforts devoted in recent years, how to precisely and efficiently perform sequence matching among videos (which may be subject to complex audio-visual transformations) from a large-scale database still remains a pretty challenging task. To address this problem, this paper proposes a multiscale video sequence matching (MS-VSM) method, which can gradually detect and locate the similar segments between videos from coarse to fine scales. At the coarse scale, it makes use of the Maximum Weight Matching (MWM) algorithm to rapidly select several candidate reference videos from the database for a given query. Then for each candidate video, its most similar segment with respect to the given query is obtained at the middle scale by the Constrained Longest Ascending Matching Subsequence (CLAMS) algorithm, and then can be used to judge whether that candidate exists near-duplicate or not. If so, the precise locations of the near-duplicate segments in both query and reference videos are determined at the fine scale by using bi-directional scanning to check the matching similarity at the segments’ boundaries. As such, the MS-VSM method can achieve excellent near-duplicate detection accuracy and localization precision with a very high processing efficiency. Extensive experiments show that it outperforms several state-of-the-art methods remarkably on several benchmarks.