Accelerating OpenVX through Halide and MLIR

• Published in: Journal of signal processing systems Vol. 95; no. 5; pp. 571 - 584
• Main Authors: Wu, Song-Lin, Wang, Xiang-Yu, Peng, Ming-Yi, Liao, Shih-Wei
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2023; Springer Nature B.V
• Subjects: Binary codes; Cameras; Cellular telephones; Circuits and Systems; Computer Imaging; Computer vision; Decoupling; Deep learning; Dialects; Electrical Engineering; Embedded systems; Engineering; Image Processing and Computer Vision; Language; Linear algebra; Mobile phones; Pattern Recognition; Pattern Recognition and Graphics; Personal computers; Scheduling; Signal,Image and Speech Processing; Social media; Subject specialists; Vision; Halide; OpenVX; MLIR; Image Processing
• ISSN: 1939-8018; 1939-8115
• DOI: 10.1007/s11265-022-01826-8

In recent years, as many social media and AI-enabled applications have become increasingly ubiquitous, camera-centric applications have emerged as the most popular category of apps on mobile phones. A programmer can develop a camera application in an hour or less without any knowledge related to this domain by using different API provided by frameworks. This allows for the rapid promotion of this technology. OpenVX is a computer vision framework with vital considerations for performance and portability. This paper proposes a new framework that effectively accelerates OpenVX with Halide and MLIR. Our framework possesses Halide’s properties of decoupling algorithms and also has schedules such as an auto-scheduler. It also has MLIR’s multi-level dialects that structure the operations and the data accesses. To generate more efficient programs, we propose a bridge that can transform the programs written in OpenVX into Halide and then translate from Halide to MLIR. In the process, Our framework attains both Halide’s scheduling and MLIR’s dialect to generate more efficient binary code for execution speed.