Real-time 2D to 3D Image Conversion Algorithm and VLSI Architecture for Natural Scene

• Published in: Circuits, systems, and signal processing Vol. 41; no. 8; pp. 4455 - 4478
• Main Authors: Hsia, Shih-Chang, Wang, Szu-Hong, Tsai, Ho-Cheng
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.08.2022; Springer Nature B.V
• Subjects: Algorithms; Circuit design; Circuits and Systems; Computer architecture; Conversion; Critical path; Electrical Engineering; Electronics and Microelectronics; Engineering; Instrumentation; Integrated circuits; Real time; Signal,Image and Speech Processing; Stereoscopy; Depth perception; 3D images; VLSI; FPGA; 2D/3D conversion
• ISSN: 0278-081X; 1531-5878
• DOI: 10.1007/s00034-022-01983-y

This paper presents a high-performance 2D-to-3D conversion algorithm technique and its VLSI architecture design for natural scenes. In this study, the depth map was generated based on the horizontal dividing line concept. Also, the dividing line was estimated by obvious color difference on the local region, and the current dividing line was a demarcation from the maximum value to the minimum one for the depth map. A Depth Based Image Rendering technology was employed to generate stereoscopic images according to the image channel of the depth map. Based on the proposed algorithm, a real-time VLSI architecture is presented. Also, for the purpose of cost efficiency, a module-based hardware consisting of a timing schedule control should be designed. The multiplications and divisions of the algorithm can be minimized by circuit design, and so this helps reduce the complexity of this system. Only one frame memory is required to generate real-time 3D images using the depth map. The depth map can be immediately calculated by referring to the location of dividing lines, which can reduce the storage size and I/O bandwidth. The circuit is simulated and verified by one FPGA chip. The critical path is at the multiplex and one flip-flop, and the maximum clock rate can achieve 205 MHz. To be VGA compatible, the stereoscopic RGB pixels can be outputted in parallel per clock to the interface. The overall maximum data rates of the proposed 3D conversion chip can achieve 610 M bytes per second. This can meet the real-time HD requirement.