MR and ultrasound cardiac image dynamic visualization and synchronization over Internet for distributed heart function diagnosis

• Published in: Computerized medical imaging and graphics Vol. 88; p. 101850
• Main Authors: Zhang, Qi, Samani, Abbas, Peters, Terry M.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.03.2021; Elsevier Science Ltd
• Subjects: Algorithms; Bidirectional Internet connection; Cardiac image fusing; Cardiovascular diseases; Computer programs; Computers; Coronary artery disease; Data visualization; Diagnosis; Digital media; Dual-modality; Dynamic medical visualization; Heart diseases; Heart function; Image acquisition; Internal Medicine; Internet; Magnetic resonance; Magnetic resonance imaging; Medical diagnosis; Medical imaging; Medical treatment; Other; Rendering; Scientific visualization; Sequences; Software; Synchronism; Synchronization; Ultrasonic imaging; Ultrasound; Visualization; Web browsers; Web-based rendering; Dual-modality; Bidirectional Internet connection; Web-based rendering; Synchronization; Dynamic medical visualization; Cardiac image fusing
• ISSN: 0895-6111; 1879-0771; 1879-0771
• DOI: 10.1016/j.compmedimag.2020.101850

•Introduce a new Internet-based algorithm and a software platform running on a Node.js server, where a series of registered cardiac images from both magnetic resonance (MR) and ultrasound are employed to display dynamic fused cardiac structures in web browsers.•Develop a novel approach that allows precise visualization of intracardiac structures within a beating heart,using Node.js framework and Socket.IO library to create real-time bidirectional connections between the server and clients.•Create algorithms for loading and processing a series of cardiac MR and ultrasound images, and outline an ECG-gated dual-modality cardiac data acquisition procedure and an intensity-based image registration algorithm .•When compared with traditional multi-modality image visualization software running on stand-alone computers, our Internet-based solutions can lead to enhanced accessibility and information sharing.•The software framework provides cardiologists with instant information in appreciating the anatomic complexity of beating heart and sharing feedback with collaborators using web browsers during the process of distributed diagnosis of cardiac function. Dual-modality 4D cardiac data visualization can convey a significant amount of complementary image information from various sources into a single and meaningful display. Even though there are existing publications on combining multiple medical images into a unique representation, there has been no work on rendering a series of cardiac image sequences, acquired from multiple sources, using web browsers and synchronizing the result over the Internet in real time. The ability to display multi-modality beating heart images using Web-based technology is hampered by the lack of efficient algorithms for fusing and visualizing constantly updated multi-source images and streaming the rendering results using internet protocols. To address this practical issue, in this paper we introduce a new Internet-based algorithm and a software platform running on a Node.js server, where a series of registered cardiac images from both magnetic resonance (MR) and ultrasound are employed to display dynamic fused cardiac structures in web browsers. Taking advantage of the bidirectional WebSocket protocol and WebGL-based graphics acceleration, internal cardiac structures are dynamically displayed, and the results of rendering and data exploration are synchronized among all the connected client computers. The presented research and software have the potential to provide clinicians with comprehensive information and intuitive feedback relating to cardiac behavior and anatomy and could impact areas such as distributed diagnosis of cardiac function and collaborative treatment planning for various heart diseases.