Accelerating Viz Pipelines Using Near-Data Computing: An Early Experience - Zheng, Qing; Atkinson, Brian; Wang, Daoce; Lee, Jason; Patchett, John; Manno, Dominic; Grider, Gary (SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2024)
Traditional scientific visualization pipelines transfer entire data arrays from storage to client nodes for processing into displayable graphics objects. However, this full data transfer is often unnecessary, as many visualization filters operate on only small subsets of data in a data array. With the rise of computational storage, smart NICs, and smart devices enabling offloaded processing, this paper examines a case where a visualization pipeline is divided into pre-filters that run near data and post-filters that execute on the client side. Pre-filters preprocess the data near it on storage nodes, reducing data volumes before transfer based on downstream pipeline needs, while post-filters complete the processing on the client node. Experiments done on two real-world simulation datasets demonstrate that this approach can significantly reduce network transfer volumes, cutting visualization pipeline data load times by up to 2.8× compared to traditional methods, and up to 11.9× when combined with data compression techniques.