Efficient Graph Processing with Invalid Update Filtration

• Published in: IEEE transactions on big data Vol. 7; no. 3; pp. 590 - 602
• Main Authors: Zheng, Long, Li, Xianliang, Ge, Xi, Liao, Xiaofei, Shao, Zhiyuan, Jin, Hai, Hua, Qiang-Sheng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Apexes; Big Data; Computational modeling; Convergence; Degradation; Filtration; Graph processing; Graph theory; invalid update; Iterative methods; Performance degradation; Performance enhancement; Prediction algorithms; Programming; pull computation model; Schedules; Visibility
• ISSN: 2332-7790; 2372-2096
• DOI: 10.1109/TBDATA.2019.2921358

Most of existing graph processing systems essentially follow pull-based computation model to handle compute-intensive parts of graph iteration for high parallelism. Considering all vertices and edges are processed in each iteration, pull model may suffers from a large number of invalid (vertex/edge) operations that do not contribute to graph convergence, leading to potential performance degradation. In this paper, we have the insight that these invalid operations can be filtered by leveraging a small fraction of critical information. However, most of critical information are often beyond the visibility of active vertices being processed. We present two novel filtration approaches to (cooperatively) identify out-of-visibility critical information with boundary-cut heuristics and speculative prediction for many graph algorithms. We have integrated both approaches and their hybrid solution into three state-of-art graph processing systems (including Ligra, Gemini, and Polymer). Experimental results using a wide variety of graph algorithms on both real-world and synthetic graph datasets show that neither of these approaches can have an absolute win for all graph algorithms. Boundary-cut, predictive, and hybrid approaches can improve the performance by 115.1, 38.1, and 136.6 percent on average.