A New Approach for Semi-External Topological Sorting on Big Graphs

• Published in: IEEE transactions on knowledge and data engineering Vol. 35; no. 12; pp. 1 - 14
• Main Authors: Gao, Tianpeng, Li, Jianzhong, Ma, Hengzhao
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Complexity theory; Costs; DAG; Memory management; Optimization; partial order; Scheduling; semi-external algorithm; Sorting; Sorting algorithms; Synthetic data; topological sorting; Topology; Transforms
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/TKDE.2023.3274528

This paper presents a new approach for semi-external topological sorting algorithm on big directed acyclic graph(DAG). Topological sorting aims to find an ordering of each node in DAG, which satisfies <inline-formula><tex-math notation="LaTeX">u</tex-math></inline-formula> precedes <inline-formula><tex-math notation="LaTeX">v</tex-math></inline-formula> in the ordering for each edge <inline-formula><tex-math notation="LaTeX">(u,v)</tex-math></inline-formula> in DAG. Topological sorting is an important subroutine for scheduling and other external graph algorithms. But, the internal topological sorting algorithm cannot handle big DAGs and the I/O complexity of total external topological sorting is too high for practical applications. Therefore, we pay attention to the semi-external topological sorting for big DAGs in this paper. We find that the existing semi-external topological sorting algorithm is mainly based on constructing a DFS-Tree in internal memory. However, this DFS-based algorithm is natively more difficult than topological sorting, because DFS-Tree determines a strict total order, while topological order is only a partial order. Therefore, a partial order level order is proposed in this paper. Based on the level order , we propose a new semi-external topological sorting algorithm. Next, two optimizations, NodeRemove and EdgeRemove , are proposed to reduce the CPU and I/O cost. In addition, we also propose a batch algorithm. Finally, we perform experimental studies using real and synthetic datasets to confirm the efficiency of our approach. According to the experimental results, our algorithms are better than the previous DFS-based algorithms.