GUIDED LOAD BALANCING OF GRAPH PROCESSING WORKLOADS ON HETEROGENEOUS CLUSTERS

• Main Authors: John Lizy Kurian, Gerstlauer Andreas, Song Shuang
• Format: Patent
• Language: English
• Published: 25.01.2018
• Subjects: CALCULATING; CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION ANDCOMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION ANDCOMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THIR OWNENERGY USE; COMPUTING; COUNTING; ELECTRIC COMMUNICATION TECHNIQUE; ELECTRIC DIGITAL DATA PROCESSING; ELECTRICITY; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; PHYSICS; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS; TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINSTCLIMATE CHANGE; TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION

A method, system and computer program product for load balancing of graph processing workloads. Synthetic proxy graphs are generated to characterize machines' graph processing speeds in a cluster. Each of the graph applications executing in the cluster is profiled using the synthetic graphs to form profiling sets. These formed profiling sets are run among the machines in the cluster to capture the machines' graph processing speeds. A metric for each of the graph applications is computed from a relative speedup among the machines in the cluster and/or the graph processing speeds. A graph file of a natural graph and a graph application are loaded. A metric out of the computed metrics is selected based on the graph application. The natural graph is then partitioned into multiple chunks which is distributed onto two or more machines in the cluster based on the selected metric and a user selected partitioning algorithm.