Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments

• Published in: 2010 IEEE 12th International Conference on High Performance Computing and Communications pp. 72 - 78
• Main Authors: Bohm, S, Engelmann, C, Scott, S L
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2010
• Subjects: Gallium nitride; Measurement; Monitoring; Peer to peer computing; Real time systems; Scalability
• ISBN: 9781424483358; 1424483352
• DOI: 10.1109/HPCC.2010.32

We present a monitoring system for large-scale parallel and distributed computing environments that allows to trade-off accuracy in a tunable fashion to gain scalability without compromising fidelity. The approach relies on classifying each gathered monitoring metric based on individual needs and on aggregating messages containing classes of individual monitoring metrics using a tree-based overlay network. The MRNet-based prototype is able to significantly reduce the amount of gathered and stored monitoring data, e.g., by a factor of ~56 in comparison to the Ganglia distributed monitoring system. A simple scaling study reveals, however, that further efforts are needed in reducing the amount of data to monitor future-generation extreme-scale systems with up to 1,000,000 nodes. The implemented solution did not had a measurable performance impact as the 32-node test system did not produce enough monitoring data to interfere with running applications.