Performance analysis and evaluation of Java-based InfiniBand Solutions

• Published in: 2020 19th International Symposium on Parallel and Distributed Computing (ISPDC) pp. 20 - 28
• Main Authors: Ruhland, Fabian, Krakowski, Filip, Schottner, Michael
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.07.2020
• Subjects: Benchmark testing; Hardware; High-speed Networks; InfiniBand; Java; Libraries; Observatories; Remote Direct Memory Access; Sockets; Throughput
• DOI: 10.1109/ISPDC51135.2020.00013

Low-latency network interconnects, such as InfiniBand, are widely used in HPC centers and are becoming available in public cloud offerings, too. For MPI applications accessing InfiniBand is transparent, but many big-data applications are written in Java, which does not provide direct access to InfiniBand networks, but relies on thid-party libraries. In this paper, we present Observatory, a benchmark for evaluating low-level libraries, providing InfiniBand access for Java applications. Observatory can be used for evaluating and comparing socket- and verbs-based libraries regarding throughput and latency. With transparency often traded for performance and vice versa, the benchmark helps developers with studying the pros and cons of each solution and supports them in their decision which solution is more suitable for their existing or new use-case. We also give an overview of existing and maintained InfiniBand libraries for Java and evaluate them with the proposed benchmark.