Neutrino: Efficient InfiniBand Access for Java Applications

• Published in: 2020 19th International Symposium on Parallel and Distributed Computing (ISPDC) pp. 12 - 19
• Main Authors: Krakowski, Filip, Ruhland, Fabian, Schottner, Michael
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.07.2020
• Subjects: Hardware; InfiniBand; Java; Java Native Interface; Layout; Libraries; Message systems; Metadata; Neutrino sources; Remote Direct Memory Access
• DOI: 10.1109/ISPDC51135.2020.00012

Fast networks like InfiniBand are important for large-scale applications and big data analytics. Current InfiniBand hardware offers bandwidths of up to 200 Gbit/s with latencies of less than two microseconds. While it is mainly used in high performance computing, there are also some applications in the field of big data analytics. In addition, some cloud providers are offering instances equipped with InfiniBand hardware. Many big data applications and frameworks are written using the Java programming language, but the Java Development Kit does not provide native support for InfiniBand. To this end we propose neutrino, a network library providing comfortable and efficient access to InfiniBand hardware in Java as well as epoll based multithreaded connection management. Neutrino supports InfiniBand message passing as well as remote direct memory access, is implemented using the Java Native Interface, and can be used with any Java Virtual Machine. It also provides access to native C structures via a specially developed proxy system, which in turn enables the developer to leverage the InfiniBand hardware's full functionality. Our experiments show that efficient access to InfiniBand hardware from within a Java Virtual Machine is possible while fully utilizing the available bandwidth.