Big Data for Autonomic Intercontinental Overlays

• Published in: IEEE journal on selected areas in communications Vol. 34; no. 3; pp. 575 - 583
• Main Authors: Brun, Olivier, Lan Wang, Gelenbe, Erol
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Big Data; Cognitive Packet Network; Computer Science; Internet; IP networks; Machine Learning; Monitoring; Network QoS; Networking and Internet Architecture; Overlay networks; Quality of service; Random Neural Network; Routing; Routing protocols; Smart Overlays; The Internet; network quality of service (QoS); cognitive packet network; random neural network; The Internet; big data; smart overlays; Network QoS; Smart Overlays; Random Neural Network; Big Data; Cognitive; Packet Network
• ISSN: 0733-8716; 1558-0008; 1558-0008
• DOI: 10.1109/JSAC.2016.2525518

This paper uses big data and machine learning for the real-time management of Internet scale quality-of-service (QoS) route optimisation with an overlay network. Based on the collection of data sampled every 2 min over a large number of source-destinations pairs, we show that intercontinental Internet protocol (IP) paths are far from optimal with respect to QoS metrics such as end-to-end round-trip delay. We, therefore, develop a machine learning-based scheme that exploits large scale data collected from communicating node pairs in a multihop overlay network that uses IP between the overlay nodes, and selects paths that provide substantially better QoS than IP. Inspired from cognitive packet network protocol, it uses random neural networks with reinforcement learning based on the massive data that is collected, to select intermediate overlay hops. The routing scheme is illustrated on a 20-node intercontinental overlay network that collects some 2 × 10 6 measurements per week, and makes scalable distributed routing decisions. Experimental results show that this approach improves QoS significantly and efficiently.