Rethinking Fine-Grained Measurement From Software-Defined Perspective: A Survey

• Published in: IEEE transactions on services computing Vol. 15; no. 6; pp. 3649 - 3667
• Main Authors: Zheng, Hao, Jiang, Yanan, Tian, Chen, Cheng, Long, Huang, Qun, Li, Weichao, Wang, Yi, Huang, Qianyi, Zheng, Jiaqi, Xia, Rui, Dou, Wanchun, Chen, Guihai
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Anomalies; Atmospheric measurements; Computer architecture; Computer memory; Current measurement; Data structures; Data transmission; Diagnosis; Loss measurement; Mice; Monitoring; Network measurement; Particle measurements; sketch; Sketches; Software; Task analysis; Traffic control; Traffic engineering; Traffic speed
• ISSN: 1939-1374; 2372-0204
• DOI: 10.1109/TSC.2021.3103968

Network measurement provides operators an efficient tool for many network management tasks such as performance diagnosis, traffic engineering and intrusion prevention. However, with the rapid and continuous growth of traffic speed, it needs more computing and memory resources to monitor traffic in per-flow or per-packet granularity. Sample-based measurement systems (e.g., NetFlow, sFlow) have been developed to perform coarse-grained measurement, but they may miss part of records, especially for mice flows, which are important for some network management tasks (e.g., anomaly detection, performance diagnosis). To address these issues, data streaming algorithms such as hash tables and sketches have been introduced to balance the trade-off among accuracy, speed, and memory usage. In this article, we present a systematic survey of various data structures, algorithms and systems which have been proposed in recent years to perform fine-grained measurement for high-speed networks. We organize these methods and systems from a software-defined perspective. In particular, we abstract fine-grained network measurement into three-layer architecture. We introduce the responsibility of each layer and categorize existing state-of-the-art works into this architecture. Finally, we conclude the article and discuss the future directions of fine-grained network measurement.