Deep Neural Network-Based Flood Monitoring System Fusing RGB and LWIR Cameras for Embedded IoT Edge Devices

• Published in: Remote sensing (Basel, Switzerland) Vol. 16; no. 13; p. 2358
• Main Authors: Lee, Youn Joo, Hwang, Jun Young, Park, Jiwon, Jung, Ho Gi, Suhr, Jae Kyu
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.07.2024
• Subjects: Algorithms; Artificial intelligence; Artificial neural networks; Cameras; Damage detection; Damage prevention; data collection; Edge computing; Environmental risk; Flood damage; Flood forecasting; flood monitoring; Flood predictions; Floods; human resources; Image acquisition; Image processing; Image segmentation; Information processing; Infrared cameras; Infrared imaging; infrastructure; Internet; Internet of Things; Messages; Methods; Monitoring; multimodal sensor; network slimming; Neural networks; Performance degradation; Photography; Property damage; Real time; risk; Rivers; Semantic segmentation; Sensors; surveillance camera; System on chip; Warning; South Korea
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16132358

Floods are among the most common disasters, causing loss of life and enormous damage to private property and public infrastructure. Monitoring systems that detect and predict floods help respond quickly in the pre-disaster phase to prevent and mitigate flood risk and damages. Thus, this paper presents a deep neural network (DNN)-based real-time flood monitoring system for embedded Internet of Things (IoT) edge devices. The proposed system fuses long-wave infrared (LWIR) and RGB cameras to overcome a critical drawback of conventional RGB camera-based systems: severe performance deterioration at night. This system recognizes areas occupied by water using a DNN-based semantic segmentation network, whose input is a combination of RGB and LWIR images. Flood warning levels are predicted based on the water occupancy ratio calculated by the water segmentation result. The warning information is delivered to authorized personnel via a mobile message service. For real-time edge computing, the heavy semantic segmentation network is simplified by removing unimportant channels while maintaining performance by utilizing the network slimming technique. Experiments were conducted based on the dataset acquired from the sensor module with RGB and LWIR cameras installed in a flood-prone area. The results revealed that the proposed system successfully conducts water segmentation and correctly sends flood warning messages in both daytime and nighttime. Furthermore, all of the algorithms in this system were embedded on an embedded IoT edge device with a Qualcomm QCS610 System on Chip (SoC) and operated in real time.