A method for nighttime tomato fruit detection and occlusion judgment based on deep learning and image processing

• Published in: Proceedings of SPIE, the international society for optical engineering Vol. 13509; pp. 135090Y - 135090Y-6
• Main Authors: Lin, Zhonglong, Zhang, Caihong, Liang, Zhi, Zou, Xiangjun, Li, Xiaojuan
• Format: Conference Proceeding
• Language: English
• Published: SPIE 05.02.2025
• ISBN: 9781510688100; 1510688102
• ISSN: 0277-786X
• DOI: 10.1117/12.3059119

Nighttime detection and harvesting are key issues for achieving all-day operation of tomato-picking robots. Currently, most general detection algorithms are limited to natural daylight conditions, with significantly reduced performance in nighttime environments. To address the issues of low accuracy and poor robustness of nighttime tomato detection algorithms, a high-precision nighttime tomato detection method based on the integration of deep learning and image processing is proposed. This study designed multiple sets of nighttime RGB lighting experiments to calculate the HSV color distance between ripe tomatoes and the background under each lighting condition, determining the optimal lighting color to enhance the contrast between tomatoes and the background. Under the optimal lighting conditions, an RGB image dataset of nighttime tomatoes was constructed, and an YOLOv8-based nighttime detection model was trained to achieve precise detection and localization of nighttime tomato targets. Within the detected target frames of ripe tomatoes, image processing methods such as OTSU, Hough detection, and connected component analysis were used to judge and analyze the occlusion situation of tomatoes, distinguishing between occlusion types (leaves or branches), and providing guidance for optimizing the robot's picking strategy. Finally, this study verifies the effectiveness of the algorithm through multiple sets of experiments. The algorithm has an overall accuracy rate of 84% and can be deployed on edge devices to achieve efficient real-time detection tasks while ensuring performance.