Recognition and Matching of Clustered Mature Litchi Fruits Using Binocular Charge-Coupled Device (CCD) Color Cameras

• Published in: Sensors (Basel, Switzerland) Vol. 17; no. 11; p. 2564
• Main Authors: Wang, Chenglin, Tang, Yunchao, Zou, Xiangjun, Luo, Lufeng, Chen, Xiong
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 07.11.2017; MDPI
• Subjects: binocular vision; Closed circuit television; Fruits; harvesting robot; Lighting; litchi recognition; Semiconductors; stereo matching; harvesting robot; litchi recognition; stereo matching; binocular vision
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s17112564

Recognition and matching of litchi fruits are critical steps for litchi harvesting robots to successfully grasp litchi. However, due to the randomness of litchi growth, such as clustered growth with uncertain number of fruits and random occlusion by leaves, branches and other fruits, the recognition and matching of the fruit become a challenge. Therefore, this study firstly defined mature litchi fruit as three clustered categories. Then an approach for recognition and matching of clustered mature litchi fruit was developed based on litchi color images acquired by binocular charge-coupled device (CCD) color cameras. The approach mainly included three steps: (1) calibration of binocular color cameras and litchi image acquisition; (2) segmentation of litchi fruits using four kinds of supervised classifiers, and recognition of the pre-defined categories of clustered litchi fruit using a pixel threshold method; and (3) matching the recognized clustered fruit using a geometric center-based matching method. The experimental results showed that the proposed recognition method could be robust against the influences of varying illumination and occlusion conditions, and precisely recognize clustered litchi fruit. In the tested 432 clustered litchi fruits, the highest and lowest average recognition rates were 94.17% and 92.00% under sunny back-lighting and partial occlusion, and sunny front-lighting and non-occlusion conditions, respectively. From 50 pairs of tested images, the highest and lowest matching success rates were 97.37% and 91.96% under sunny back-lighting and non-occlusion, and sunny front-lighting and partial occlusion conditions, respectively.