A Robotic AI Algorithm for Fusing Generative Large Models in Agriculture Internet of Things

• Published in: IEEE internet of things journal Vol. 12; no. 10; pp. 13511 - 13524
• Main Authors: Hou, Guangyu, Niu, Runxin, Chen, Haihua, Wang, Yancong, Zhu, Zhenqiang, Zhang, Jingyao, Ma, Yike, Li, Tongbin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 15.05.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Agricultural robots; Agriculture; Agriculture Internet of Things (AIoT); Algorithms; Artificial intelligence; Automation; Computational modeling; data fine-tuning production; Data models; Efficiency; Generative artificial intelligence; generative big model (GBM); Internet of Things; Machine learning; Perception; Real time; Real-time systems; recommendation methods. robotics; reinforcement learning method based on human solid feedback (S-RLHF); Robot learning; Robot sensing systems; Robots; Sensors; Servers; Service robots
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2024.3516729

Robot perception difficulties in complex environments seriously affect robot operational efficiency. The application of generative big model (GBM) technology on robots through the Agriculture Internet of Things (AIoT) can solve the difficulty of low-operational efficiency. Therefore, this article proposes an automatic decision-making localization algorithm for robots fused with GBMs in AIoT. First, in the AIoT, the complex agricultural scene information is acquired in real-time by the Realsense D435i equipped on the robot, which is accessed in the GBM through a proprietary network to realize real-time intelligent sensing and control between the environmental information and the robot. Then, the innovative reinforcement learning method based on human solid feedback (S-RLHF) and the automatic generation method of weakly supervised fine-tuning data (WS-FT DAGM) are designed in the generative large model. At the same time, by combining the characteristics of the robot operation, two generative significant model recommendation methods are designed, which solves the problem of the difficulty of the target perception in the complex agricultural scene. Finally, by integrating the AIoT and generative large models, the critical method of real-time analysis of crop shading characteristics by the large model is innovatively proposed to solve the problem of low efficiency in robot operation. In the robot test experiments, the operation efficiency using the fusion of AIoT and generative large model reaches more than 92%, significantly improving the operation efficiency compared to the small model (CNN) method without AIoT and generative large model in the traditional agricultural robot.