基于多模态信息融合的四足机器人避障方法
TP242.6; 提出了一种全新的基于多模态信息融合技术的四足机器人避障方法.该方法将机器人的本体传感器信息与外部传感器信息相结合,以提高机器人在复杂环境下的决策能力.具体而言,该方法设计了一种多模态强化学习神经网络模型,使机器人能够从自身的传感数据和外部传感器数据中学习.该模型采用监督和非监督学习技术相结合的方法进行训练,以优化机器人在避障任务中的表现.此外,还创新地引入了Transformer层和注意力机制,使机器人能够有选择地关注相关的传感信息并过滤掉无关信息,提高在未知动态环境中的规划可靠性.该方法在具有不同障碍物、不平坦地形等具有挑战性的模拟环境中进行了评估.实验结果表明,所提出的方...
Saved in:
| Published in | 工程科学学报 Vol. 46; no. 8; pp. 1426 - 1433 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | Chinese |
| Published |
中国科学技术大学先进技术研究院,合肥 230026
01.07.2024
中国电子科学研究院,北京 100049%中国电子科学研究院,北京 100049 |
| Subjects | |
| Online Access | Get full text |
| ISSN | 2095-9389 |
| DOI | 10.13374/j.issn2095-9389.2023.07.01.002 |
Cover
| Abstract | TP242.6; 提出了一种全新的基于多模态信息融合技术的四足机器人避障方法.该方法将机器人的本体传感器信息与外部传感器信息相结合,以提高机器人在复杂环境下的决策能力.具体而言,该方法设计了一种多模态强化学习神经网络模型,使机器人能够从自身的传感数据和外部传感器数据中学习.该模型采用监督和非监督学习技术相结合的方法进行训练,以优化机器人在避障任务中的表现.此外,还创新地引入了Transformer层和注意力机制,使机器人能够有选择地关注相关的传感信息并过滤掉无关信息,提高在未知动态环境中的规划可靠性.该方法在具有不同障碍物、不平坦地形等具有挑战性的模拟环境中进行了评估.实验结果表明,所提出的方法相较于对照组可以显著提高四足机器人的避障成功率.此外,由于引入了注意力机制,所提出的算法在动态未知环境下也具有一定的可靠性,使其在实际应用中更加实用.本文的意义主要在于引入多模态信息融合技术和Transformer层,以提高机器人在避障任务中的表现.通过仿真环境的实验结果显示,该学习策略能够显著改善机器人的运动控制能力,并且多模态Transformer模型进一步增强了其性能使其具备优越的泛化性.此外,进一步的分析和可视化也表明了学习策略利用外部输入进行决策的有效性. |
|---|---|
| AbstractList | TP242.6; 提出了一种全新的基于多模态信息融合技术的四足机器人避障方法.该方法将机器人的本体传感器信息与外部传感器信息相结合,以提高机器人在复杂环境下的决策能力.具体而言,该方法设计了一种多模态强化学习神经网络模型,使机器人能够从自身的传感数据和外部传感器数据中学习.该模型采用监督和非监督学习技术相结合的方法进行训练,以优化机器人在避障任务中的表现.此外,还创新地引入了Transformer层和注意力机制,使机器人能够有选择地关注相关的传感信息并过滤掉无关信息,提高在未知动态环境中的规划可靠性.该方法在具有不同障碍物、不平坦地形等具有挑战性的模拟环境中进行了评估.实验结果表明,所提出的方法相较于对照组可以显著提高四足机器人的避障成功率.此外,由于引入了注意力机制,所提出的算法在动态未知环境下也具有一定的可靠性,使其在实际应用中更加实用.本文的意义主要在于引入多模态信息融合技术和Transformer层,以提高机器人在避障任务中的表现.通过仿真环境的实验结果显示,该学习策略能够显著改善机器人的运动控制能力,并且多模态Transformer模型进一步增强了其性能使其具备优越的泛化性.此外,进一步的分析和可视化也表明了学习策略利用外部输入进行决策的有效性. |
| Abstract_FL | This paper proposes a multimodal information fusion neural network model that integrates visual,radar,and proprioceptive information.The model uses a spatial crossmodal attention mechanism to fuse the information,allowing the robot to focus on the most relevant information for obstacle avoidance.The attention mechanism enables the robot to selectively focus on the most relevant informative sensory inputs,which improves its ability to navigate complex terrain.The proposed method was evaluated using multiple experiments in challenging simulated environments,and the results showed a significant improvement in the obstacle avoidance success rate.The proposed method uses an actor-critic architecture and a proximal policy optimization(PPO)algorithm to train the robot in a simulated environment.The training process aims to reduce the difference between the robot's performance in simulated and real-world environments.To achieve this,we randomly adjust the simulation environment's parameters and add random noise to the robot's sensory inputs.This approach allows the robot to learn a robust planning strategy that can be deployed in real-world environments.The multimodal information fusion neural network model is designed using a transformer-based architecture.The model shares the encoding of three types of tokens and generates features for the robot's proprioceptive,visual,and point cloud inputs.The transformer encoder layers are stacked such that the token information from the three modalities can be fuzed at multiple levels.To balance the information from the three modalities,we first separately collect information for each modality and calculate the average value of all tokens from the same modality to obtain a single feature vector.This multimodal information fusion approach improves the robot's decision-making capabilities in complex environments.The novelty of the proposed method lies in the introduction of a spatial crossmodal attention mechanism that allows the robot to selectively attend to the most informative sensory inputs.This attention mechanism improves the robot's ability to navigate complex terrain and provides a certain degree of reliability for the quadruped robot in dynamic unknown environments.The combination of multimodal information fusion and attention mechanism enables the robot to adapt better to complex environments,thus improving its obstacle avoidance capabilities.Therefore,the proposed method provides a promising approach for improving the obstacle avoidance capabilities of quadruped robots in complex environments.The proposed method is based on the multimodal information fusion neural network model and spatial crossmodal attention mechanism.The experimental results demonstrate the effectiveness of the proposed method in improving the robot's obstacle avoidance success rate.Moreover,the potential applications of the proposed method include search and rescue missions,exploration,and surveillance in complex environments. |
| Author | 庄圆 吕友豪 董琦 贾袁骏 |
| AuthorAffiliation | 中国科学技术大学先进技术研究院,合肥 230026;中国电子科学研究院,北京 100049%中国电子科学研究院,北京 100049 |
| AuthorAffiliation_xml | – name: 中国科学技术大学先进技术研究院,合肥 230026;中国电子科学研究院,北京 100049%中国电子科学研究院,北京 100049 |
| Author_FL | ZHUANG Yuan JIA Yuanjun LYU Youhao DONG Qi |
| Author_FL_xml | – sequence: 1 fullname: LYU Youhao – sequence: 2 fullname: JIA Yuanjun – sequence: 3 fullname: ZHUANG Yuan – sequence: 4 fullname: DONG Qi |
| Author_xml | – sequence: 1 fullname: 吕友豪 – sequence: 2 fullname: 贾袁骏 – sequence: 3 fullname: 庄圆 – sequence: 4 fullname: 董琦 |
| BookMark | eNo9jz1Lw0Ach2-oYK39HOKQ-L-X5O5GKb5BwaV7ucvlpFFS8BA7GnBwUVwUTItScOjmUBDM0i-TxPZbWFCcfsMDz8NvCzXSYRojtIPBx5Rytpf4A-dSAjLwJBXSJ0CoD9wH7AOQBmr-o03Udm6gIcCUY0mgiWT1VpTFQ_We17NpfZOVi2mdfSxf76vHu-_8thqPl5_zelJUL7OyKFbZYpVP6uevev60jTasunBx-29bqHd40Osce93To5POftdzGJj0iI5jrIwgzFATGtDCBsxYrZgVXAjKeBCFEQcMQjEaWqEjZYJQa8MNs0BbaPdXe61Sq9KzfjK8ukzXwb5OzhMzGun1XQYCQNIfeOhjgw |
| ClassificationCodes | TP242.6 |
| ContentType | Journal Article |
| Copyright | Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| Copyright_xml | – notice: Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| DBID | 2B. 4A8 92I 93N PSX TCJ |
| DOI | 10.13374/j.issn2095-9389.2023.07.01.002 |
| DatabaseName | Wanfang Data Journals - Hong Kong WANFANG Data Centre Wanfang Data Journals 万方数据期刊 - 香港版 China Online Journals (COJ) China Online Journals (COJ) |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| DocumentTitle_FL | Obstacle avoidance approach for quadruped robot based on multi-modal information fusion |
| EndPage | 1433 |
| ExternalDocumentID | bjkjdxxb202408009 |
| GrantInformation_xml | – fundername: (网络空间安全态势感知与评估安徽省重点实验室开放课题); (国家自然科学基金) funderid: (网络空间安全态势感知与评估安徽省重点实验室开放课题); (国家自然科学基金) |
| GroupedDBID | -0C -SC -S~ 2B. 2RA 4A8 5VR 92I 92M 93N 9D9 9DC AAITT AFUIB ALMA_UNASSIGNED_HOLDINGS CAJEC CQIGP FA0 GROUPED_DOAJ JUIAU PB1 PB6 PSX Q-- Q-2 R-C RT3 T8S TCJ U1F U5C |
| ID | FETCH-LOGICAL-s1049-2bee1ad824d3d6d0b8f54dfba4f87883475c6c70108a436f8bcad56bbd7d4f03 |
| ISSN | 2095-9389 |
| IngestDate | Thu May 29 04:07:32 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Issue | 8 |
| Keywords | 信息融合 Transformer information fusion 强化学习 quadruped robot path planning 四足机器人 路径规划 reinforcement learning |
| Language | Chinese |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-s1049-2bee1ad824d3d6d0b8f54dfba4f87883475c6c70108a436f8bcad56bbd7d4f03 |
| PageCount | 8 |
| ParticipantIDs | wanfang_journals_bjkjdxxb202408009 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-07-01 |
| PublicationDateYYYYMMDD | 2024-07-01 |
| PublicationDate_xml | – month: 07 year: 2024 text: 2024-07-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | 工程科学学报 |
| PublicationTitle_FL | Chinese Journal of Engineering |
| PublicationYear | 2024 |
| Publisher | 中国科学技术大学先进技术研究院,合肥 230026 中国电子科学研究院,北京 100049%中国电子科学研究院,北京 100049 |
| Publisher_xml | – name: 中国电子科学研究院,北京 100049%中国电子科学研究院,北京 100049 – name: 中国科学技术大学先进技术研究院,合肥 230026 |
| SSID | ssib051371920 ssib023167159 ssj0003313525 ssib022319478 ssib041261352 ssib036435564 |
| Score | 2.396232 |
| Snippet | TP242.6; 提出了一种全新的基于多模态信息融合技术的四足机器人避障方法.该方法将机器人的本体传感器信息与外部传感器信息相结合,以提高机器人在复杂环境下的决策能力.具... |
| SourceID | wanfang |
| SourceType | Aggregation Database |
| StartPage | 1426 |
| Title | 基于多模态信息融合的四足机器人避障方法 |
| URI | https://d.wanfangdata.com.cn/periodical/bjkjdxxb202408009 |
| Volume | 46 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals issn: 2095-9389 databaseCode: DOA dateStart: 20230101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.doaj.org/ omitProxy: true ssIdentifier: ssj0003313525 providerName: Directory of Open Access Journals |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnR1Na9VAMNQK4kUUFb8p4uJBUnez38fkNY8i6KlCbyV5SZQKT7AtlJ4sePCieFGwLZaCh948FATfpX-m79n-C2c2eelCi1ThEeZNZmbnI5udDbuzQfCAMc0ymRVhyXUWClnR0OaShsLA2CFUr9Al7nd--kzNPhdP5uX8xLltb9XSynI-3Vs7dV_J_0QVcBBX3CX7D5FthQICYIgvXCHCcD1TjEkqie2SJCapwKtJERMLYgGjSGxIzBAwFBJGR9MdYxiJuyQ1xKbEzDg5QGNIqpHXCIdJ8Ac0iSIJRy7bcW3BLYvC60YRY1EgCAcA2IEMiYHLIgC89QmX4yQYJSSaxBKbAzkmcQA0zZz-MyRWHgDaxkA8fjQaZa1EwHQdN-gIBsXHJIAA5VIE4sgZDxqDYV1fStJa2iFG-cygSMydMyio4H8ZiUS7irZ-lp0XDOpauyyZOc0YZwOEoXYiut7FCch8g410MTDoSnT9SS6QDKYLB1gMDHrcoo1Rx4si-DQClz2CmR-tSwXUL_uI4pmZvD5OaTwyCeX1QOMNM0w0vGXzty4mcmI45FwLNx5iG20T0-As7krWMvyeeJwJtOsz88VXi8Xqah65Cnhug-z5CIZN6n2zgBc2pJbMevXgIiyr4OXHHNJd6RUjEgzm61y2-bRkXLPxaWmYOnGOt3HZcavsheDh2JLHf7fDbbvrV1n_hZchzl0OLjVTu6m47qdXgom1l1cDO9weHAw-Dr9vjHZ3Rm_XD_Z3Rus_Dr99GH56_3vj3XBz8_Dn3mhrMPy6ezAYHK3vH21sjb78Gu19vhbMddO5zmzYHFcSLjGYZ4dRXpYsK0wkCl6oguamkqKo8kxURhvDhZY91dOUUZMJriqT97JCqjwvdCEqyq8Hk_3X_fJGMFVprCoYsUr1SmGozJXVXESVjnKjK05vBvcbSxeat9HSwomQ3ToL0e3g4nHPuRNMLr9ZKe9Clr2c33OR_gOnQZl_ |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E5%9F%BA%E4%BA%8E%E5%A4%9A%E6%A8%A1%E6%80%81%E4%BF%A1%E6%81%AF%E8%9E%8D%E5%90%88%E7%9A%84%E5%9B%9B%E8%B6%B3%E6%9C%BA%E5%99%A8%E4%BA%BA%E9%81%BF%E9%9A%9C%E6%96%B9%E6%B3%95&rft.jtitle=%E5%B7%A5%E7%A8%8B%E7%A7%91%E5%AD%A6%E5%AD%A6%E6%8A%A5&rft.au=%E5%90%95%E5%8F%8B%E8%B1%AA&rft.au=%E8%B4%BE%E8%A2%81%E9%AA%8F&rft.au=%E5%BA%84%E5%9C%86&rft.au=%E8%91%A3%E7%90%A6&rft.date=2024-07-01&rft.pub=%E4%B8%AD%E5%9B%BD%E7%A7%91%E5%AD%A6%E6%8A%80%E6%9C%AF%E5%A4%A7%E5%AD%A6%E5%85%88%E8%BF%9B%E6%8A%80%E6%9C%AF%E7%A0%94%E7%A9%B6%E9%99%A2%2C%E5%90%88%E8%82%A5+230026&rft.issn=2095-9389&rft.volume=46&rft.issue=8&rft.spage=1426&rft.epage=1433&rft_id=info:doi/10.13374%2Fj.issn2095-9389.2023.07.01.002&rft.externalDocID=bjkjdxxb202408009 |
| thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=http%3A%2F%2Fwww.wanfangdata.com.cn%2Fimages%2FPeriodicalImages%2Fbjkjdxxb%2Fbjkjdxxb.jpg |