Intelligent ship route planning via an A∗ search model enhanced double-deep Q-network

• Published in: Ocean engineering Vol. 327; p. 120956
• Main Authors: Chen, Xinqiang, Hu, Ruiyang, Luo, Kai, Wu, Huafeng, Biancardo, Salvatore Antonio, Zheng, Yiwen, Xian, Jiangfeng
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 30.05.2025
• Subjects: Deep reinforcement learning; Double deep Q-network; Ship route planning; Smart ship; Double deep Q-network; Deep reinforcement learning; Smart ship; Ship route planning
• ISSN: 0029-8018
• DOI: 10.1016/j.oceaneng.2025.120956

Ship route planning provides important on-site information for the purpose of enhancing intelligent ship safety. The study proposes a novel ship route planning method by considering weather conditions, carbon emissions, etc. Initially, a convolutional neural network is employed to build a model that simulates the navigation environment, capturing the maritime conditions. Then, an A∗ guiding DDQN (A-DDQN) route planning approach is developed by integrating the global pathfinding capability of the A∗ algorithm with the adaptive learning mechanism of the Double Deep Q-Network (DDQN) algorithm, which enhances navigation efficiency while minimizing randomness and unnecessary course deviations. Based on the designed reward function and the inputted ship state information, the A-DDQN algorithm determines the optimal action strategy for ship navigation. By iteratively executing the optimal action at each step, it generates a route a route from the departure area to the target area. The experimental results indicate that the improved method leads to a 11.93 % reduction in fuel consumption and a 12.16 % decline in carbon emissions, demonstrating superior performance compared to the conventional DDQN algorithm. The research findings can help the maritime community make more reasonable ship routing decisions under varied ship navigation conditions. •The study proposes a novel navigation environment model by considering both of maritime geographical information and meteorological condition.•The study introduces a novel yet global ship route planning method by integrating A∗ algorithm and Double Deep Q-Network.•A novel reward mechanism is introduced to help ship find optimal action strategy during ship route optimizaiton procedure.•The research clearly defines the research problem and employs a dynamic approach based on the STEAM2 model to compute the emission inventory of ship air pollutants.