Multi-target USV Patrol via DAPF-Guided Deep Reinforcement Learning

Unmanned surface vehicles (USVs) are widely used in marine resource exploration, ocean data collection, and maritime patrols. However, when USVs perform tasks that require continuous passage through multiple target points in complex obstacle-filled environments, existing path algorithms often face several limitations: reduced autonomous decision-making capabilities, longer algorithm convergence times, and poor path quality. To address these issues, this paper proposes a path planning algorithm based on dynamic artificial potential field – DQN (DAPF-DQN). First, a novel comprehensive reward function is introduced to balance the path quality and patrol efficiency of the USV. Then, a target state representation mechanism based on the APF is established, enabling the USV to make optimal patrol decisions by analyzing the potential field forces in the current environment. Additionally, Bézier curves are used for path smoothing, making the paths more feasible. Finally, the algorithm’s effectiveness is validated through simulation experiments. Simulation results demonstrate that the DAPF-DQN algorithm improves the convergence and path generation quality, enhancing the USV’s decision-making capability.