A Q-learning-based Algorithm for UAV Path Planning under Obstacle Constraints and Wind Disturbances

Unmanned aerial vehicles (UAVs) require reliable and adaptive path planning algorithms to operate in complex environments. However, accurately finding an optimal collision-free path remains a significant challenge, particularly in the presence of environmental disturbances and limited prior knowledge. Conventional graph search methods rely on complete environment knowledge and are difficult to extend to unknown or dynamic scenarios. In this paper, we formulate grid-based UAV path planning in obstacle-rich environments with wind disturbances as a markov decision process (MDP). We propose a model-free Q-learning-based framework that enables the UAV to learn the collision-free shortest paths through interaction with the environment. Our method employs a four-direction action space and a reward design that balances path efficiency and obstacle avoidance under wind disturbances. Without assuming prior knowledge of the environment dynamics, the UAV incrementally updates its action-value function using temporal-difference (TD) learning. Simulation results demonstrate that the learned policy reduces the average path length by 47.3% and 43.5% relative to the two benchmark methods of Dijkstra's and A*, respectively. In addition, our proposed approach improves the success rate by 63.3% and 53.0% under stochastic wind conditions compared to the two benchmark methods of Dijkstra's and A∗ algorithms, respectively. These results highlight the effectiveness of reinforcement learning (RL) for UAV navigation in uncertain and dynamic environments.