SIM: A Scenario IMagination Based Deep Reinforcement Learning Method for Outdoor Transportation Environment Exploration

Autonomous exploration is very important for robotics, especially for mapping, navigation, and planning in an unknown environment. In recent years, automatic exploration methods in the indoor environment have been extensively studied, but there is little research on exploration in the outdoor transportation environment. Due to the limitations of outdoor traffic rules and the scale of the environment, the methods for the indoor environment are difficult to apply to the transportation environment. Aiming at exploration in the transportation environment, this paper proposes a deep reinforcement learning algorithm based on Scenario IMagination(SIM), which has two important components: 1) a mid-level action space, which combines the classical robot control algorithm, addressing the inefficient learning and unstable navigation of deep reinforcement learning algorithms in automatic exploration. With this action space, the deep reinforcement learning algorithm achieves excellent exploration performance in both normal scale environments and large-scale branchless environments; 2) a scenarios buffer, which relieves hard exploration problems of deep reinforcement learning due to serious imbalances of samples in large-scale multibranch scenarios. Compared to the map-less navigation approaches, SIM achieves excellent exploration performance in large-scale multi-branch environments.