Unmanned surface vehicle escape strategy based on hybrid sampling deep Q-network

Objective Aiming at the encirclement tactics adopted by enemy ships, this study focuses on the problem of planning an escape strategy when an unmanned surface vehicle (USV) is surrounded by enemy ships.

Methods A hybrid sampling deep Q-network (HS-DQN) reinforcement learning algorithm is proposed which gradually increases the playback frequency of important samples and retains a certain level of exploration to prevent it from falling into local optimization. The state space, action space and reward function are designed to obtain the USV's optimal escape strategy, and its performance is compared with that of the deep Q-network (DQN) algorithm in terms of reward and escape success rate.

Results The simulation results show that using the HS-DQN algorithm for training increases the escape success rate by 2% and the convergence speed by 20%.

Conclusions The HS-DQN algorithm can reduce the number of useless explorations and speed up the convergence of the algorithm. The simulation results verify the effectiveness of the USV escape strategy.