Objectives Aiming at the intelligent escape problem of marine target unmanned surface vehicles (USVs), this paper proposes an enhanced cooperative target hunting strategy. This strategy is designed to improve the performance of multi-USV systems in capturing escaping targets through a combination of advanced learning and optimization techniques.
Methods Firstly, a Markov game process for USV hunting and escaping scenarios is established, with success criteria defined using distance and angle metrics. To enhance the hunting performance against intelligent escapes, a training framework is developed using the centralized training - decentralized execution paradigm and long short-term memory (LSTM) networks. This is integrated with the multi-agent soft actor-critic (MASAC) algorithm for cooperative hunting training. Additionally, a stage-induced cooperative hunting reward method is introduced. This method optimizes the training process based on the current states of both the hunter and the target, guiding the USVs to achieve the hunting task progressively from easier to more challenging stages. It also mitigates the issue of "inertia hunting vehicles" and increases the hunting success rate.
Results Simulation results, particularly in a 3-USV versus 1-target scenario, validate the feasibility and effectiveness of the proposed strategy. Compared to existing methods such as MASAC with only stage-induced rewards, MASAC with only LSTM, and basic MASAC, the proposed strategy shows significant improvements in hunting success rate, with increases of 3.3%, 6.1%, and 24.4% respectively.
Conclusions The proposed stage-induced cooperative target hunting strategy offers valuable technical insights for the development of offensive and defensive strategies in USV operations, enhancing the capabilities of multi-USV systems in complex marine scenarios.
DownLoad: