基于阶段诱导学习的多无人艇协同目标围捕策略

Stage-inducted learning based cooperative target hunting strategy for multiple unmanned surface vehicles

  • 摘要:
    目的 针对海上目标无人艇智能逃逸问题,提出一种基于阶段诱导学习的多无人艇协同目标围捕策略。
    方法 首先构建针对无人艇围捕逃逸的马尔科夫博弈模型,明确基于距离和角度的围捕成功判定条件。为提升智能逃逸下多无人艇的目标围捕性能,采用集中式训练−分布式执行框架和长短时记忆网络相结合的方法,基于多智能体柔性行动−评判(multi-agent soft actor-critic, MASAC)算法开展协同围捕训练。同时,设计基于阶段诱导的协同围捕奖励机制,依据双方当前状态来优化训练进程,避免“惰性围捕艇”现象,提高围捕成功率,引导无人艇由易到难地完成围捕任务。
    结果 仿真结果表明,与仅采用阶段诱导奖励的MASAC、仅采用长短时记忆网络的MASAC和MASAC围捕策略相比,所提策略的围捕成功率分别提高3.3%,6.1%和24.4%,验证了其可行性和有效性。
    结论 所提策略为无人艇攻防对抗提供了有价值的技术参考,有助于推动无人艇技术在相关领域的应用与发展。

     

    Abstract:
    Objectives  Aiming at the intelligent escape problem of marine target unmanned surface vehicles (USVs), this paper proposes an enhanced cooperative target hunting strategy. This strategy is designed to improve the performance of multi-USV systems in capturing escaping targets through a combination of advanced learning and optimization techniques.
    Methods Firstly, a Markov game process for USV hunting and escaping scenarios is established, with success criteria defined using distance and angle metrics. To enhance the hunting performance against intelligent escapes, a training framework is developed using the centralized training - decentralized execution paradigm and long short-term memory (LSTM) networks. This is integrated with the multi-agent soft actor-critic (MASAC) algorithm for cooperative hunting training. Additionally, a stage-induced cooperative hunting reward method is introduced. This method optimizes the training process based on the current states of both the hunter and the target, guiding the USVs to achieve the hunting task progressively from easier to more challenging stages. It also mitigates the issue of "inertia hunting vehicles" and increases the hunting success rate.
    Results Simulation results, particularly in a 3-USV versus 1-target scenario, validate the feasibility and effectiveness of the proposed strategy. Compared to existing methods such as MASAC with only stage-induced rewards, MASAC with only LSTM, and basic MASAC, the proposed strategy shows significant improvements in hunting success rate, with increases of 3.3%, 6.1%, and 24.4% respectively.
    Conclusions The proposed stage-induced cooperative target hunting strategy offers valuable technical insights for the development of offensive and defensive strategies in USV operations, enhancing the capabilities of multi-USV systems in complex marine scenarios.

     

/

返回文章
返回