博弈学习系统在战斗机引导中的应用

陶伟

doi:10.19693/j.issn.1673-3185.01937

博弈学习系统在战斗机引导中的应用

doi: 10.19693/j.issn.1673-3185.01937

陶伟^,

中国舰船研究设计中心，湖北武汉 430064

详细信息

作者简介:
陶伟，男，1978年生，博士，高级工程师。研究方向：舰船电子信息系统设计。E-mail：tw_missu@163.com

通信作者:
陶伟

中图分类号: U662.9
计量
- 文章访问数: 101
- HTML全文浏览量: 31
- PDF下载量: 14
- 被引次数: 0
出版历程
- 收稿日期: 2020-04-28
- 修回日期: 2020-06-17
- 网络出版日期: 2020-12-10

Application of game learning system for fighter guidance

Wei TAO^,

China Ship Development and Design Center, Wuhan 430064, China

摘要

摘要: 目的为通过深度强化学习探索预警机对战斗机的引导策略，设计预警机（EWA）引导战斗机任务的博弈学习系统。方法该系统包括深度学习智能体、与智能体交互的战场仿真系统、博弈管理系统和分布式训练系统。针对强化学习博弈需要与环境进行大量交互的问题，在自博弈训练平台上加入分布式训练系统以提高训练效率。在分布式系统中，提出分布式训练系统中Actor和Learner解耦、各训练Learner之间定期分享更新梯度，以及择优保存并剔除无效智能体等新机制。结果通过该博弈学习系统，实现了红、蓝深度强化学习智能体在预警机引导战斗机任务中的博弈，获得了具有更优异表现的对抗策略，提升了预警机引导作战的能力。结论所做研究可为提升预警机引导作战能力提供参考。
- 预警机 /
- 战斗机引导 /
- 深度强化学习 /
- 博弈
Abstract: Objectives In order to explore the guidance strategy of early warning aircraft (EWA) for fighters through deep reinforcement learning, a game learning system for fighter guidance based on EWA is presented. Methods The game learning system includes a deep learning reinforcement agent, battleground simulation system which can interact with the agent, game management system and distributed training system. For reinforcement learning requires significant interaction with the environment, a distributed training system is introduced to the self-training game platform to improve training efficiency. In the distributed system, the new mechanisms include decoupling the Learner and Actor, the periodic sharing of update gradients among training learners, and selecting the best agents while eliminating invalid agents. Results Through the game learning system, a better EWA guidance strategy can be obtained after games between the blue agent and red agent, thereby enhancing the guidance operational capability of EWA. Conclusions This paper provides a reference for improving the guidance combat capability of early warning aircraft.
- early warning aircraft (EWA) /
- fighter guidance /
- deep reinforcement learning /
- game

HTML全文

图 1 强化学习与环境交互过程

Figure 1. Interactive process between reinforcement learning and environment

下载: 全尺寸图片幻灯片

图 2 深度强化学习智能体示意图

Figure 2. Schematic diagram of deep reinforcement learning agent

下载: 全尺寸图片幻灯片

图 3 战场仿真系统结构图

Figure 3. Structure of battleground simulation system

下载: 全尺寸图片幻灯片

图 4 博弈管理系统结构图

Figure 4. Structure of game management system

下载: 全尺寸图片幻灯片

图 5 分布式训练系统结构图

Figure 5. Structure of distributed training system

下载: 全尺寸图片幻灯片

图 6 博弈学习系统工作流程图

Figure 6. Flow chart of game learning system

下载: 全尺寸图片幻灯片

图 7 训练成功率对比图

Figure 7. Training success rate comparison

下载: 全尺寸图片幻灯片

参考文献(12)

宫颖, 高晓光, 赵峰. 预警机雷达在协同空战中的多目标探测[J]. 火力与指挥控制, 2005, 30(5): 54–58. doi: 10.3969/j.issn.1002-0640.2005.05.016

GONG Y, GAO X G, ZHAO F. Study of AWACS radar detecting multi-group targets in multi-aircraft cooperative combat[J]. Fire Control and Command Control, 2005, 30(5): 54–58 (in Chinese). doi: 10.3969/j.issn.1002-0640.2005.05.016

董阳, 王瑾, 柏鹏. 预警机引导下的编队协同作战仿真分析[J]. 计算机仿真, 2014, 31(9): 80–83. doi: 10.3969/j.issn.1006-9348.2014.09.018

DONG Y, WANG J, BAI P. Research on simulation of formation cooperative operation under early warning aircraft guidance[J]. Computer Simulation, 2014, 31(9): 80–83 (in Chinese). doi: 10.3969/j.issn.1006-9348.2014.09.018

张艳霞, 张安, 孙海洋. 预警机指挥引导编队协同对海作战系统建模与仿真[J]. 系统工程与电子技术, 2016, 38(1): 90–95. doi: 10.3969/j.issn.1001-506X.2016.01.15

ZHANG Y X, ZHANG A, SUN H Y. Research on modeling and simulation of formation cooperation under air early warning command and guidance on the sea battles[J]. Systems Engineering and Electronics, 2016, 38(1): 90–95 (in Chinese). doi: 10.3969/j.issn.1001-506X.2016.01.15

罗爱民, 修胜龙, 罗雪山, 等. 防空导弹网络化作战C⁴ISR系统体系结构研究[J]. 国防科技大学学报, 2004, 26(6): 86–90. doi: 10.3969/j.issn.1001-2486.2004.06.020

LUO A M, XIU S L, LUO X S, et al. Research on network-centric operation C⁴ISR architecture of antiaircraft missile system[J]. Journal of National University of Defense Technology, 2004, 26(6): 86–90 (in Chinese). doi: 10.3969/j.issn.1001-2486.2004.06.020

刘忠, 张秀山, 鄢来斌, 等. 现代军用仿真技术基础[M]. 北京: 国防工业出版社, 2007.

LIU Z, ZHANG X S, YAN L B, et al. Base of modern military simulation technology[M]. Beijing: National Defense Industry Press, 2007 (in Chinese).

SUTTON R S, BARTO A G. Reinforcement learning: an introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5): 1054.

MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236

孙涛, 夏维, 李道飞. 基于模型预测控制的协同式自适应巡航控制系统[J]. 中国机械工程, 2017, 28(4): 486–491. doi: 10.3969/j.issn.1004-132X.2017.04.018

SUN T, XIA W, LI D F. CACC system based on MPC[J]. China Mechanical Engineering, 2017, 28(4): 486–491 (in Chinese). doi: 10.3969/j.issn.1004-132X.2017.04.018

周来, 靳晓伟, 郑益凯. 基于深度强化学习的作战辅助决策研究[J]. 空天防御, 2018, 1(1): 31–35. doi: 10.3969/j.issn.2096-4641.2018.01.009

ZHOU L, JIN X W, ZHENG Y K. Researchon operational decision support based on deep reinforcement learning[J]. Air & Space Defense, 2018, 1(1): 31–35 (in Chinese). doi: 10.3969/j.issn.2096-4641.2018.01.009

赵星宇, 丁世飞. 深度强化学习研究综述[J]. 计算机科学, 2018, 45(7): 1–6. doi: 10.11896/j.issn.1002-137X.2018.07.001

ZHAO X Y, DING S F. Research on deep reinforcement learning[J]. Computer Science, 2018, 45(7): 1–6 (in Chinese). doi: 10.11896/j.issn.1002-137X.2018.07.001

刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1–27. doi: 10.11897/SP.J.1016.2019.00001

LIU Q, ZHAI J W, ZHANG Z C, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1): 1–27 (in Chinese). doi: 10.11897/SP.J.1016.2019.00001

SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484–489. doi: 10.1038/nature16961

施引文献

资源附件(0)

访问统计