A Coordinated Attack Method for Multi-UAV Based on MADDPG

ZHANG Bo; LIU Manguo; LIU Mengyan

doi:10.15892/j.cnki.djzdxb.2025.03.011

Journal of Projectiles, Rockets, Missiles and Guidance >

2025 , Vol. 45 >Issue 3: 344 - 350

DOI: https://doi.org/10.15892/j.cnki.djzdxb.2025.03.011

A Coordinated Attack Method for Multi-UAV Based on MADDPG

ZHANG Bo ,
LIU Manguo ,
LIU Mengyan

Expand

Xi'an Mordern Control Technology Research Institute,Xi'an 710065,Shaanxi,China

Received date: 2024-12-30

Online published: 2025-07-09

Fold

Abstract

It is an important direction for the future development of UAV military field to coordinated attack of multi-UAV to accomplish specific strike tasks. Aiming at the problem of coordinated attack of multi-UAV, a typical confrontation scenario is constructed. The unmanned aerial vehicle cooperative attack problem is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a unique reward function is designed. The multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to train the attack strategy. Monte Carlo method is used to analyze the simulation experiment, and the results show that after the training of the multi-agent reinforcement learning algorithm, the completion rate of the UAV cooperative attack task reaches 82.9% in specific confrontation scenarios.

Key words： multi-agent; deep reinforcement learning; decentralized partially observable Markov decision process (Dec-POMDP); multi-agent deep deterministic policy gradient (MADDPG); UAV cluster

Cite this article

ZHANG Bo , LIU Manguo , LIU Mengyan . A Coordinated Attack Method for Multi-UAV Based on MADDPG[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2025 , 45(3) : 344 -350 . DOI: 10.15892/j.cnki.djzdxb.2025.03.011

[an error occurred while processing this directive]

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	黄雷. 美军小精灵无人机群项目发展现状综述[J]. 飞航导弹, 2018(7): 44-47. HUANG L. Overview of the development status of the U.S. Army's Pixie UAV swarm project[J]. Aerospace Missiles, 2018(7): 44-47.

[2]	王璐菲. DARPA研发“小精灵”无人机用于分布式空中作战[J]. 防务视点, 2015(11): 62-63. WANG L F. DARPA develops Gremlin UAV for distributed air warfare[J]. Defense Perspectives, 2015(11): 62-63.

[3]	吕震华, 高亢. 美国无人集群城市作战应用发展综述[J]. 中国电子科学研究院学报, 2020, 15(8): 738-745. LU Z H, GAO K. Review on the development of unmanned cluster urban warfare applications in the United States[J]. Journal of the China Academy of Electronic Science, 2020, 15(8): 738-745.

[4]	QUAN L, YIN L J, XU C, et al. Distributed swarm trajectory optimization for formation flight in dense environments[C]//2022 International Conference on Robotics and Automation (ICRA), June 23-27, 2022, Philadelphia: ICRA, 2022: 4979-4985.

[5]	SUTTON R, BARTO A. 强化学习[M]. 北京: 电子工业出版社, 2019: 15-20. SUTTON R, BARTO A. Reinforcement learning[M]. Beijing: Publishing House of Electronics Industry, 2019: 15-20.

[6]	邹启杰, 蒋亚军, 高兵, 等. 协作多智能体深度强化学习研究综述[J]. 航空兵器, 2022, 29(6): 78-88. ZOU Q J, JIANG Y J, GAO B, et al. An overview of cooperative multi-agent deep reinforcement learning[J]. Aero Weaponry, 2022, 29(6): 78-88.

[7]	ARULKUMARAN K, DEISENROTH M P, BRUNDAGE M, et al. Deep reinforcement learning: a brief survey[J]. IEEE Signal Processing Magazine, 2017, 34(6): 26-38.

[8]	张婷婷, 杨学军. 基于强化学习的城市场景下巡飞弹自主协同饱和攻击方法[J]. 指挥与控制学报, 2023, 9(4): 457-468. ZHANG T T, YANG X J. Autonomous coordination saturation attacks method for loitering munitions in urban scenarios based on reinforcement learning[J]. Journal of Command and Control, 2023, 9(4): 457-468

[9]	仵博. 动态不确定环境下的智能体序贯决策方法及应用研究[D]. 长沙: 中南大学, 2013. WU B. A sequential decision making method for intelligent bodies under dynamic uncertainty and its application[D]. Changsha: Zhongnan University, 2013.

[10]	谢宇轩. 基于集成学习的多智能体强化学习算法研究[D]. 哈尔滨: 哈尔滨工业大学, 2022. XIE Y X. Research on multi-agent reinforcement learning algorithm based on ensemble learning[D]. Harbin: Harbin Institute of Technology, 2022.

[11]	尹华一, 尤雅丽, 黄新栋, 等. 基于MADDPG的多AGVs路径规划算法[J]. 厦门理工学院学报, 2024, 32(1): 37-46. YIN H Y, YOU Y L, HUANG X D, et al. Multi-AGVs path planning algorithm based on MADDPG[J]. Journal of Xiamen University of Technology, 2019, 32(1): 37-46.

[12]	宋明惠. 基于深度强化学习的无人机自主飞行控制算法的研究与实现[D]. 北京: 北京交通大学, 2022. SONG M H. Research and implementation of UAV autonomous flight control algorithm based on deep reinforcement learning[D]. Beijing: Beijing Jiaotong University, 2022.

[13]	李超, 王瑞星, 黄建忠, 等. 稀疏奖励下基于强化学习的无人集群自主决策与智能协同[J]. 兵工学报, 2023, 44(6): 1537-1546. DOI LI C, WANG R X, HUANG J Z, et al. Autonomous decision making and intelligent cooperation of unmanned clusters based on reinforcement learning under sparse reward[J]. Acta Armamentarii, 2023, 44(6): 1537-1546.

[14]

郭宏达, 娄静涛, 杨珍珍, 等. 基于拍卖多智能体深度确定性策略梯度的多无人车分散策略研究[J]. 电子与信息学报, 2024, 46(1): 287-298.

GUO

H D

, LOU

J T

, YANG

Z Z

, et al. Research on multi-unmanned vehicle decentralization strategy based on auction multi-agent depth deterministic strategy gradient[J]. Journal of Electronics and Information Technology, 2019, 46(1): 287-298.

[15]	戴凤琪. 基于深度强化学习的智能体算法研究[D]. 北京: 北京交通大学, 2023. DAI F Q. Research on agent algorithm based on deep reinforcement learning[D]. Beijing: Beijing Jiaotong Univer-sity, 2023.

[16]	林萌龙, 陈涛, 任棒棒, 等. 基于多智能体深度强化学习的体系任务分配方法[J]. 指挥与控制学报, 2023, 9(1): 93-102. LIN M L, CHEN T, REN B B, et al. System task assignment method based on multi-agent deep reinforcement Learning[J]. Journal of Command and Control, 2023, 9(1): 93-102.

[17]	李波, 越凯强, 甘志刚, 等. 基于MADDPG的多无人机协同任务决策[J]. 宇航学报, 2021, 42(6): 757-765. LI B, YUE K Q, GAN Z G, et al. Multi-UAVs collaborative mission decision based on MADDPG[J]. Journal of Astronautics, 2021, 42(6): 757-765.

[18]	杨书恒, 张栋, 任智, 等. 基于多智能体强化学习的无人机集群对抗方法研究[J]. 无人系统技术, 2022, 5(5): 51-62. YANG S H, ZHANG D, REN Z, et al. Research on UAV swarm confrontation method based on multi-agent reinforcement learning[J]. Unmanned Systems Technology, 2022, 5(5): 51-62.

Options

Outlines