[an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive]
[an error occurred while processing this directive]

UAV Maneuvering Target Tracking based on IMM-PPO

  • CHENG Xuming ,
  • CONG Yuhua ,
  • OUYANG Quan ,
  • WANG Zhisheng
Expand
  • College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

Received date: 2022-09-24

  Online published: 2025-02-25

Abstract

Focusing on the problem of UAV maneuvering target tracking in complex obstacle environment, this paper proposes a navigation and tracking strategy based on IMM-PPO. The state information of maneuvering targets with multiple models is estimated, a reward and punishment function based on target tracking performance, tracking approaching time and obstacle constraints is designed. The algorithm framework of near end strategy optimization is designed under the Actor-Critical network structure, and the network parameters under the maximum reward are trained through the interaction between agents and the environment. The trained tracking strategy network can complete obstacle avoidance navigation and achieve stable tracking of maneuvering targets according to environmental information. The simulation results show that, compared with the traditional obstacle avoidance and tracking algorithm, the navigation and tracking strategy based on IMM-PPO has better tracking performance, faster tracking speed, and shorter obstacle avoidance navigation path. The algorithm also has a certain degree of autonomous tracking ability when the initial conditions change, and has greater advantages when applied to the UAV maneuvering target tracking task.

Cite this article

CHENG Xuming , CONG Yuhua , OUYANG Quan , WANG Zhisheng . UAV Maneuvering Target Tracking based on IMM-PPO[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2022 , 42(6) : 46 -54 . DOI: 10.15892/j.cnki.djzdxb.2022.06.007

[an error occurred while processing this directive]
[1]
符小卫, 王辉, 徐哲. 基于DE-MADDP的多无人机协同追捕策略[J]. 航空学报, 2022, 43(5):530-543.

[2]
缪永飞. 多UAV联合搜救任务规划建模及优化方法研究[D]. 武汉:武汉理工大学, 2017.

[3]
DUCHOŇ F, BABINEC A, KAJAN M, et al. Path planning with modified a star algorithm for a mobile robot[J]. Procedia Engineering, 2014, 96: 59-69.

[4]
HUANG Y, GUPTA K. RRT-SLAM for motion planning with motion and map uncertainty for robot exploration[C]// IEEE. Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2008: 1077-1082.

[5]
LI P, DUAN H B. Path planning of unmanned aerial vehicle based on improved gravitational search algorithm[J]. Science China Technological Sciences, 2012, 55(10): 2712-2719.

[6]
WATKINS C J C H, DAYAN P. Q-learning[J]. Machine Learning, 1992, 8(3): 279-292.

[7]
ZHAO Y, ZHENG Z, ZHANG X Y, et al. Q learning algorithm based UAV path learning and obstacle avoidence approach[C]// IEEE. Proceedings of the 2017 36th Chinese Control Conference (CCC). New York: IEEE, 2017: 3397-3402.

[8]
YAN C, XIANG X. A path planning algorithm for uav based on improved q-learning[C]// IEEE. Proceedings of the 2018 2nd International Conference on Robotics and Automation Sciences (ICRAS). New York: IEEE, 2018: 1-5.

[9]
JIANG W, BAO C, XU G, et al. Research on autonomous obstacle avoidance and target tracking of UAV based on improved dueling DQN algorithm[C]// IEEE. Proceedings of the 2021 China Automation Congress (CAC). New York: IEEE, 2021: 5110-5115.

[10]
GUO T, NAN J, LI B Y, et al. UAV navigation in high dynamic environments: a deep reinforcement learning approach[J]. Chinese Journal of Aeronautics, 2021, 34(2): 479-489.

[11]
LI B, WU Y. Path planning for UAV ground target tracking via deep reinforcement learning[J]. IEEE Access, 2020, 8: 29064-29074.

[12]
LI B, YANG Z, CHEN D, et al. Maneuvering target tracking of UAV based on MN-DDPG and transfer learning[J]. Defence Technology, 2021, 17(2): 457-466.

DOI

[13]
SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[D]. Washington: Cornell University, 2017.

[14]
SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.

Outlines

/

[an error occurred while processing this directive]