Deakin University
Browse

File(s) under permanent embargo

UAV Dynamic Object Tracking with Lightweight Deep Vision Reinforcement Learning

journal contribution
posted on 2023-06-09, 04:35 authored by Hy Nguyen, Srikanth ThudumuSrikanth Thudumu, Hung Du, Kon MouzakisKon Mouzakis, Rajesh VasaRajesh Vasa
Several approaches have applied Deep Reinforcement Learning (DRL) to Unmanned Aerial Vehicles (UAVs) to do autonomous object tracking. These methods, however, are resource intensive and require prior knowledge of the environment, making them difficult to use in real-world applications. In this paper, we propose a Lightweight Deep Vision Reinforcement Learning (LDVRL) framework for dynamic object tracking that uses the camera as the only input source. Our framework employs several techniques such as stacks of frames, segmentation maps from the simulation, and depth images to reduce the overall computational cost. We conducted the experiment with a non-sparse Deep Q-Network (DQN) (value-based) and a Deep Deterministic Policy Gradient (DDPG) (actor-critic) to test the adaptability of our framework with different methods and identify which DRL method is the most suitable for this task. In the end, a DQN is chosen for several reasons. Firstly, a DQN has fewer networks than a DDPG, hence reducing the computational resources on physical UAVs. Secondly, it is surprising that although a DQN is smaller in model size than a DDPG, it still performs better in this specific task. Finally, a DQN is very practical for this task due to the ability to operate in continuous state space. Using a high-fidelity simulation environment, our proposed approach is verified to be effective.

History

Journal

Algorithms

Volume

16

Article number

227

Pagination

1-23

Location

Basel, Switzerland

ISSN

1999-4893

eISSN

1999-4893

Language

English

Publication classification

C1 Refereed article in a scholarly journal

Issue

5

Publisher

MDPI