Deep reinforcement learning-based control of multicopters under actuator failure
Translated title
Tiefes bestärkendes Lernen der Regelung von Multikoptern unter Motorausfällen
Publication date
2026-06-23
Document type
Konferenzbeitrag
Author
Organisational unit
Conference
9th International Conference on Industrial Cyber-Physical Systems (ICPS 2026) ; Perth, Australia ; 11-14 May 2026
Publisher
IEEE
Book title
2026 IEEE 9th International Conference on Industrial Cyber-Physical Systems (ICPS)
Peer-reviewed
✅
Part of the university bibliography
✅
Language
English
Keyword
Deep reinforcement learning
Low-level control
Fault-tolerant control
dtec.bw
Abstract
This work presents a Deep Reinforcement Learning (DRL) approach for a low-level fault-tolerant control agent on redundant X8 multicopters. We propose a general design for Reinforcement Learning (RL) agents, which augments the observations with error integrals and utilizes relative control increments to directly control the drone’s actuators. A key focus is the training of agents without explicit anomaly knowledge to investigate their flight performance under various actuator loss-of-effectiveness situations. We also compare cost function designs based on classical additive versus a multiplicative approach based on the geometric mean. Furthermore, these agents are benchmarked against “anomaly-aware” agents with perfect fault knowledge to establish an upper performance bound. Benchmarked against a state-of-the-art Incremental Nonlinear Dynamic Inversion (INDI) controller and an existing RLFTC approach (which combines a classical controller with RLbased updated) our approach demonstrates comparable nominal performance and an increased robustness in anomaly scenarios without the need for explicit domain modeling. Notably, the anomaly-unaware agent with an additive cost function outperformed the INDI controller by a factor of 2.9, while the reference RL-FTC approach achieved a factor of 1.7. Despite these gains, agents without anomaly knowledge are still outperformed by ideal anomaly-aware agents, which achieved up to 3.75 times higher performance.
The simulation environment and evaluation code are available on GitHub and are designed to be run as Gym environments in RL frameworks like Stable-Baselines3.
The simulation environment and evaluation code are available on GitHub and are designed to be run as Gym environments in RL frameworks like Stable-Baselines3.
Version
Published version
Access right on openHSU
Metadata only access
