Publications Neural Network Control
Reinforcement Learning for UAV Attitude Control
ACM Transactions on Cyber-Physical Systems, February 2019
Abstract
Autopilot systems are typically composed of an “inner loop” providing stability and control, while an “outer loop” is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control. arXiv:1804.04154v1 [cs.RO] 11 Apr 2018
Cite this paper
@article{UAV_att_rl,
title = {{Reinforcement Learning for UAV Attitude Control}},
author = {Koch, William and Mancuso, Renato and West, Richard and Bestavros, Azer},
journal = {ACM Transactions on Cyber-Physical Systems},
year = 2019,
month = feb,
volume = {3},
number = {2},
pages = {22:1--22:21},
publisher = {ACM},
address = {New York, NY, USA},
doi = {10.1145/3301273},
url = {https://doi.org/10.1145/3301273}
}
TY - JOUR AU - Koch, William AU - Mancuso, Renato AU - West, Richard AU - Bestavros, Azer TI - Reinforcement Learning for UAV Attitude Control JO - ACM Transactions on Cyber-Physical Systems PY - 2019 DA - 2019/02// VL - 3 IS - 2 PB - ACM CY - New York, NY, USA SP - 22:1 EP - 22:21 DO - 10.1145/3301273 UR - https://doi.org/10.1145/3301273 AB - Autopilot systems are typically composed of an “inner loop” providing stability and control, while an “outer loop” is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control. arXiv:1804.04154v1 [cs.RO] 11 Apr 2018 ER -
%0 Journal Article %A Koch, William %A Mancuso, Renato %A West, Richard %A Bestavros, Azer %T Reinforcement Learning for UAV Attitude Control %J ACM Transactions on Cyber-Physical Systems %D 2019 %V 3 %N 2 %P 22:1-22:21 %I ACM %C New York, NY, USA %R 10.1145/3301273 %U https://doi.org/10.1145/3301273 %X Autopilot systems are typically composed of an “inner loop” providing stability and control, while an “outer loop” is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control. arXiv:1804.04154v1 [cs.RO] 11 Apr 2018
[
{
"id": "UAV_att_rl",
"type": "article-journal",
"title": "Reinforcement Learning for UAV Attitude Control",
"author": [
{
"family": "Koch",
"given": "William"
},
{
"family": "Mancuso",
"given": "Renato"
},
{
"family": "West",
"given": "Richard"
},
{
"family": "Bestavros",
"given": "Azer"
}
],
"container-title": "ACM Transactions on Cyber-Physical Systems",
"issued": {
"date-parts": [
[
2019,
2
]
]
},
"volume": "3",
"issue": "2",
"page": "22:1-22:21",
"publisher": "ACM",
"publisher-place": "New York, NY, USA",
"DOI": "10.1145/3301273",
"abstract": "Autopilot systems are typically composed of an “inner loop” providing stability and control, while an “outer loop” is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control. arXiv:1804.04154v1 [cs.RO] 11 Apr 2018"
}
]
W. Koch, R. Mancuso, R. West, and A. Bestavros, “Reinforcement Learning for UAV Attitude Control,” ACM Transactions on Cyber-Physical Systems, vol. 3, no. 2, pp. 22:1–22:21, Feb. 2019, doi: 10.1145/3301273.
Koch, W., Mancuso, R., West, R., & Bestavros, A. (2019). Reinforcement Learning for UAV Attitude Control. ACM Transactions on Cyber-Physical Systems, 3(2), 22:1–22:21. ACM. https://doi.org/10.1145/3301273