Regularizing Action Policies for Smooth Control with Reinforcement Learning

@inproceedings{regul_RL_ICRA21,
  title     = {{Regularizing Action Policies for Smooth Control with Reinforcement Learning}},
  author    = {Mysore, Siddharth and El Mabsout, Bassel and Mancuso, Renato and Saenko, Kate},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA’21)},
  year      = 2021,
  month     = may,
  address   = {Xi’an, China},
  doi       = {10.1109/ICRA48506.2021.9561138},
  url       = {https://doi.org/10.1109/ICRA48506.2021.9561138}
}

CopyDownload .bib

TY  - CONF
AU  - Mysore, Siddharth
AU  - El Mabsout, Bassel
AU  - Mancuso, Renato
AU  - Saenko, Kate
TI  - Regularizing Action Policies for Smooth Control with Reinforcement Learning
T2  - IEEE International Conference on Robotics and Automation (ICRA’21)
PY  - 2021
DA  - 2021/05//
CY  - Xi’an, China
DO  - 10.1109/ICRA48506.2021.9561138
UR  - https://doi.org/10.1109/ICRA48506.2021.9561138
AB  - A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-toaction mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps
ER  - 

CopyDownload .ris

%0 Conference Paper
%A Mysore, Siddharth
%A El Mabsout, Bassel
%A Mancuso, Renato
%A Saenko, Kate
%T Regularizing Action Policies for Smooth Control with Reinforcement Learning
%B IEEE International Conference on Robotics and Automation (ICRA’21)
%D 2021
%C Xi’an, China
%R 10.1109/ICRA48506.2021.9561138
%U https://doi.org/10.1109/ICRA48506.2021.9561138
%X A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-toaction mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps

CopyDownload .enw

[
    {
        "id": "regul_RL_ICRA21",
        "type": "paper-conference",
        "title": "Regularizing Action Policies for Smooth Control with Reinforcement Learning",
        "author": [
            {
                "family": "Mysore",
                "given": "Siddharth"
            },
            {
                "family": "El Mabsout",
                "given": "Bassel"
            },
            {
                "family": "Mancuso",
                "given": "Renato"
            },
            {
                "family": "Saenko",
                "given": "Kate"
            }
        ],
        "container-title": "IEEE International Conference on Robotics and Automation (ICRA’21)",
        "issued": {
            "date-parts": [
                [
                    2021,
                    5
                ]
            ]
        },
        "publisher-place": "Xi’an, China",
        "DOI": "10.1109/ICRA48506.2021.9561138",
        "abstract": "A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-toaction mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps"
    }
]

CopyDownload .json

S. Mysore, B. El Mabsout, R. Mancuso, and K. Saenko, “Regularizing Action Policies for Smooth Control with Reinforcement Learning,” in IEEE International Conference on Robotics and Automation (ICRA’21), May. 2021, doi: 10.1109/ICRA48506.2021.9561138.

CopyDownload .txt

Mysore, S., El Mabsout, B., Mancuso, R., & Saenko, K. (2021). Regularizing Action Policies for Smooth Control with Reinforcement Learning. In IEEE International Conference on Robotics and Automation (ICRA’21). https://doi.org/10.1109/ICRA48506.2021.9561138

CopyDownload .txt