Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL

@inproceedings{AC_Compr_CoG21,
  title     = {{Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL}},
  author    = {Mysore, Siddharth and El Mabsout, Bassel and Mancuso, Renato and Saenko, Kate},
  booktitle = {IEEE Conference on Games (CoG 2021)},
  year      = 2021,
  month     = aug,
  pages     = {01--08},
  publisher = {IEEE Press},
  address   = {Copenhagen, Denmark},
  doi       = {10.1109/CoG52621.2021.9619008},
  url       = {https://doi.org/10.1109/CoG52621.2021.9619008}
}

CopyDownload .bib

TY  - CONF
AU  - Mysore, Siddharth
AU  - El Mabsout, Bassel
AU  - Mancuso, Renato
AU  - Saenko, Kate
TI  - Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL
T2  - IEEE Conference on Games (CoG 2021)
PY  - 2021
DA  - 2021/08//
PB  - IEEE Press
CY  - Copenhagen, Denmark
SP  - 01
EP  - 08
DO  - 10.1109/CoG52621.2021.9619008
UR  - https://doi.org/10.1109/CoG52621.2021.9619008
AB  - Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 99% reduction in the number of network weights with an average reduction of 77% over multiple actor-critic algorithms on 9 independent tasks. Given that reducing actor complexity results in a direct reduction of run-time inference cost, we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently, particularly in resource-constrained applications or when deploying multiple actors simultaneously.
ER  - 

CopyDownload .ris

%0 Conference Paper
%A Mysore, Siddharth
%A El Mabsout, Bassel
%A Mancuso, Renato
%A Saenko, Kate
%T Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL
%B IEEE Conference on Games (CoG 2021)
%D 2021
%P 01-08
%I IEEE Press
%C Copenhagen, Denmark
%R 10.1109/CoG52621.2021.9619008
%U https://doi.org/10.1109/CoG52621.2021.9619008
%X Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 99% reduction in the number of network weights with an average reduction of 77% over multiple actor-critic algorithms on 9 independent tasks. Given that reducing actor complexity results in a direct reduction of run-time inference cost, we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently, particularly in resource-constrained applications or when deploying multiple actors simultaneously.

CopyDownload .enw

[
    {
        "id": "AC_Compr_CoG21",
        "type": "paper-conference",
        "title": "Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL",
        "author": [
            {
                "family": "Mysore",
                "given": "Siddharth"
            },
            {
                "family": "El Mabsout",
                "given": "Bassel"
            },
            {
                "family": "Mancuso",
                "given": "Renato"
            },
            {
                "family": "Saenko",
                "given": "Kate"
            }
        ],
        "container-title": "IEEE Conference on Games (CoG 2021)",
        "issued": {
            "date-parts": [
                [
                    2021,
                    8
                ]
            ]
        },
        "page": "01-08",
        "publisher": "IEEE Press",
        "publisher-place": "Copenhagen, Denmark",
        "DOI": "10.1109/CoG52621.2021.9619008",
        "abstract": "Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when considering actor and critic architectures independently. By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 99% reduction in the number of network weights with an average reduction of 77% over multiple actor-critic algorithms on 9 independent tasks. Given that reducing actor complexity results in a direct reduction of run-time inference cost, we believe configurations of actors and critics are aspects of actor-critic design that deserve to be considered independently, particularly in resource-constrained applications or when deploying multiple actors simultaneously."
    }
]

CopyDownload .json

S. Mysore, B. El Mabsout, R. Mancuso, and K. Saenko, “Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL,” in IEEE Conference on Games (CoG 2021), pp. 01–08, Aug. 2021, doi: 10.1109/CoG52621.2021.9619008.

CopyDownload .txt

Mysore, S., El Mabsout, B., Mancuso, R., & Saenko, K. (2021). Honey. I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL. In IEEE Conference on Games (CoG 2021) (pp. 01–08). IEEE Press. https://doi.org/10.1109/CoG52621.2021.9619008

CopyDownload .txt