Know your Enemy: Benchmarking and Experimenting with Insight as a Goal

Mattia Nicolella; Denis Hoornaert; Shahin Roozkhosh; Andrea Bastoni; Renato Mancuso

Publications Workload Profiling

Know your Enemy: Benchmarking and Experimenting with Insight as a Goal

Mattia Nicolella, Denis Hoornaert, Shahin Roozkhosh, Andrea Bastoni, Renato Mancuso

WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022), 2022, Houston, TX, USA

Read the PDF Code and artifacts Cite

Fig. 2. Clustering of benchmarks and their suite based on instruction per clock (IPC) and last-level cache refills per clock (LLC-RPC).

Abstract

Available benchmark suites are used to provide realistic workloads and to understand their run-time characteristics. However, they do not necessarily target the same platforms and often offer a diverse set of metrics, leading to the lack of a knowledge base that could be used for both systems and theoretical research. RT-Bench, a new benchmark framework environment, tries to address these issues by providing a uniform interface and metrics while maintaining portability. This demo illustrates how to leverage this framework and its recentlyadded features to improve the understanding of the benchmarks’ interaction with its system.

Code and hardware artifacts

RT-Bench Software Introduced in this paper
An extensible framework that turns existing benchmarks into periodic real-time workloads.

Cite this paper

@inproceedings{RT_Bench_WiP_RTSS22,
  title     = {{Know your Enemy: Benchmarking and Experimenting with Insight as a Goal}},
  author    = {Nicolella, Mattia and Hoornaert, Denis and Roozkhosh, Shahin and Bastoni, Andrea and Mancuso, Renato},
  booktitle = {WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022)},
  year      = 2022,
  address   = {Houston, TX, USA},
  url       = {https://cs-people.bu.edu/rmancuso/files/papers/RTBench_RTSS22.pdf}
}

Download .bib

TY  - CONF
AU  - Nicolella, Mattia
AU  - Hoornaert, Denis
AU  - Roozkhosh, Shahin
AU  - Bastoni, Andrea
AU  - Mancuso, Renato
TI  - Know your Enemy: Benchmarking and Experimenting with Insight as a Goal
T2  - WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022)
PY  - 2022
CY  - Houston, TX, USA
AB  - Available benchmark suites are used to provide realistic workloads and to understand their run-time characteristics. However, they do not necessarily target the same platforms and often offer a diverse set of metrics, leading to the lack of a knowledge base that could be used for both systems and theoretical research. RT-Bench, a new benchmark framework environment, tries to address these issues by providing a uniform interface and metrics while maintaining portability. This demo illustrates how to leverage this framework and its recentlyadded features to improve the understanding of the benchmarks’ interaction with its system.
ER  -

Download .ris

%0 Conference Paper
%A Nicolella, Mattia
%A Hoornaert, Denis
%A Roozkhosh, Shahin
%A Bastoni, Andrea
%A Mancuso, Renato
%T Know your Enemy: Benchmarking and Experimenting with Insight as a Goal
%B WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022)
%D 2022
%C Houston, TX, USA
%X Available benchmark suites are used to provide realistic workloads and to understand their run-time characteristics. However, they do not necessarily target the same platforms and often offer a diverse set of metrics, leading to the lack of a knowledge base that could be used for both systems and theoretical research. RT-Bench, a new benchmark framework environment, tries to address these issues by providing a uniform interface and metrics while maintaining portability. This demo illustrates how to leverage this framework and its recentlyadded features to improve the understanding of the benchmarks’ interaction with its system.

Download .enw

[
    {
        "id": "RT_Bench_WiP_RTSS22",
        "type": "paper-conference",
        "title": "Know your Enemy: Benchmarking and Experimenting with Insight as a Goal",
        "author": [
            {
                "family": "Nicolella",
                "given": "Mattia"
            },
            {
                "family": "Hoornaert",
                "given": "Denis"
            },
            {
                "family": "Roozkhosh",
                "given": "Shahin"
            },
            {
                "family": "Bastoni",
                "given": "Andrea"
            },
            {
                "family": "Mancuso",
                "given": "Renato"
            }
        ],
        "container-title": "WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022)",
        "issued": {
            "date-parts": [
                [
                    2022
                ]
            ]
        },
        "publisher-place": "Houston, TX, USA",
        "abstract": "Available benchmark suites are used to provide realistic workloads and to understand their run-time characteristics. However, they do not necessarily target the same platforms and often offer a diverse set of metrics, leading to the lack of a knowledge base that could be used for both systems and theoretical research. RT-Bench, a new benchmark framework environment, tries to address these issues by providing a uniform interface and metrics while maintaining portability. This demo illustrates how to leverage this framework and its recentlyadded features to improve the understanding of the benchmarks’ interaction with its system."
    }
]

Download .json

M. Nicolella, D. Hoornaert, S. Roozkhosh, A. Bastoni, and R. Mancuso, “Know your Enemy: Benchmarking and Experimenting with Insight as a Goal,” in WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022), 2022.

Download .txt

Nicolella, M., Hoornaert, D., Roozkhosh, S., Bastoni, A., & Mancuso, R. (2022). Know your Enemy: Benchmarking and Experimenting with Insight as a Goal. In WiP Session @ 43rd IEEE Real-Time Systems Symposium (RTSS@Work 2022).

Download .txt