Bounding-Box of User Expectation (BBoUE) Tuning Workflow Assessment for Performance Engineering Guidance
Wednesday, June 1, 2022 1:52 PM to 1:56 PM · 4 min. (Europe/Berlin)
Hall D - 2nd Floor
HPC Workflows
Information
Benchmarking has been used frequently to evaluate modern complex systems by covering each individual aspect such as I/O. Evaluating I/O performance itself is notoriously difficult to do due to various intertwined variables. To answer this challenge, there are several benchmarks available out there with the IO500 benchmark being the current de-facto leader of the benchmark.
We use the Bounding-Box of User Expectation (BBoUE) Tuning Workflow that we proposed in our previous work and the public data available on the IO500 website to showcase how the real-world systems and various applications interact with the benchmark. This work aims to provide insights for users to help them understand benchmark numbers and use them for performance improvements.
In our experiment to map the application's performance into the BBoUE Tuning Workflow, we observe several aspects that can be added to the benchmark, such as caching data. Using the same tuning workflow, we also observe the system design changes in the IO500 list that affect the default config of the IO500 benchmark. With these observations, we are aiming to create a unified understanding between performance tools output and benchmark output, by experimenting with more applications and performance tools to make sense of the numbers produced.
Contributors:
We use the Bounding-Box of User Expectation (BBoUE) Tuning Workflow that we proposed in our previous work and the public data available on the IO500 website to showcase how the real-world systems and various applications interact with the benchmark. This work aims to provide insights for users to help them understand benchmark numbers and use them for performance improvements.
In our experiment to map the application's performance into the BBoUE Tuning Workflow, we observe several aspects that can be added to the benchmark, such as caching data. Using the same tuning workflow, we also observe the system design changes in the IO500 list that affect the default config of the IO500 benchmark. With these observations, we are aiming to create a unified understanding between performance tools output and benchmark output, by experimenting with more applications and performance tools to make sense of the numbers produced.
Contributors:
- Radita Liem (RWTH Aachen University)
Format
On-site
