May 7, 2024 — As a result from a collaboration that started at the Lorentz Center Workshop in March 2022, our paper on how to apply the FAIR Principles (Findable, Accessible, Interoperable, and Reusable) to sharing data in auto-tuning research has now been published at the International Conference on Performance Engineering (ICPE ‘24). The workshop brought together researchers in auto-tuning from many different countries and communities, which is why it was an ideal platform for the creation of a new community-driven approach in data sharing. The paper published today investigates how the FAIR principles may be applied in various use cases where sharing data could be beneficial to auto-tuning research. To also provide a concrete step forward, we have created a JSON schema that can serve as a common template for how to structure data sets produced by auto-tuning software. The JSON format is known as the ‘T4 format’ and is supported as an output format by Kernel Tuner.
Abstract
Autotuning is an automated process that selects the best computer program implementation from a set of candidates to improve performance, such as execution time, when run under new circumstances, such as new hardware. The process of autotuning generates a large amount of performance data with multiple potential use cases, including reproducing results, comparing included methods, and understanding the impact of individual tuning parameters. We propose the adoption of FAIR Principles, which stands for Findable, Accessible, Interoperable, and Reusable, to organize the guidelines for data sharing in autotuning research. The guidelines aim to lessen the burden of sharing data and provide a comprehensive checklist of recommendations for shared data. We illustrate three examples that could greatly benefit from shared autotuning data to advance the research without time- and resource-demanding data collection. To facilitate data sharing, we have taken a community-driven approach to define a common format for the data using a JSON schema and provide scripts for their collection. The proposed comprehensive guide for collecting and sharing performance data in autotuning research can promote further advances in the field and encourage research collaboration.
Citation
Jana Hozzová, Jacob O. Tørring, Ben van Werkhoven, David Strelák, Richard Vuduc. FAIR Sharing of Data in Autotuning Research (Vision Paper). ICPE ‘24 Companion: Companion of the 15th ACM/SPEC International Conference on Performance Engineering doi: https://doi.org/10.1145/3629527.3651429
