March 7-11, 2022 — After preparations that spanned more than a year in total, the week of Lorentz Center workshop is finally here! For a long time, it was unsure how the Covid-19 pandemic would develop and just until two months before the workshop, there were still multiple scenarios (online-only, hybrid) that we also considered. However, things were starting to look much better early 2022 and about a month before the workshop, we got the green light to host the workshop in person at the Lorentz Center in Leiden. We were even upgraded to the larger facility in the Oort building at the Lorentz Center, allowing us to host up to 25 participants even if the 1.5 meter distancing rule would be reinstated. Unfortunately, several participants that we had invited from several US National Labs still couldn’t join because of the travel ban by the DoE.
Preparations for the workshop started a little over a year earlier when I reached out to Jiri Filipovic and Ari Rasch, the principle investigators behind Kernel Tuning Toolkit (KTT) and Auto-Tuning Framework (ATF), respectively. This first contact marked the start of what would become a long-standing collaboration between three groups of researchers developing three generic auto-tuning frameworks for GPU applications, with our group working on Kernel Tuner. The fact that three different research groups in Europe are working on three different tools shows the need and importance of the research topic is generally recognized. Interestingly, the three tools are quite complementary in terms of functionality and research focus. Where Kernel Tuner focuses on optimization algorithms for compile-time auto-tuning, KTT focuses on run-time auto-tuning, and ATF of search space construction.
A well-known shortcoming of generic auto-tuners is that they only automate the experimentation and benchmarking phase of the development cycle, and do not necessarily provide much functionality to make it easier to develop tunable code. Therefore, we invited Gabriele Keller, researcher in programming languages for parallel computing to join us as the fourth organizer of the workshop, and to bring together the auto-tuning and programming language research communities. In addition, we were also joined by a number of users of auto-tuning tools, including Chiel van Heerwaarden, who is using auto-tuning to accelerate the GPU code his research group is developing for atmospheric simulations.
During the week-long workshop, we focused on various topics related to generic auto-tuning technology. For example, one of the topics that we covered was to establish a single interface for auto-tuners to make it easier to port tunable code from one tuner to the next. Another group at the workshop was working to establish a commonly agreed output data format that would allow us to exchange data collected by different tuners. Another important topic covered by the workshop was the establishment of a community-driven methodology for benchmarking optimization algorithms for auto-tuning. A project that continued to be researched after the workshop by PhD student Floris-Jan Willemsen. Finally, one of the groups was looking in particular at high-level programming models for scheduling languages, which should eventually make it easier for programmers to express tunable optimizations in high-level programming models. The workshop also initiated a collaboration on BAT, a Benchmark suite for Auto-Tuners, developed by Jacob Torring from NTNU.