May 25, 2026 — I travelled to New Orleans to present our paper “Constraint-aware Optimization in Auto-Tuning” at the International Workshop on Automatic Performance Tuning (iWAPT 2026) at the International Parallel & Distributed Processing Symposium (IPDPS). After the presentation, I got to join in as a panelist discussing the state and future of our field! Many thanks to William F. Godoy for organizing this and the attendees for great discussions.
In this paper with Ben van Werkhoven, Stijn Heldens and Rob van Nieuwpoort, we introduce constraint-aware optimization algorithms for auto-tuning. This prevents optimization algorithms from getting bogged down in the swamp of complex auto-tuning search spaces, resulting in a ~40% increase in real-world performance.
These algorithms now included in Kernel Tuner to allow all Kernel Tuner users to speedup their auto-tuning runs in complex search spaces.
Abstract
Automatic performance tuning, or auto-tuning, is a key technique in high-performance computing, enabling applications to adapt to complex and evolving hardware architectures. A central challenge is the need to optimize over large discrete, constrained parameter spaces, where many candidate configurations are invalid due to hardware or software correctness constraints. Traditional evolutionary algorithms, such as Differential Evolution, Particle Swarm Optimization, and Genetic Algorithms, are not inherently constraint-aware and thus often waste computational resources evaluating invalid solutions.
In this work, we present and evaluate constraint-aware variants of four evolutionary algorithms for auto-tuning. Through extensive experiments on a representative benchmark suite, we show that constraint-aware optimization leads to faster convergence and improved performance over unconstrained methods. Furthermore, we demonstrate that our methods outperform the pyATF methods, a state-of-the-art framework for constraint-based auto-tuning. Our results demonstrate that incorporating constraint-awareness into the optimization process significantly enhances their applicability and effectiveness in real-world auto-tuning problems. Constraint-awareness improved algorithm efficiency by ${\sim}39\%$ on average, correlated with search space sparsity. The algorithms developed in this study are publicly available as open-source contributions to the Kernel Tuner framework, facilitating future research and benefitting users.
Citation
F.J. Willemsen, S. Heldens, R.V. van Nieuwpoort, B. van Werkhoven “Constraint-aware Optimization in Auto-Tuning” International Workshop on Automatic Performance Tuning (iWAPT 2026), at IPDPS 2026, New Orleans, USA, May 25-29, 2026 preprint
