A Hybrid Quantum-Classical Workflow for Hyperparameter Optimization of Neural Networks

Hyperparameter optimization of neural networks is a computationally expensive procedure as a large number of networks, all with different hyperparameter configurations need to be fully trained. While the goal is to find the hyperparameters that show the best performance, a lot of configurations among these do not show strong performance which can already be detected during the training. Early stopping of these under-performing configurations can reduce computational costs. This work presents a hybrid quantum-classical workflow for performing early stopping in hyperparameter optimization, combining a Quantum Annealer (QA) with a classical supercomputer. The workflow consists of a three-step process: first, the different neural network configurations are trained partially on the GPU partition of the supercomputer. After reaching half the number of epochs, the training is stopped and the validation accuracy learning curves are transferred to the QA. The partial learning curves are then extrapolated, using a quantum implementation of the Support Vector Regression (Q-SVR) algorithm to predict which configurations will show a strong final performance. These results are transferred back to the classical supercomputer, where the second half of training is only resumed for the most promising configurations. Subsequently, all other trials are terminated. Results on the optimization of a simple neural network show that the hybrid early-stopping approach reduces the runtime of the hyperparameter optimization by 30% while still discovering optimal configurations.

---

Contributors:

• Eric Wulff
• Edoardo Pasetto
• Amer Delilbasic
• Rakesh Sarma
• Eray Inanc
• Maria Girone
• Morris Riedel
• Andreas Lintermann