A Mixed Precision Randomized Preconditioner for the LSQR Solver on GPUs

Randomized preconditioners for large-scale regression problems have become extremely popular over the past decade. Such preconditioners are known to accelerate large-scale regression solvers both from a theoretical and a practical perspective. In this paper, we present a mixed precision randomized preconditioner for LSQR solvers, focusing on overdetermined, dense least squares problems. We implement and evaluate our method on GPUs and we demonstrate that it outperforms the standard double precision version of randomized, preconditioned LSQR by up to $20\%$ on the NVIDIA A100. We present extensive numerical experiments utilizing the half-precision and tensorcore units to demonstrate that, in many cases, constructing the preconditioner in \textit{reduced precision} does not affect the convergence of LSQR solvers. This leads to important speedups without loss of accuracy.

---

Contributors:

• Christos Boutsikas
• Petros Drineas
• Hartwig Antz