Expression Isolation of Compiler-Induced Numerical Inconsistencies in Heterogeneous Code

As the demand for developing and porting numerical applications to heterogeneous computing platforms increases, such programs may exhibit numerical inconsistencies caused by architectural differences and aggressive compiler optimizations. These numerical inconsistencies can negatively impact reproducibility and debugging. This paper presents Ciel, designed to identify the root cause of compiler-induced numerical inconsistencies in heterogeneous programs. Ciel uses a floating-point precision enhancement strategy, guided by a recursive bisection search algorithm with increasing search granularity, to identify the program expressions that induce numerical inconsistencies due to compiler optimizations. Ciel achieves 99.4% precision in isolating numerical inconsistencies in both CPU and GPU programs, including 330 synthetic GPU programs, benchmark applications like NAS Parallel Benchmarks and Rodinia, and real-world scientific applications such as CLOUDSC, a cloud microphysics parameterization mini-app for the ECMWF IFS. Furthermore, when compared with the state of the art, which only isolates lines of code in CPU programs, Ciel runs 24.5% fewer searches for statement isolation, and produces more precise results for 84.9% of the programs. Finally, manual inspection of hundreds of compiler-induced numerical inconsistencies in heterogeneous programs reveals common characteristics.

---

Contributors:

• Ignacio Laguna
• Cindy Rubio-González