Designing Next-Generation Linear Algebra Libraries for Large-Scale Matrix Computations

Exploiting data sparsity in dense matrices is an algorithmic bridge between highly parallel architectures that are increasingly memory-austere on a per-core basis and extreme-scale scientific applications. The Hierarchical matrix Computations on Manycore Architectures (HiCMA) library tackles this challenging problem by achieving significant reductions in time to solution and memory footprint while preserving a specified accuracy requirement of the application. HiCMA provides high-performance implementations on various massively parallel systems of one of the most widely used matrix factorizations in large-scale scientific applications, i.e., the Cholesky factorization, as well as the more traditional matrix-vector multiplication operation. It supports mixed-precision computations and/or Tile Low-Rank (TLR) data format to compress the dense data-sparse off-diagonal tiles of the matrix. HiCMA eventually translates the matrix computations into interdependent tasks, which may operate on dense or TLR data structures, while additionally mixing precision arithmetics determined by a tile-centric approach. HiCMA relies on dynamic runtime systems not only for asynchronous out-of-order scheduling but also on-the-fly precision conversion during data movement. Performance comparisons and memory footprint on large matrix dimensions show a performance gain and memory saving of more than an order of magnitude for both metrics on thousands of cores against state-of-the-art open-source and vendor optimized numerical libraries. This represents an important milestone in enabling large-scale matrix computations toward solving big data problems in seismic imaging for carbon sequestration, geospatial statistics for climate/weather predictions, or genome-wide association studies for mapping specific genetic variations with particular diseases.