Achieving Portable Performance on Extreme-Scale Systems with the PETSc Numerical Toolkit

The Portable Extensible Toolkit for Scientific computation (PETSc) library provides scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization. Throughout its more than 25 year history, a major goal of PETSc development has been achieving scalability required to fully utilize leadership-class supercomputers. In this talk I will describe some of the algorithmic developments made during the era in which achieving inter-node scalability was the primary challenge to enabling extreme-scale computation, and then survey the challenges posed by the current era in which harnessing the abundant fine-scale parallelism within compute nodes -- primarily in the form of GPU-based accelerators -- has assumed at least equal importance. I will discuss how the PETSc design for performance portability addresses these challenges while stressing flexibility and extensibility by separating the programming model used by the application from that used by the library. Additionally, I will discuss recent developments in PETSc's communication module, PetscSF, that enable flexibility and scalable performance across large GPU-based systems while overcoming some of the difficulties posed by working directly with the Message Passing Interface (MPI) on these systems.