This presentation addresses the following topic(s):
- Machine learning and HPC: Each worthy of its own investment, or better together? (Alternately: Marriage for the ages, or heading for divorce?)
Being a niche market that is not attractive for dedicated solution from hardware vendors, HPC has a long tradition of adopting hardware not designed initially for HPC to be useful in HPC. For example in the 90s, the Beowulf project sparked the era of using of-the-self consumer grade processors to construct clusters and supercomputers. Similarly, 15 years ago HPC researchers started building systems that use discrete GPUs originally intended for the gaming market. Most recently, the world fastest supercomputer, Fugaku, took the bold decision to be built using a special extension of ARM CPUs, otherwise used in embedded and mobile devices. With the same maverick spirit, HPC has an opportunity of riding the wave of change in processor design driven by the growing demand for hardware optimized for ML workloads. However, there are also challenges for adopting ML-optimized hardware, such as the lower precision favored in ML, over-emphasize on matrix engines that might not be of direct utility for many HPC applications etc. In this talk we highlight areas where HPC can align itself to the evolving adaptions of hardware to ML, while also discussing the areas to be cautious of.
