Introduction to HPC Applications, Systems, Programming Models and Machine Learning and Data Analytics

In the first part of this introductory tutorial, you will learn what "high performance computing" (HPC) means and what differentiates it from more mainstream areas of computing. You will also be introduced to the major applications that use HPC for research and industry, and how Artificial Intelligence (AI) and HPC interact with each other. Then, we present the major HPC system architectures needed to run these applications. Finally, we provide an overview of the languages and paradigms used to program HPC applications and systems. Contents Part 1 • HPC Application Areas • Basic Terminology • Evaluating Program Performance • HPC Hardware Architectures (Shared and Distributed Memory Systems, Hybrid Systems, Accelerators) • Parallel Programming (MPI, OpenMP, OpenACC, CUDA, HIP) The second part offers basics of analyzing data with machine learning and data analytics algorithms to understand foundations of learning from large quantities of data. It consists of HPC methods for data analysis to perform clustering and classification. This includes a short discussion of test datasets, training datasets, and validation datasets. Easy application examples in context are given to foster the theoretical tutorial elements that also will illustrate problems like overfitting followed by mechanisms such as validation and regularization that prevent such problems. Finally, short insights into training deep learning algorithms at scale and hyperparameter search will be given. Contents Part 2 • Introduction to Machine Learning and Data Analytics • Basic Terminology • Parallel and Scalable Machine Learning Algorithms • Distributed Training of Deep Learning Models • Hyperparameter search