Memory Centric Exploration of Neural Network

This presentation addresses the following topic(s):

• Machine learning and HPC: Each worthy of its own investment, or better together? (Alternately: Marriage for the ages, or heading for divorce?)

Machine Learning has become pervasive in our daily lives. The ML algorithms of choice for most applications are Deep Neural Networks (DNNs). DNNs are extremely large ML models composed of cascading layers of compute. The memory capacity and bandwidth requirements of these models are quite large and has continued to grow in recent years. We analyze several state-of-the-art neural network models and their performance in relation to main memory. Additionally, we propose metrics and best practices to improve DNN design and performance from a memory centric perspective.