Innovative High-Performance Storage for AI/HPC in Exascale Era

This presentation addresses the following topic(s):

• What needs to be done in software to reduce the widening gap between theoretical performance and actual performance?
• Machine learning and HPC: Each worthy of its own investment, or better together? (Alternately: Marriage for the ages, or heading for divorce?)
• What under-appreciated aspects of data management are the real secrets to scalable computing?

Artificial intelligence has changed the scientific research landscape. As a result, more and more users run both AI and HPC applications on their clusters. However, mixed workloads of AI and HPC complicate the I/O pattern and cause storage performance degradation, and there is a strong need for unified storage for the converged AI/HPC environment. In addition, workflows such as weather prediction, bio-science, and AI imaging require an innovative storage system to fulfill high bandwidth and high IOPS needs.

The session covers how QCT addresses the storage challenges with Distributed Asynchronous Object Storage (DAOS). QCT works with Intel to deliver unparalleled IOPS and throughput with only a few DAOS servers. We will demonstrate how DAOS allows users to run AI and HPC workloads with less I/O wait and lower latency with IO benchmarks and real-world applications. Besides the performance, DAOS v2 also supports more telemetry and self-test tools and provides more accessible and user-friendly management.