Memory-Centric Storage for Exascale (MCSE)
Monday, May 13, 2024 3:00 PM to Wednesday, May 15, 2024 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Project Poster
Parallel File SystemsPerformance MeasurementStorage Technologies and Architectures
Information
Poster is on display.
Our goal in this project is to remove the separation of main memory and non-volatile memory and bring them together in a distributed computing system under a unified interface so that different classes of storage media can be used flexibly in HPC workflows. This goal is implemented through the following steps: First, different input/output (I/O) semantics verbs will be evaluated. Based on the results, a low-level API, the IO verbs, will be created, which allows to map elementary input/output (I/O) of parallel applications. Furthermore, based on the IML system (Infinite Memory Layer) of Fraunhofer ITWM, an ecosystem for highly efficient and memory-centric input/output for application scenarios in campaigns are being developed. For the integration of HPC workflows and campaigns, an execution model will be implemented that efficiently manages the data for communication and keeps intermediate and final results persistent in memory-centric storage. The project will significantly improve the scalability of CODA's I/O to execute a new class of problems in a scalable manner. The potential of the new solution approaches will be validated in the project. The synthetic benchmarks of IO500, a NetCDF benchmark, and the I/O intensive parallel flow solver CODA serve as test scenarios.
Contributors:
Our goal in this project is to remove the separation of main memory and non-volatile memory and bring them together in a distributed computing system under a unified interface so that different classes of storage media can be used flexibly in HPC workflows. This goal is implemented through the following steps: First, different input/output (I/O) semantics verbs will be evaluated. Based on the results, a low-level API, the IO verbs, will be created, which allows to map elementary input/output (I/O) of parallel applications. Furthermore, based on the IML system (Infinite Memory Layer) of Fraunhofer ITWM, an ecosystem for highly efficient and memory-centric input/output for application scenarios in campaigns are being developed. For the integration of HPC workflows and campaigns, an execution model will be implemented that efficiently manages the data for communication and keeps intermediate and final results persistent in memory-centric storage. The project will significantly improve the scalability of CODA's I/O to execute a new class of problems in a scalable manner. The potential of the new solution approaches will be validated in the project. The synthetic benchmarks of IO500, a NetCDF benchmark, and the I/O intensive parallel flow solver CODA serve as test scenarios.
Contributors:
Format
On-site
