On-demand storage for data processing and analysis in high energy physics: a case of LIT JINR data and compute infrastructure.

Joint Institute of Nuclear Research (JINR) is a world leading multinational research organization in high energy physics. It’s a site for a novel experiment NICA (Nuclotron-based Ion Collider fAсility), which is designed to study properties of dense baryonic matter – the Quark-Gluon Plasma. Due to start in 2022, a data flow from NICA experiment is expected from hundreds of GB/sec and more with several petabyte volume for one experimental run. JINR’s Mescheryakov Laboratory for Information Technology (MLIT) is designated to create a system for data collection and processing in real time, offline data processing and, at the same time, providing high-performance computing resource for it’s theoreticians. Such task is challenging, given budget and datacenter limitations.

Addressing the problem, MLIT deployed “Govorun” high performance facility, with dis-aggregated infrastructure and smart orchestration for problem-specific configuration. The hyperconverged storage utilizes both dedicated nodes as well as computational nodes NVMe drives with NVM-over-fabric technology to pool resource. Intel Optane storage class memory is used as either meta-data storage or main memory extension. RSC BasIS orchestration enables on-demand configuration for an assortment of filesystems: Lustre, Intel DAOS or others. The orchestration engine is capable to set up filesystem configuration even per individual user job. “Govorun” was stressed with IO500 benchmark and was shown it’s up to data collection challenge (more than 130 GiB/s, 400 kIOPS). The same machine is routinely re-configured for computational jobs. Finally, “Govorun” is extremely energy efficient and compact: it’s compute and storage nodes, Intel OmniPath interconnect switches, are 100% liquid cooled.