Securing HPC Without Air Gapping - (SHAG)

HPC systems typically offer users direct access to the host operating system, thus any local vulnerability can be immediately exploited. Meanwhile, there are continuously new exploits discovered which are potentially exposing the HPC system. This problem is even exacerbated by the need to support a predefined software stack, including a scheduler, like Slurm, or parallel filesystems like BeeGFS or Lustre, which are usually not compatible with the newest Linux kernel and therefore prohibiting early adoption. This exposure was demonstrated when in late 2019 hackers got access to a single user account and started on a single HPC system to escalate their privileges. In order to prevent those incidents, different compute centers tackle IT security from different angles, for instance by blocking ssh connections from other centers to prevent a hoping of attackers or by requiring two-factor authentication. In addition, some centers deploy an intrusion detection system or utilize a number of different networks to isolate management nodes from compute nodes. However the big challenge is to find the correct balance between security and functionality for the users, particularly in Tier-3 systems. Despite all those exiting angles on security, the HPC community does not tap into the potential of an agreed on HPC security guideline (best practices + patterns for it security). Currently this potential is scattered among the different HPC sites. In this BoF, we want to bring together the community to identify how to overcome the challenges and foster a critical discussion for this often neglected topic.