HPC On-Demand: Dynamic On-Demand Cluster - Computing for SMEs on COTS Computers
Tuesday, June 10, 2025 3:00 PM to Thursday, June 12, 2025 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Project Poster
Runtime Systems for HPC
Information
Poster is on display.
Data Science is rarely applied in small to medium-sized enterprises due to the lack of dedicated server-administration and/or hardware. Reducing the investment cost of infrastructure could motivate the exploration of data science and machine learning solutions that may improve existing processes and work, e.g in quality control. Our approach is to build a reproducible and scalable Plug & Play Live-ISO with minimal configuration requirements that runs a multi purpose cluster on existing infrastructure in the SMEs, serving well established tools and frameworks that enable users to develop and train machine learning models. Thus, SMEs can use e.g. the developers' machines during off-business hours to train machine learning models, without the need for dedicated hardware or cloud solutions. This reduces overall costs and contributes to data security and protection of intellectual property. The Live-ISOs are built using the Nix package manager and can be customized for various use cases via Nix files. These ISOs operate with NixOS as the underlying operating system and run services defined within the nix files, such as JupyterHub, Slurm, and NFS. By utilizing Live-ISOs, existing hardware can be temporarily repurposed for computing tasks without altering the current operating system, making this solution flexible and cost-effective for SMEs.
Contributors:
Data Science is rarely applied in small to medium-sized enterprises due to the lack of dedicated server-administration and/or hardware. Reducing the investment cost of infrastructure could motivate the exploration of data science and machine learning solutions that may improve existing processes and work, e.g in quality control. Our approach is to build a reproducible and scalable Plug & Play Live-ISO with minimal configuration requirements that runs a multi purpose cluster on existing infrastructure in the SMEs, serving well established tools and frameworks that enable users to develop and train machine learning models. Thus, SMEs can use e.g. the developers' machines during off-business hours to train machine learning models, without the need for dedicated hardware or cloud solutions. This reduces overall costs and contributes to data security and protection of intellectual property. The Live-ISOs are built using the Nix package manager and can be customized for various use cases via Nix files. These ISOs operate with NixOS as the underlying operating system and run services defined within the nix files, such as JupyterHub, Slurm, and NFS. By utilizing Live-ISOs, existing hardware can be temporarily repurposed for computing tasks without altering the current operating system, making this solution flexible and cost-effective for SMEs.
Contributors:
Format
On DemandOn Site
