Real Time Data Processing Environment for Radio Astronomy

The Pawsey Supercomputing Research Centre (Pawsey) is an Australian organisation contributing to the Square Kilometre Array (SKA) project. Pawsey is a Tier-0 data and supercomputing support site for two SKA precursor projects, the CSIRO’s Australian Square Kilometre Array Pathfinder (ASKAP) and the Murchison Widefield Array (MWA). The two projects were launched in late 2012 and are serving as important technological demonstrators for the SKA and currently deliver ground-breaking science. These precursors are located at the Murchison Radio-astronomy Observatory, the centre of a radio-quiet zone, near Boolardy in Western Australia, about 700km northeast of Perth. Data ingest at the centre is currently implemented using separate per-project HPC and storage resources. In this presentation we discuss how all the data ingest, processing and storage tasks will be performed within a single HPC system at a larger scale, taking advantage of software defined configuration to dynamically partition the environment as needed, while achieving near-zero downtime.