ESAPCA: Enabling the Analysis of Extremely Large Data Sets by Scalable and Hardware-Accelerated PCA and DMD
Tuesday, June 10, 2025 3:00 PM to Thursday, June 12, 2025 4:00 PM · 2 days 1 hr. (Europe/Berlin)
Foyer D-G - 2nd floor
Project Poster
High-Performance Data AnalyticsHW and SW Design for Scalable Machine LearningNumerical Libraries
Information
Poster is on display.
Singular Value Decomposition (SVD) is indispensable and ubiquitous in data science and engineering: either it is part of important tools (PCA, POD, DMD etc.) or it is used as pre-/post-processing by dimensionality reduction. However, in the context of very large data sets---as they nowadays arise in many disciplines inside and outside the space sector---this becomes computationally challenging as runtime and memory footprint usually grow superlinearly as a function of data size. Specific use-cases at the European Space Agency (ESA) include ---but are not limited to---, long-term thermospheric density data, earth observation SAR and optical imaging data, and in situ measurements of powder bed solidification. The goal of the project ESAPCA is to develop a parallel, GPU-accelerated implementation of SVD and related techniques, optimized for scalability on high-performance computing (HPC) systems, and with a focus on interoperability within the Python (NumPy/SciPy/scikit-learn) data science ecosystem. Hereby, we will exploit the existing infrastructure for multi-node array computing within the Heat research software library. With this project, we want to fill the gap between the ease of use of the Python NumPy/SciPy/scikit-learn ecosystem, and the need for highly-efficient, hardware-accelerated matrix decomposition in space science and engineering.
The poster provides a concise overview over the goals of the project and shows preliminary numerical results obtained both on Nvidia- and AMD-GPUs.
Acknowledgement and Disclaimer: This research is supported by the European Space Agency through the Open Space Innovation Platform (https://ideas.esa.int) as a Early Technology Development Agreement and carried out under the Discovery Program ESA Early Technology Development (Research Agreement No. 4000144045/24/NL/GLC/ov). The view expressed in this publication can in no way be taken to reflect the official opinion of the European Space Agency.
Contributors:
Singular Value Decomposition (SVD) is indispensable and ubiquitous in data science and engineering: either it is part of important tools (PCA, POD, DMD etc.) or it is used as pre-/post-processing by dimensionality reduction. However, in the context of very large data sets---as they nowadays arise in many disciplines inside and outside the space sector---this becomes computationally challenging as runtime and memory footprint usually grow superlinearly as a function of data size. Specific use-cases at the European Space Agency (ESA) include ---but are not limited to---, long-term thermospheric density data, earth observation SAR and optical imaging data, and in situ measurements of powder bed solidification. The goal of the project ESAPCA is to develop a parallel, GPU-accelerated implementation of SVD and related techniques, optimized for scalability on high-performance computing (HPC) systems, and with a focus on interoperability within the Python (NumPy/SciPy/scikit-learn) data science ecosystem. Hereby, we will exploit the existing infrastructure for multi-node array computing within the Heat research software library. With this project, we want to fill the gap between the ease of use of the Python NumPy/SciPy/scikit-learn ecosystem, and the need for highly-efficient, hardware-accelerated matrix decomposition in space science and engineering.
The poster provides a concise overview over the goals of the project and shows preliminary numerical results obtained both on Nvidia- and AMD-GPUs.
Acknowledgement and Disclaimer: This research is supported by the European Space Agency through the Open Space Innovation Platform (https://ideas.esa.int) as a Early Technology Development Agreement and carried out under the Discovery Program ESA Early Technology Development (Research Agreement No. 4000144045/24/NL/GLC/ov). The view expressed in this publication can in no way be taken to reflect the official opinion of the European Space Agency.
Contributors:
Format
On DemandOn Site
