Emerging Paradigms in the Convergence of Cloud and High-Performance Computing
Time: Fri 2023-12-15 10.00
Location: Visualization Studio, Lindstedtsvägen 9
Language: English
Subject area: Computer Science
Doctoral student: Daniel Araújo De Medeiros , Beräkningsvetenskap och beräkningsteknik (CST)
Opponent: Associate Professor Valeria Cardellini, University of Rome, Tor Vergata via del Politecnico 1, 00133 Roma, Italy
Supervisor: Assistant professor Ivy Bo Peng, Beräkningsvetenskap och beräkningsteknik (CST); Professor Stefano Markidis, Beräkningsvetenskap och beräkningsteknik (CST); Associate professor Pawel Herman, Beräkningsvetenskap och beräkningsteknik (CST)
QC 20231122
Abstract
Traditional HPC scientific workloads are tightly coupled, while emerging scientific workflows exhibit even more complex patterns, consisting of multiple characteristically different stages that may be IO-intensive, compute-intensive, or memory-intensive. New high-performance computer systems are evolving to adapt to these new requirements and are motivated by the need for performance and efficiency in resource usage. On the other hand, cloud workloads are loosely coupled, and their systems have matured technologies under different constraints from HPC.
In this thesis, the use of cloud technologies designed for loosely coupled dynamic and elastic workloads is explored, repurposed, and examined in the landscape of HPC in three major parts. The first part deals with the deployment of HPC workloads in cloud-native environments through the use of containers and analyses the feasibility and trade-offs of elastic scaling. The second part relates to the use of workflow management systems in HPC workflows; in particular, a molecular docking workflow executed through Airflow is discussed. Finally, object storage systems, a cost-effective and scalable solution widely used in the cloud, and their usage in HPC applications through MPI I/O are discussed in the third part of this thesis.