CONNECTING YOU TO THE CONTINUUM

MultiScale-HPC Hybrid Architectures: Developing Computing Continuum Towards Sustainable Advanced Computing

Carlos J. Barrios H. (LIG/INRIA - DataMove, Grenoble, France, CITI/INRIA - Syndi, Lyon, France, SC3UIS-CAGE, Bucaramanga, Colombia)

May 28th, 2025 at 9 am PDT.
ADD TO CALENDAR

The concept of a computing continuum pertains to the seamless integration of diverse computing resources, ranging from edge devices to cloud infrastructures. This continuum facilitates efficient data processing across varying scales and types of workloads. Researchers can optimize resource utilization by employing a hybrid approach, ensuring that computations are executed most efficiently, whether performed on-premises or in the cloud, utilizing a multi-scale computing methodology. Consequently, we encounter a critical intersection of HPC and sustainability. This work illustrates how the Multi-Scale HPC hybrid architecture concept evolves into a computing continuum framework that combines traditional HPC approaches to support the execution of large-scale multiscale applications, enhancing performance and sustainability. The research addresses computational needs by identifying intensive and massive workloads (from scientific applications to AI and Datalakes applications) and developing frameworks for diverse and scalable computing resources. It also notes that as the demand for computational power grows, the environmental impact of multiscale HPC systems becomes increasingly significant. Therefore, it is necessary to use different approaches in performance evaluation to define computing efficiency in a sustainable mode.

Contributions to the Convergence of High Performance Computing and Cloud Computing

Márcio Castro, Federal University of Santa Catarina

April 29th, 2025 at 9 am PDT.
WATCH RECORDING

The cloud computing paradigm democratized compute infrastructure access to millions of resource-strained organizations, applying economics of scale to massively reduce infrastructure costs. In the High Performance Computing (HPC) context, the benefits of using public cloud resources make it an attractive alternative to expensive on-premises HPC clusters. In this talk, we discuss the challenges and limitations of current solutions and present our three recent contributions to this area: (i) HPC@Cloud: a provider-agnostic software framework that comprises a set of key software tools to assist in the migration, test and execution of HPC applications in public clouds; (ii) fault tolerance strategies based on in-memory checkpoint restart considering the particularities of unreliable cloud computing environments (a.k.a spot market); and (iii) an empirical approach for estimating cloud infrastructure costs to help researchers gain a quick impression of the pros and cons of the reserved and spot cloud options for CPU-intensive physics simulations and similar classes of applications.

Building Next-Generation Scientific Workflows for Autonomous Research Facilities

Rafael Ferreira da Silva, Oak Ridge National Laboratory, US

March 25th, 2025 at 9 am PDT.
WATCH RECORDING SLIDES

The landscape of scientific workflows is undergoing a transformative shift as research facilities evolve towards greater autonomy and intelligence. This presentation examines emerging trends and challenges in scientific workflows, with particular emphasis on the convergence of AI/ML with HPC systems, time-sensitive operations, and multi-facility integrations. Drawing from recent community discussions and Oak Ridge Leadership Computing Facility’s experiences, we explore how modern workflows are adapting to handle unprecedented data volumes, near real-time processing requirements, and complex cross-facility collaborations. Special attention is given to OLCF’s efforts in supporting the Department of Energy’s Integrated Research Infrastructure initiative and the Labs of the Future vision, including the development of standardized protocols for authentication, data movement, and workflow orchestration across distributed research environments. The presentation highlights key technical challenges and proposed solutions for creating sustainable, AI-enhanced workflow ecosystems that can effectively bridge traditional scientific methods with autonomous laboratory operations, while maintaining essential human oversight and scientific rigor.

An Architecture for the Management of Data in the Computing Continuum

Prof. Jesus Carretero, Universidad Carlos III de Madrid

February 25th, 2025 at 9 am PST.
WATCH RECORDING

Data distribution across different facilities offers benefits such as enhanced resource utilization, increased resilience through replication, and improved performance by processing data near its source. However, managing such data is challenging due to heterogeneous access protocols, disparate authentication models, and the lack of a unified coordination framework. This talk presents an architecture for managing data in the computing continuum across heterogeneous storage systems. At the core of architecture are data containers, an abstraction that provides standardized interfaces for seamless data management, irrespective of the underlying storage systems. Multiple data container connections create a cohesive wide-area storage network, ensuring resilience using erasure coding policies. Furthermore, a load-balancing algorithm ensures equitable and efficient utilization of storage resources.

AI & HPC: Trends

Bruno Raffin, INRIA, Grenoble, France

September 24th, 2024 at 8 am PDT.
WATCH RECORDING

AI is emerging as a strong driving force in HPC, influencing hardware, software, as well as scientific computing. During this talk we will overview the changes brought by AI in the context of traditional HPC applications. The AI software stack is addressing HPC programming with a different perspective, Python being the standard user interface, JIT compilation the backbone to ensure performance portability. How this is preluding a paradigm shift in HPC programming ? The differentiable-programming paradigm that is key to deep learning, coupled with powerful minimization algorithms is also making its way into scientific computing in many different ways for enabling new classes of surrogate models, PDE solvers, end-to-end differentiable simulation codes combining components with trainable parameters while others rely on traditional discretization of PDEs. We will present these different points with examples from resent research results and engage with the audience into a discussion about foreseen changes and limitations.

Optimizing Deployment Time in Edge Computing Environments: Challenges and Innovations

Tiago Ferreto, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS)

August 21st, 2024 at 9 am PDT.
WATCH RECORDING

As edge computing continues to evolve, the rapid and efficient deployment of applications has become critical for maximizing performance and minimizing latency. This talk will delve into the importance of controlling deployment time in edge environments, highlighting the unique challenges faced during this process. We will explore the main issues, such as network constraints, hardware heterogeneity, and resource allocation. Additionally, we will present innovative strategies developed by our research team to address these challenges, demonstrating how these solutions can enhance the overall efficiency and reliability of edge deployments.

Can software-defined storage solve long-standing HPC storage problems?

Ricardo Gonçalves Macedo, INESC TEC & Universidade do Minho

July 18th, 2024 at 9 am PDT.
WATCH RECORDING

Modern I/O applications running on HPC infrastructures, including deep learning and large language model workloads, are increasingly becoming data-intensive, read-dominated, and generate massive bursts of metadata operations. The saturation of shared parallel file system metadata resources has become a critical issue for most HPC facilities. This talk will explore the underlying causes of these challenges and discuss innovative software-defined storage solutions being developed by our research group to address these persistent problems, ensuring sustained performance and I/O fairness.

How to Design a One-size-fits-all HPC Center OR
How to Make All Users Happy

Bernd Mohr, Division Head Application Support, Jülich Supercomputing Centre

May 21st, 2024 at 9 am PDT.
WATCH RECORDING

Designing a modern HPC center like a regional or national center is a taunting task. These centers are supposed to serve all needs and requirements of a large and diverse user community. This can range from highly scalable scientific simulation runs to complex workflows of a huge number of smaller tasks to specific data analytics and visualization requirements. This talk will present the Modular System Architecture (MSA) developed at Jülich Supercomputing Center and meanwhile in use on various sites all over Europe. It will introduce the MSA, and experiences using them in the design of three generations of Jülich HPC systems: JURECA, JUWELS, and the first Exascale system in Europe, JUPITER, which will be put into production in the coming 12 months.

Exploring High Performance Computing Paradigms Across the Computing Continuum

Silvio Rizzi, Argonne National Laboratory

April 30th, 2024 at 9 am PDT.
WATCH RECORDING

In this talk, we will discuss new computing paradigms that are emerging beyond traditional high-performance computing methods. Specifically, we will examine in situ visualization, which is increasingly crucial in exascale computing, and explore how these ideas can be leveraged in edge computing. Additionally, we will delve into the field of digital twin and laboratory automation, which are both highly active areas of research. Our talk aims to provide an informative overview while inspiring the audience to get involved in these exciting new developments across the computing continuum.

MScheduler: Leveraging Spot Instances for High-Performance Reservoir Simulation in the Cloud

Lúcia Drummond, Universidade Federal Fluminense

February 20th, 2024 at 9 am PDT.
WATCH RECORDING

Petroleum reservoir simulation uses computer models to predict fluid flow in porous media, aiding in forecasting oil production. Engineers execute numerous simulations with different geological realizations to refine the accuracy of the model. These experiments require considerable computational resources, which are not always available within the on-premises infrastructure. Commercial public cloud platforms can offer many advantages, such as virtually unlimited scalability and pay-per-use pricing. This paper introduces MScheduler, a meta scheduler framework for reservoir simulations at Petrobras, a Brazilian energy company. It efficiently executes jobs in the cloud, utilizing spot Virtual Machines (VMs) to reduce costs and ensure job completion even with VM termination. Contributions include a novel methodology for reservoir simulation checkpointing, a cost-based scheduler, and an analysis of the strategy using real production jobs from Petrobras.

Corralling the Computing Continuum: Tools and Applications for Usable Continuum Infrastructure

Matt Baughman, University of Chicago

January 23th, 2024 at 9 am PDT.
WATCH RECORDING

The keys to the future within the computing continuum will be usability and utility. The computing continuum describes a paradigm of transparent use of resources across a global compute fabric or grid. Some may argue this infrastructure already exists. However, the barrier to effective usage is creating tools on top of it that provide greater utility than the expense required to implement or use them. This means the computing continuum must be exceptionally easy to use or must provide exceptional value. The more you can do both, the better. In this talk, we will discuss some of the approaches we are taking to build tools for the computing continuum and then discuss some of the use cases already taking advantage of it.

Performance and Energy Optimization of Parallel Applications on Modern Multicore Architectures

Arthur Francisco Lorenzon, Federal University of Rio Grande do Sul

October 31th, 2023 at 9 am PDT.
WATCH RECORDING

Approximate Computing: Leveraging Imprecision to Speed up Computations

Gabriel Freytag, Federal University of Rio Grande do Sul

September 26th, 2023 at 9 am PDT.
WATCH RECORDING