This project has ended | -
ESCAPE aimed to develop world-class, extreme-scale computing capabilities for European operational numerical weather prediction (NWP) and future climate models.
The biggest challenge for state-of-the-art NWP arises from the need to simulate complex physical phenomena within tight production schedules. Existing extreme-scale application software of weather and climate services is (i) not very efficient on existing CPU-type processors reaching only 5% of the peak performance, mostly due a lack of arithmetic intensity, and (ii) ill-equipped to adapt to rapidly evolving options for new processor hardware, mostly due to a lack of flexibility for mapping specific computational problems onto heterogeneous computing units. This problem is exacerbated by other drivers for hardware development that are not necessarily optimal for weather and climate simulations.
ESCAPE aimed to redress this imbalance through innovation actions that fundamentally reform Earth-system modelling. ESCAPE addressed the ETP4HPC Strategic Research Agenda 'Energy and resiliency' priority topic, developing a holistic understanding of energy-efficiency for extreme-scale applications using heterogeneous architectures, accelerators and special compute units. The three key steps towards much enhanced performance and energy-efficiency for weather and climate modelling are:
- Defining and encapsulating the fundamental algorithmic building blocks ('Weather & Climate Dwarfs') of weather and climate prediction models. This is the prerequisite for any subsequent co-design, optimization, and adaptation efforts.
- Combining ground-breaking frontier research on algorithm development for use in extreme-scale, high-performance computing applications aiming to minimize time- and cost-to-solution.
- Synthesizing the complementary skills of all project partners in all necessary domains, at the interface between applied and computational science. ECMWF and leading European regional forecasting consortia are teaming up with excellent university research and experienced high-performance computing centres, two world-leading hardware companies, and one European start-up SME, providing entirely new knowledge and technology to the field.
In its 36 months, the project developed and tested the concept of fundamental building blocks called dwarfs. Dwarfs represent functional units in the forecasting model, such as an advection or a physics parametrization scheme, which also come with specific computational patterns for processor memory access and data communication.
Assessing numerical methods and algorithms for dwarfs rather than entire models reduces the complexity of the code. It enables HPC centres, research groups and hardware vendors to focus on specific aspects of performance for which code restructuring and adaptation to novel processor architectures is more straightforward.
The resulting dwarfs were then adapted and optimised for different types of Intel CPU and NVIDIA GPU processors, and a new technique particularly suited for performing Fourier transformations with a novel optical device.
For spectral transforms on CPUs, efficiency gains of up to 40% were achieved. Code optimization for GPU delivered speed-up factors of about 10 to 50 on a single node, and again by a factor of 2 to 3 when deployed on multiple GPUs connected by the recently developed NVSwitch interconnect.
However, using accelerators only for a small part of the code destroys a lot of the benefit in terms of the overall cost if the CPUs are idle while the accelerators perform their computations Therefore, either moving a larger part of the code to the accelerator or overlapping computations on the host-CPUs with computations on the accelerator is much more beneficial. Fully implementing either or both options will require further work.
So-called domain specific languages (DSL) were another focus of the ESCAPE project. They are a very promising tool to enable good performance on multiple architectures while still having a single, portable code base. This is particularly relevant for the existing, rather monolithic weather and climate prediction codes for which manual code adaptation to different (and changing) architectures is not sustainable. When adapted to GPUs with a DSL, a dwarf calculating the advection of air showed a speed improvement by a factor of 2 compared to the manually adapted version. However, designing a DSL that is shared by many dwarfs, user-friendly whilst delivering good performance on each architecture is a challenge.
Beyond code adaptation and optimisation, a range of numerical methods exploiting multi-grid solvers and different types of spatial discretisation and time stepping have been investigated. This is highly relevant because sufficient efficiency gains cannot be obtained from adapting codes to specialised processors, but require a substantial investment in enhancing the arithmetic intensity that limits the exploitation of the computational power of any processor in the first place.
ESCAPE: Energy-efficient Scalable Algorithms for Weather Prediction at Exascale
About the project
ESCAPE will develop world-class, extreme-scale computing capabilities for European operational numerical weather prediction (NWP) and future climate models. The biggest challenge for state-of-the-art NWP arises from the need to simulate complex physical phenomena within tight production schedules.
Existing extreme-scale application software of weather and climate services is ill-equipped to adapt to the rapidly evolving hardware. This is exacerbated by other drivers for hardware development, with processor arrangements not necessarily optimal for weather and climate simulations. ESCAPE will redress this imbalance through innovation actions that fundamentally reform Earth system modelling.
ESCAPE addresses the ETP4HPC Strategic Research Agenda 'Energy and resiliency' priority topic, developing a holistic understanding of energy-efficiency for extreme-scale applications using heterogeneous architectures, accelerators and special compute units.
The three key reasons why this project will provide the necessary means to take a huge step forward in weather and climate modelling as well as interdisciplinary research on energy-efficient high-performance computing are:
- Defining and encapsulating the fundamental algorithmic building blocks ('Weather & Climate Dwarfs') underlying weather and climate services. This is the prerequisite for any subsequent co-design, optimization, and adaptation efforts.
- Combining ground-breaking frontier research on algorithm development for use in extreme-scale, high-performance computing applications, minimizing time- and cost-to-solution.
- Synthesizing the complementary skills of all project partners. ECMWF and leading European regional forecasting consortia are teaming up with excellent university research and experienced high-performance computing centres, two world-leading hardware companies, and one European start-up SME, providing entirely new knowledge and technology to the field.
ESCAPE website
For more information, please visit www.hpc-escape.eu (for project partners, please use the ESCAPE confluence page)
Related projects
ESCAPE partners
ESCAPE combines expertise from world-leading global and regional numerical weather prediction centres, academia, high-performance computing centres and hardware vendors.
European Centre for Medium-Range Weather Forecasts
|
|
Royal Meteorological Institute of Belgium (RMI)
|
Danish Meteorological Institute (DMI)
|
|
|
|
Irish Centre for High-End Computing (ICHEC)
|
Poznań Supercomputing and Networking Center (PSNC)
|
|
|
|