Back to ResourcesCase Study
Elastic compute capacity with Intel® Xeon® Scalable processors drives advances in weather modeling and prediction for the National Oceanic and Atmospheric Administration (NOAA)
As one of the world's preeminent weather forecasting organizations, NOAA needs ever-increasing HPC capacity to advance its numerical weather prediction models. NOAA is developing and prototyping its next-generation Rapid Refresh Forecast System (RRFS) on Intel Xeon Scalable processor-based cloud instances at Amazon Web Services (AWS).
Intel HPC solutions are designed to meet the volume, performance, and throughput requirements of workloads such as NOAA's. Intel on AWS allows for granular provisioning of clusters for diverse workloads. Under General Dynamics Information Technology's (GDIT) prime Research and Development HPC System (RDHPCS) contract with NOAA, Parallel Works middleware streamlines management of NOAA's cloud-based resources.
Together, this collaborative effort helps NOAA speed time-to-science while remaining within budget constraints.
From floods to fires, climate change is making extreme weather events more frequent and the consequences more devastating. Climate change also makes timely, accurate forecasts—and the high-performance computers that enable them—more important than ever. High-quality forecasts provide a basis for planning and decision-making, helping save lives and property and enhancing the sustainability of businesses, communities, and regions.
NOAA develops and runs the mathematical weather prediction models that guide U.S. meteorologists in creating their forecasts. NOAA, which includes the National Weather Service, has a mission to make the United States a weather-ready nation—one with the resilience to handle high-impact weather events. NOAA scientists say that increases in HPC capacity are crucial for advancing their weather models and achieving the mission.
"HPC and numerical weather prediction have been attached at the hip since the inception of weather modeling over 50 years ago. The quality of forecasts is strongly correlated with the rise in HPC performance. Having more HPC resources available allows for us to continually introduce scientific advances, ultimately leading to more timely and accurate forecasts."
— Dr. Jacob Carley, Physical Scientist, NOAA's Environmental Modeling Center (EMC)
Carley is part of a team developing and prototyping the next generation of NOAA's Rapid Refresh Forecast System (RRFS). The RRFS model takes in new weather data and runs multiple times each day, focusing on severe and hazardous weather. It's one of more than 20 models EMC creates and implements.
The RRFS team started with a long list of capabilities it wanted to implement. Many aimed at taking advantage of higher compute performance to improve the model's ability to predict severe weather. Enhancements include expanding the model's geographic coverage and increasing the number of ensemble members (or alternate weather scenarios).
Scientists were also excited by promising algorithmic approaches that would add more complex physics to the weather simulations and increase forecast accuracy for severe and hazardous storm systems.
But NOAA's HPC clusters are in high demand. Developing, prototyping, and testing RRFS would require significantly more resources than developers had available to them through NOAA's on-premises HPC systems.
Like most research organizations, NOAA is always working to speed innovations through its research-to-operations (R2O) pipeline. To gain the HPC capacity needed for developing RRFS in a timely fashion, NOAA turned to the cloud. While some of NOAA's forecasting models have run in the cloud before, RRFS would be a test case for building a major model in the cloud.
"There's a growing sense of excitement at NOAA around cloud computing. RRFS is a way to explore cloud capabilities while prototyping NOAA's next-generation, high-resolution system for predicting severe weather."
— Dr. Jacob Carley
"The cloud gives us flexibility to try new algorithms and approaches, which helps us make RRFS more powerful and accelerate its development…We're shaving months or years off the development schedule."
— James Abeles, Computational Scientist, NOAA Affiliate
NOAA is developing RRFS on Amazon Web Services EC2 instances, which offer options to run on 1st, 2nd, and as of Q2 2021, 3rd Generation Intel® Xeon® Scalable processors. AWS has more than 250 EC2 instance types and sizes across its global footprint featuring Intel processors, FPGAs, and other accelerators and technologies. AWS and Intel work closely to deliver offerings tailored to applications such as massively scalable data analytics, artificial intelligence, and the Internet of Things, as well as HPC.
The RRFS team chose Intel processors because its members had extensive experience with Intel architecture and a high degree of confidence in the technologies. They saw value in being consistent with NOAA's in-house infrastructure, much of which runs on Intel platforms.
"We're very familiar with Intel technologies. We know how to use the Intel Message Passing Interface and the compiler options to get the best performance."
— Rajendra Panda, Computational Scientist, EMC
NOAA uses AWS EC2 instance types based on Intel® Xeon® Platinum processors:
These HPC-oriented AWS instances enable NOAA's codes to take transparent advantage of Intel® Advanced Vector Extensions-512 (Intel® AVX-512) and Intel® Turbo Boost Technology. Intel AVX-512 instructions accelerate performance for a range of scientific, artificial intelligence, and other workloads. Intel Turbo Boost Technology optimizes CPU performance while balancing power and temperature limits.
"Cloud vendors make it easy to explore new technologies and see how they will benefit our applications. This is a huge benefit in helping us understand what types of technologies we should use in the future and then deploy them quickly."
— Rajendra Panda
Intel on AWS increases solution performance by providing rapid access to the latest Intel HPC innovations.
NOAA's cloud strategy also provides the flexibility to match workloads to the most suitable resources—something that's not always feasible with in-house infrastructure.
"Our on-premises systems tend to be fairly homogeneous, but not every application can use the resources efficiently. With the cloud, we optimize our cost-performance by allocating different workflow elements to the AWS instance types that run them most efficiently."
— James Abeles
To gain the portability for moving its research workflows between its supercomputers and multiple cloud service providers (CSPs), NOAA deployed Parallel Works hybrid-cloud workflow-management software.
"Parallel Works is a web-based access platform that makes it easy to launch a parallel cluster and submit jobs on any supported cloud. We use the same authentication to access the cloud that we do for accessing our in-house systems, which are a great convenience."
— Rajendra Panda
GDIT, as the prime systems integrator for NOAA's RDHPC contract, managed the Parallel Works deployment and played an essential role in this joint effort. GDIT is a leading systems integrator for the federal government and has supported the U.S. Department of Commerce NOAA RDHPCS effort since 2010.
GDIT evaluated multiple cloud HPC software frameworks for NOAA and selected Parallel Works to create NOAA's hybrid cloud framework. GDIT teams collaborate closely with NOAA, Intel, AWS, Microsoft, and Google to achieve supercomputing performance levels for NOAA's key weather application codes on all three clouds.
Parallel Works software also simplifies cost reporting and budget management controls for the RRFS cloud environment.
"We have a lot going on in the cloud. We have multiple compute instances. We store large amounts of data on the cloud, and we move data back and forth between our on-premises and cloud platforms. We also bring publicly available data from the internet onto the cloud each day. Parallel Works' cost-reporting system helps us track our resource usage, which helps us use our resources efficiently and remain within our budget."
— Rajendra Panda
The ability to manage its scientists' cloud HPC budgets on a per-project basis was an important requirement for NOAA. The Parallel Works software supports all three major CSPs, and other NOAA teams use the platform to access Intel instances on Google Cloud Platform (GCP) and Microsoft's Azure Cloud.
This transparent multi-cloud capability is key to maintaining a cost-effective, CSP-agnostic environment and making NOAA's vast troves of data available to the public at no cost.
GDIT and Parallel Works developed a baseline HPC configuration optimized for performance using Intel processors on AWS, Azure, and GCP. They based the configuration on a NOAA Finite-Volume Cubed-Sphere, Dynamical Core Global Forecast System (FV3GFS) benchmark application. NOAA scientists calibrated the platform to run scientific models such as FFaIR and the Hazardous Weather Testbed (HWT).
With the elasticity of the AWS cloud and the performance of the Intel Xeon Scalable processors, NOAA has the capacity it needs to significantly expand the RRFS model.
"Having more computational capacity lets us do higher-resolution forecasts and perform a higher volume of experiments. This allows us to get results and make improvements to the RRFS quickly without sacrificing scientific rigor. It all adds up to being able to make more accurate predictions."
— Environmental Modeling Center
The RRFS is data rich and computationally intensive.
"We model the atmosphere in 65 horizontal layers or slabs, and each slab has about 9 million grid cells. We run nine members in an ensemble approach to better sample and capture the uncertainties in our forecasts. It's a very large computational problem."
— James Abeles
"With the elastic capacity and high performance of the cloud, we could run and test the prototype model in real time and do exciting, new science at NOAA's flagship testbeds. This collaboration will accelerate our pace of development leading toward a high-quality operational forecast system in the coming years."
— Dr. Jacob Carley
NOAA's HPC cloud, powered by Intel technologies and Parallel Works software, makes it all possible—and it is improving the project's R2O effort.
"Our HPC cloud lets us scale up elastically to develop and test this system in ways we couldn't otherwise, because the problem size is so large. The cloud gives us flexibility to try new algorithms and approaches, which helps us make RRFS more powerful and accelerate its development. We can test the system in a more complete way. We can move faster. We're shaving months or years off the development schedule."
— James Abeles
The prototype version of the new RRFS model has already been used with several of NOAA's flagship testbed environments and has shown significant results. Forecasters used the prototype in NOAA's 2021 Hazardous Weather Testbed Spring Forecast experiment and FFaIR experiment and characterized both as great successes.
The new RRFS model is planned for transition to operations at the National Weather Service around late 2023. While the model will initially run on NOAA's in-house supercomputers, its development is providing data and wisdom to inform future decisions about operational forecasting in the cloud.
NOAA is looking to move other applications to the cloud. NOAA is also exploring the use of artificial intelligence to improve weather models and forecasting. Here again, NOAA scientists see value in the cloud.
"NOAA typically keeps its on-premises systems for five years or longer. Cloud vendors make it easy to explore new technologies and see how they can benefit our applications. This is a huge benefit in helping us understand what types of technologies we should use in the future and then deploy them quickly."
— Rajendra Panda
Meanwhile, RRFS is an important step toward fulfilling NOAA's vision of weather readiness.
"In an environment with more impactful hazardous weather, where communities may be especially vulnerable, it's imperative that the new RRFS provide better forecasts. Our mission is to protect lives and property and enhance the national economy. Early and accurate weather forecast guidance from the RRFS is a significant step toward achieving our mission."
— Dr. Jacob Carley