Easier accessibility promises scientific and economic benefits
NOAA has entered into separate multiyear contracts with AWS, Google Cloud, and Microsoft to provide the public with cloud-based access to the agency’s environmental data sets, in accordance with the agency’s full and open data policies. NOAA generates tens of terabytes of data every day from satellites, radars, ships, weather models, and other sources.
“NOAA’s wealth of world-class environmental data will now be more accessible through partnerships with commercial cloud providers, which will allow the agency to better manage a rapidly increasing volume of data going forward,” said Neil Jacobs, Ph.D., acting NOAA administrator. “Cloud-based storage and processing is the future. Not only will this improved accessibility enhance NOAA’s core mission to protect life and property, but it will also open up new and exciting areas of research at universities and significant market opportunities for the private sector.”
This is the latest milestone of the ongoing NOAA Big Data Project, which seeks to remove obstacles to public use of the agency’s data by combining three powerful resources: NOAA’s expansive collection of high-quality environmental data and expertise; the vast infrastructure and scalable computing capabilities of our industry collaborators; and the innovative energy of the American economy.
“This builds on AWS’s long standing support of NOAA and their mission, and we look forward to continuing our collaboration,” said Dave Levy, vice president, U.S. government at AWS. “As customers gather more data about Earth, cloud offers new opportunities to understand our world and contribute to sustainable future. With initiatives like the Big Data Project, researchers are leveraging AWS services to lower the barriers to affordable, on demand, and scalable data analysis. As a result, researchers are gaining greater access to NOAA data, and speeding up the time to discovery, at a fraction of the price and time previously required.”
“Technology is transforming how we understand our ever changing world,” said Kate Brandt, Google sustainability officer. “Through the NOAA Big Data Project, Google Cloud can help researchers, innovators, and organizations analyze data to tackle a range of environmental challenges — regardless of their size or computing power.”
“Improving the world around us starts with a better understanding of it, and pairing NOAA’s world-class data sets with world-class cloud computing power offers a big step forward,” said Dr. Lucas Joppa, chief environmental officer at Microsoft. “Microsoft is proud to further its work with NOAA with this contract, building on existing work including hosting NOAA’s largest and most valuable weather data on Azure Open Datasets and specific projects on camera trapping and hydrophones to more accurately count animal populations. We’re thrilled to see our technology play a role in democratizing access to accelerate scientific discovery and prioritize the protection of natural resources.”
The NOAA Big Data Project leverages operational public-private partnerships with the cloud computing and information services industries to avoid the costs and risks associated with data access services. Under the contracts, the cloud platform providers can charge for compute or other services related to additional processing of NOAA data, but are required to provide free and open access to the data itself.
“The NOAA Big Data Project, which is the first public-private partnership of its kind in the U.S. government, will help accelerate new areas of scientific and economic growth,” added Jacobs.
Today’s announcement is the culmination of NOAA’s efforts to broaden access to its data resources and facilitate full and open data access to foster innovation. Rather than utilize a traditional, large-value contract award, these contracts represent a unique relationship with the Cloud Service Providers: NOAA contributes its high-quality environmental information and scientific expertise, and the CSPs provide their cloud storage and technical capabilities, improving the public’s ability to use these data without being charged for cloud storage or data access.
“The Big Data Project’s Cloud Service Providers have shown incredible commitment to open data principles, and they clearly understand the value of NOAA’s data to their customers and to the Nation’s economy,” said Ed Kearns, Ph.D., acting chief data officer at the Department of Commerce.
Following an initial research phase that began in April 2015 and operated under Cooperative Research and Development Agreements, the contract awardees were selected through a Request for Proposal conducted in spring 2019. The NOAA Big Data Project is part of the Department of Commerce 2018-2022 Strategic Planoffsite link priority to reduce extreme weather impacts by improving access to NOAA’s data and strengthening partnerships with America’s weather industry and other members of the weather, water, and climate enterprise. The NOAA Big Data Project also supports the White House’s goal of Leveraging Data as a Strategic Asset, and incorporates many of the practices promoted by the new Federal Data Strategy.
Last month, the NOAA Big Data Project received a Best in Class 2019 Government Innovation Award in the category of Public Sector Innovation, which recognizes innovative ways that government applies technology to better meet its mission and serve the public.