How Big Data Can Boost Weather Forecasting

In various locations across the globe, big data systems are being used to provide next-generation weather forecasting capabilities, which could prove essential as climatic systems become more volatile over the next decades.

The Upvoters that cared:
    This is a community post, untouched by our editors.
    Steve Hamm

    Steve Hamm

    By Steve Hamm, IBM Writer/Researcher

    Last September, when Typhoon Sanba smashed into the Korean peninsula, it packed winds so strong that they sent rocks flying through the air like missiles and caused massive power outages.“Hwangsa” storms, carrying dense clouds of yellow dust from China’s GobiDesert that are sometimes loaded with heavy metals and carcinogens, sweep across the peninsula from West to East.

    Menaced by such destructive weather phenomena, South Korea is upgrading its national weather information system with the goal of understanding weather patterns better and predicting better the location and ferocity of weather events. The upgrade being installed by the Korean Meteorological Administration increases the agency’s data storage capacity by nearly 1000% to 9.3 petabytes, making it Korea’s most capable storage system. IBM provides the storage hardware and software.

    Typhoon Sanba (Credit: NASA)

    Typhoon Sanba (Credit: NASA)

    The KMA project dramatically illustrates today’s big data phenomenon and its impact on weather forecasting. Thanks to the rapid spread of sensors and satellites, and to the increase in computer number-crunching speeds, it’s possible to forecast weather changes more accurately and with improved detail–potentially saving thousands of lives and safeguarding property.

    Increasing evidence of climate change worldwide is prompting governments and scientists to take action to protect people and property from its effects. But, to take effective action, they need to know understand a lot more about the weather–everything from what’s going to happen tomorrow to what’s coming next year. For instance, leaders of the city of Hoboken, N.J., in the United States, which flooded badly last fall during Hurricane Sandy, are considering building a wall around the city to keep the tidal Hudson River at bay, but, the problem is, if they don’t build high enough the wall could end up turning Hoboken into a giant bathtub rather than keeping rising waters out.

    IBM Research scientists are taking the lead in bringing the most sophisticated data analytics to bear on weather forecasting. Their long-term weather analysis project, called Deep Thunder, combines data with sophisticated mathematical algorithms and computing power.

    The scientists established at test bed in the New York City metropolitan area, where they set up a three-dimensional grid of thousands of blocks. That makes it possible for them to run calculations that produce very precise weather forecasts for a particular locale. Using this capability, the team was able to predict with remarkable accuracy the snowfall totals in New York City during the mammoth snow storm that blanked the northeastern United States in February–and also to predict accurately when the snowfall would start and stop.

    The IBM Research team is putting their algorithms to work on behalf of cities around the world. For instance, Rio de Janeiro, because of its climate and terrain, has recurring flooding and landslide problems in many hilly neighborhoods. The researchers used data describing the physics of the atmosphere to create a mathematical model of how storms are likely to unfold in Rio. With it, they can predict up to 40 hours ahead of time how much rain will fall in a particular location—with 90% accuracy.

    In recent months, the Deep Thunder team, lead by Lloyd Treinish, has developed new techniques for ingesting many more measurements from weather sensors. The team is also extending its technology to new applications, including agriculture and wind farming.

    For detailed and super-accurate weather information to have maximum impact, it has to be accessible by a large number of people. That’s why we have created iPad and cloud applications that deliver the power of Deep Thunder to people’s hands wherever they may be. Hopefully, by the time Rio hosts the summer Olympics in 2016, practically everybody who attends will be able to get their hands on Deep Thunder data so they know exactly what to expect when they venture out to the games.

    ______________________________________________

    This story first appeared on the Smarter Planet blog.