The return of spring in the Northern Hemisphere ushers in tornado season. A twisting funnel of dust and debris from a tornado seems like an unmistakable sight. But that view can be obscured from radar, a tool for meteorologists. It’s difficult to say exactly when a tornado formed or even why.
The recent dataset may contain answers. It contains radar data from thousands of tornadoes that have hit the United States over the past 10 years. Storms that produce tornadoes are accompanied by other severe storms, some with almost identical conditions, that have never occurred. Researchers from MIT Lincoln Laboratory who curated the data set, the so-called TorNet, they have now made it open source. They hope to enable a breakthrough in detecting one of nature’s most mysterious and violent phenomena.
“Much progress comes from easily accessible, comparative data sets. We hope that TorNet will lay the foundation for machine learning algorithms for both detecting and predicting tornadoes,” says Mark Veillette, co-principal investigator of the project with James Kurdzo. Both researchers work in the Air Traffic Control Systems Team.
Along with the dataset, the team releases the models trained on it. The models show promise in machine learning’s ability to detect the twister. Building on this work could open recent frontiers for forecasters, helping them provide more right warnings that could save lives.
Swirling uncertainty
Approximately 1,200 tornadoes occur in the United States each year, causing millions or even billions of dollars in damage. economic damage and causing an average of 71 fatalities. Last year there was one unusual one long-lasting tornado killed 17 people and injured at least 165 others along a 59-mile route in Mississippi.
However, tornadoes are extremely arduous to predict because scientists don’t have a clear picture of why they form. “We are seeing two identical storms, one of which will produce a tornado and the other will not. We don’t fully understand it,” says Kurdzo.
The basic ingredients of a tornado are thunderstorms with instability caused by rapidly rising toasty air and wind shear causing rotation. The primary tool used to monitor these conditions is weather radar. However, tornadoes lie too low to be detected even when they are moderately close to radar. As a radar beam at a given angle of inclination moves away from the antenna, it rises higher above the ground, seeing mainly reflections of rain and hail carried by a “mesocyclone,” a broad, rotating updraft of a storm. A mesocyclone does not always produce a tornado.
With this circumscribed view, forecasters must decide whether to issue a tornado warning. They are often cautious. As a result, the false alarm rate for tornado warnings is over 70 percent. “This can lead to the boy-cries-wolf syndrome,” says Kurdzo.
In recent years, researchers have begun using machine learning to better detect and predict tornadoes. However, the raw datasets and models were not always available to the wider community, hampering progress. TorNet fills this gap.
The dataset contains over 200,000 radar images, 13,587 of which depict tornadoes. The remaining photos do not show tornadoes, they are from storms that fall into one of two categories: randomly selected severe storms or false alarm storms (those that prompted the forecaster to issue a warning but did not produce a tornado).
Each storm or tornado sample consists of two sets of six radar images. Both sets correspond to different radar deflection angles. The six images show different radar data products, such as reflectance (showing rainfall intensity) or radial velocity (indicating whether winds are heading toward or away from the radar).
The challenge in organizing the dataset was finding the tornadoes first. Tornadoes are extremely occasional events in the weather radar dataset. The team then had to balance the tornado samples with arduous non-tornado samples. If the dataset was made too basic, for example by comparing tornadoes to snowstorms, the algorithm trained on the data would likely reclassify the storms as tornadoes.
“The beautiful thing about a real benchmark data set is that we’re all working with the same data, at the same level of difficulty, and we can compare the results,” Veillette says. “This makes meteorology more accessible to data scientists and vice versa. It is easier for both sides to work on a common problem.”
Both researchers represent the progress that can result from mutual cooperation. Veillette is a mathematician and algorithm developer who has long been fascinated by tornadoes. Kurdzo is a trained meteorologist and an expert in signal processing. In college, he chased tornadoes with custom-built mobile radars, collecting data for analysis in recent ways.
“This dataset also means that a graduate student doesn’t have to spend a year or two building a dataset. They can start their research immediately,” says Kurdzo.
This project was funded by Lincoln Laboratory Climate Change Initiativewhich aims to leverage the laboratory’s diverse technical strengths to lend a hand address climate challenges that threaten human health and global security.
Finding answers with deep learning
Using the dataset, researchers developed basic artificial intelligence (AI) models. They were particularly interested in using deep learning, a form of machine learning that specializes in processing visual data. Deep learning itself can extract features (key observations that the algorithm uses to make decisions) from images across the entire dataset. Other machine learning approaches require humans to manually label features.
“We wanted to see if deep learning could rediscover what people typically look for in tornadoes, or even identify new things that forecasters don’t typically look for,” Veillette says.
The results are promising. Their deep learning model performed similarly to or better than all tornado detection algorithms known in the literature. The trained algorithm correctly classified 50 percent of the weaker EF-1 tornadoes and more than 85 percent of the tornadoes rated EF-2 or higher, representing the most destructive and costly occurrences of these storms.
They also evaluated two other types of machine learning models and one time-honored model for comparison. The source code and parameters of all these models are publicly available. The models and data set are also described in wa paper submitted to the journal of the American Meteorological Society (AMS). Veillette presented this work at the AMS annual meeting in January.
“The biggest reason for sharing our models is for the community to improve them and do other great things,” says Kurdzo. “The best solution may be a deep learning model, but someone may decide that a model without deep learning is actually better.”
TorNet may be useful in the weather community for other applications as well, such as conducting large-scale case studies of storms. It can also be extended to include other data sources such as satellite imagery or lightning maps. Combining multiple types of data can improve the accuracy of machine learning models.
Taking steps towards surgery
In addition to detecting tornadoes, Kurdzo hopes the models will lend a hand uncover knowledge about what causes them to form.
“As scientists, we see all of these tornado precursors – low-level spin increases, hook echoes in reflection data, specific differential phase (KDP) rates, and differential reflection (ZDR) arcs. But how does it all come together? Are there physical symptoms that we don’t know about?” He’s asking.
Coming up with these answers may be possible with easy-to-explain AI. Explainable AI refers to methods that allow a model to provide a rationale, in a human-readable format, for why it made a particular decision. In this case, these explanations may reveal the physical processes that occur before tornadoes. This knowledge can lend a hand train forecasters and models to recognize signs faster.
“None of these technologies will replace a fortune teller. But perhaps one day it will be able to direct forecasters’ eyes in intricate situations and give a visual warning to an area where tornadoes are predicted to occur,” Kurdzo says.
Such assistance could be particularly useful as radar technology improves and future networks potentially become more dense. The next-generation radar network’s data refresh rate is expected to boost from every five minutes to about one minute, perhaps faster than forecasters can interpret the recent information. Because deep learning can quickly process huge amounts of data, it may be well suited to monitoring radar signals in real time, alongside humans. Tornadoes can form and disappear in a matter of minutes.
But the road to an operational algorithm is long, especially in safety-critical situations, Veillette says. “I think the forecasting community is still, understandably, skeptical of machine learning. One way to ensure trust and transparency is to have public benchmarking datasets like this one. This is the first step.”
The team hopes that the next steps will be taken by researchers from around the world, inspired by the data set and energized to create their own algorithms. These algorithms will, in turn, go to testbeds, where they will ultimately be shown to forecasters to begin the implementation process.
Ultimately, the path may return to trust.
“With these tools, we may never receive a tornado warning for more than 10 to 15 minutes. But if we could lower the false positive rate, we could start to make progress in public perception,” Kurdzo says. “People are going to use these warnings to take the actions necessary to save lives.”