A Note of Apology For a Bad Tornado Forecast
I must admit, if you told me back then [2007-2008] that there'd be relatively minimal improvement in the false alarm rate for this regime over the next 20 years (and even that cutting-edge AI/ML methods would highlight them as big events), it would've surprised me.
I has surprised me, too. The NWS reached its peak of tornado warning quality from ~2005 to 2011 but is has only become apparent in the last couple of years that our forecasting ability ( the tornado "watches") has either peaked or flattened (I've requested the official accuracy statistics, but they are withholding them). I wish I knew why. I've even gone back to tornado forecasting checklists from the 1970's as a backup and they are not working.
My guess is there is something in the modern day computer models, or the way we are interpreting them, that is misleading us. Whatever it is, I'm going to do the best I can do to figure out the problem and make better forecasts going forward.
As always, thank you for reading my blog!
P.S. 1:50pm, Friday. One of the chasers objects to me calling the small tornado a gustnado. He says there was rotation above the whirling dust and published a photo on the storm chase discussion group. So, there was a tornado in a rural area near Guthrie, Oklahoma. I have changed the text above and I technically verified my (red) tornado forecast. That stated, it was still a lousy forecast because it overstated the risk.
Hi Mike!! Been a long time Sir! This is something I have really noticed over the last 5-7 years. Forecasting in general had dramatically fallen off from days 1-3. I don’t have any solid data, but just from my years of experience with you and other great meteorologists and forecasting in Kansas for over 35 years. I honestly believe that it comes down to whatever the new modeling algorithms or who is developing those, are not meteorologists or have no idea what they are doing. The models we used in the 90s and early 2000s were some much better and we could learn there pluses and minuses and could use them to make accurate forecasts. What is really concerning, and you are correct, the severe weather modeling and forecasting through SPC is embarrassing!! Four Moderate to High Risk forecasts this year in the Central Plains were a complete bust. I hear the confidence in the public everyday now and how their trust in the weather forecasts are gone.
ReplyDeleteI know we had really great education through school and through training, I don’t think the meteorologists coming out of school now know how to use real data, upper air, surface, trends….they rely on a model to tell them what to do. Which is sad. I hope it changes, but from what I’ve seen, it’s getting worse.
Take care Mike!
Thank you, Beau. Nice to hear from you and I hope you are doing well.
DeleteI feel the same way, always scaring people. It's the beginning to remind me of the boy always scaring crying wolf all the time it's beginning to wear nobody pays any attention.
DeleteI was a forecaster with the NWS for over 25 years, retiring at the end of 2017. To me you summarized the accuracy curve correctly. Before I retired I noticed forecasters being pushed to go with the models or MOS guidance over their experience with those models/MOS. Now, my local NWS office has gotten so bad, that I almost always do not trust a forecast beyond day 2! They also routinely get overnight low temperatures wrong by 5 or more degrees too warm in my area during frost and near frost events.
DeleteIt's not just the NWS. The local TV mets were having a tough time figuring out how to find rotation in a tropical storm tornado event during one of the tropical storms that went through here in September. I can't say that their regular weather forecasts are much better then the local NWS office. At least I can still do my own forecasting when necessary.
Would be very interested in your thoughts as to the future impact of AI upon meteorological forecasts and generally within the profession. Thanks.
ReplyDeleteI provide some information here: https://www.mikesmithenterprisesblog.com/2024/10/the-nws-is-falling-further-behind.html
DeleteIn general, I believe than in 5 to 15 years AI will be able to do much of the day-to-day forecasting.. That creates an opportunity and a huge problem. The fact is that human beings do not respond well to extreme conditions (aviation has learned this the hard way) unless they are engaged in an activity every day. The idea that we can allow AI to take over when it is clear and 70° but then have the human handle the extreme blizzard well is wrong. The crash of Asiana Airlines Flight 214 in San Francisco is an example of that issue.
Weather science has been resistant to bringing human factors study into our toolbox.
That must change or the introduction of AI could leader to disaster.
Thank you for your question.
If interested, a recommended look at the pitfalls and promise of AI is offered in the recently published book, "NEXUS: A Brief History of Information Networks from the Stone Age to AI"
ReplyDeletehttps://www.penguinrandomhouse.com/books/762444/nexus-by-yuval-noah-harari/