A Few More Observations on Modeling

I have occasionally been intemperate in my references to models and the modelers who build them.  That frustration is really directed at the poor communication that has occurred in regard to the models and how they should be used.  Models are important because they can provide information to guide policy-making.  But the modelers know better than anyone what the limitations of their creations are.  They have a special obligation to ensure that the outputs of their models doesn’t get misused.  They should be very clear when there is lack of good data to inform assumptions that go into model formulas that this will necessarily result in wide ranges of projected outcomes and a strong likelihood of the actual outcome being extremely different from those projected.  If you say, as the Minnesota modelers did, that we used very conservative assumptions, meaning those geared toward the worst case, you should make it clear that the midpoint of your range of scenarios is not then the most likely outcome; in fact the most likely outcome is probably at the low end of your range of projected outcomes.  Or you should run more scenarios with different plausible assumptions.  If you know, and we all do, that the press is going to pick the highest, most sensational outcomes, you insist that those not even be released.  And because you know that politicians will be fearful of anything that makes it look like they didn’t do everything they could to prevent even one death, you must be very careful how you present your forecasts to them and you must encourage them to be balanced in how they evaluate potential outcomes.  It is all about communication and so far that communication has been very poor.

Secondly, I have been very impressed with some of the comments and observations I have received from members of the public on the models.  I am not a statistician or mathematician, although I know enough to understand what I read.  There are a lot of people out there in the public who have very good ideas regarding improving model performance and I have encouraged them to communicate their observations to the modelers.  As with most things, the insights and work of a larger group is likely to produce better outcomes than that of an individual or small group.  So keep thinking about the issues and share what you think you have learned.