Minnesota Model

I have finally worked my way through the materials provided in the version 3 release of the Minnesota epidemic model.  I will have at least a couple of longer, more technical posts on my impressions.  But I can give a few bigger picture comments now.

The model does a better job of lining up with clinical reality, but still has flaws.  It assumes that everyone can get infected, which isn’t consistent with experience.  It has far too low a rate of undetected infections, and that alone accounts for its wildly unrealistic death counts.  It sends some undetected infections to the hospital.  Uhhh, if a case is undetected, it is by definition asymptomatic, so how does the person go to the hospital?  It treats ventilator and ICU use as one thing, which is completely inconsistent with current care guidelines.   It doesn’t break out the residents of congregate living situations or in any manner treat them as a separate population either in the contact model or the formulas.

I know this is technical, but they try to parametize the formulas by fitting them to actual Minnesota experience, and they say they have an excellent fit, but then the model is projecting a number of deaths through the month of May, 1700, that simply isn’t going to happen.  So how can your model formulas be right, and why would you even bother to put them out with that number, when it so obviously is wrong.

The other subtle thing that would be easily missed, and the Governor scolded us all about this in his concession speech, is that social distancing has been less than projected under the model in version 2, so contacts have not been reduced as much as the model assumed.  More contacts means more infections.  If that was the case, we should have seen many more infections and ultimately deaths in the data.  We didn’t.

The answer to all these issues is obvious to me.  The “experts” used bad reasoning.  They assumed this epidemic would be like every other epidemic, particularly for respiratory illnesses.  It isn’t.  It has a different shape and course.  If you don’t “model” that course, you are of course going to end up with wildly inaccurate scenarios.

We have gotten the attention of some other statisticians.  William Briggs is an independent statistician who comments on a number of subjects.  Here is his perspective on the Minnesota model and on our Governor’s saying it shows social distancing works. (Briggs Post)   His point, which is basic but hard to grasp, is that models don’t tell you anything, especially the ones that are designed to forecast or project events in the future, they only show what the modelers tell them to show.  A model is just computer code embodying formulas and parameters all of which are set by the modeler.  A model doesn’t “discover” anything.  So the modelers in Minnesota, and elsewhere, are telling the model that social distancing and other mitigation measures reduce contacts and that this reduction of contacts means less infections, serious cases and deaths.  The modelers tell the model how much social distancing will change the number of those infections, etc.  Now the modelers may be basing those estimates on what they think are good data and good mathematical relationships, but that is the fundamental point–the modelers are telling the model what it should show, the model isn’t independently doing anything.

Finally, for this post, the state has actually been very responsive and transparent on data.  I had suggested changing the way they report cases and deaths by age group and while my suggestion probably had little to do with it, they are now giving us exactly the table I wanted.  And here is why that is important.  Let’s look at some Minnesota statistics.  People 29 years old and younger had 3184 out of 13,435 reported infections, or 24%.  They had zero deaths.  People 30 to 49 had 4777 of the infections or 35.5%.  They had nine deaths (all by the way, apparently had some underlying health condition).  There were 663 deaths in total.  So people under age 50 had 60% of infections and 1.3% of deaths.  Those 70 and older had 2034 infections, or 15% of the total.  They accounted for 548 of the deaths or 82%. If you look at these numbers on a per capita basis, the difference is even starker.   Come on, Governor, how much clearer can it be that this is only a serious issue for the elderly.

One more observation:  we are still seeing, on a daily basis, 80% or more of deaths coming from the long-term care residents.  How is it that the state just can’t seem to address this issue?  And that skewing really makes the failure of the models to deal with this population separately a massive flaw.