The Complexity of Identifying Vaccine Effectiveness

A recent post from Trust the Evidence, the excellent substack written by the principals at the Center for Evidence-Based Medicine, is a good reminder of the complexity involved in calculating vaccine efficacy against infection, hospitalization or death.  While written in the context of the CV-19 vaccines, the caveats about properly identifying effectiveness would apply to any vaccine.  We ran into the issues discussed as part of the outstanding work Dave Dixon did on the relative rates of infection, hospitalization and death among vaccinated and unvaxed groups.  And we also noted that assumptions about population size in total and in each group were critical to figuring out relative rates.  In Minnesota, the state’s use of outdated population estimates made the vaccines appear more effective than they were.  (TTE Post)

The complications in calculating the effectiveness make various measures, like relative risk, fairly useless.  Better in my judgment to just compare relative rates in the groups.  The real problem, however, which is basically unsolveable, is identifying just what groups are being compared in an onmnipresent epidemic.  Everyone, literally everyone, was “exposed” at some point, meaning they were in an area where the virus was circulating–a room, a building, even outside at some event.  These exposures may or may not have resulted in a clinically relevant “infection”, meaning substantial numbers of virus were in the body, in cells and replicating and causing symptoms.  The exposures may have prompted lasting immune responses of various strengths.

THIS IS IMPORTANT–VACCINES DON’T STOP EXPOSURE, NO VACCINE CAN, BUT THEY CAN LIMIT INFECTION.  This is important because if you have very sensitive tests, you might pick up virus in a vaccinated person, but because of the immune response created by vaccination, any “infection” will be very limited.  I think it would be best if we only defined an infection as one in which the person has serious symptoms that impact their daily life–they feel sick enough to skip work or just want to rest in bed, are vomiting or in some distress.  Otherwise, we are under-valuing the protective effect of vaccines in regard to real disease, versus just exposure and some replication.  And the tests we were using, even before home testing became prevalent, were not of highest accuracy, with many false positives and the identification of sub-clinical “infections”.

At different points in the epidemic of the number of people who had actually been infected was estimated to be anywhere from two to ten times more than reported in the official number of cases.  Part way into the epidemic you start vaccinating people.  Some of those vaccinated had already been infected.  Some who were vaccinated subsequently became infected.  As is almost always the case, the immune response provoked by vaccination declines over time, so at some point the vaxed people essentially become unvaxed.  So if you are trying to understand vaccine efficacy at any point in time, you have to account for all the immune response-provoking events prior to the time period for which you are studying efficacy.  It simply is difficult to impossible to do with such a widespread virus and so many unreported and over-reported infections.  You might assume that unreported and over-reported infections are equally spread among the vaxed and unvaxed, but you don’t know that.

I explained all this in multiple posts on vaccine efficacy.  You had to identify every immune-provoking event in the various sub-groups, and when it occurred, and that was not done.  I will say that it appears to me that for a limited period of time the vaccines had some effect on limiting infections that produced serious symptoms, and for a longer time on preventing hospitalization and deaths.  But they were not nearly as effective as morons like the Incompetent Blowhard Little Timmy Walz and public health officials portrayed them to be.  And they certainly weren’t effective enough to either stop the epidemic or to justify forcing people to get vaccinated.