Skip to main content

How to Lie With Statistics and Bad Study Design, Mask Version

By September 2, 2021Commentary

This new study supposedly showing benefits of community masking has been touted for showing mask efficacy and pilloried by true scientists for its abysmal design and statistical methods.  To me, the level of garbage in the study is evident by two basic factors, which I will get to momentarily, but I will go through the whole thing rather than just dismissing it, which is all it deserves.  I will do that to fully demonstrate the desperation of the mask religionists to find anything that supports mask effectiveness at stopping community spread of CV-19.   That last part is the critical issue.  In a world in which real science was respected, this preprint would get laughed out of peer review, but we will see what happens.  (Mask Study)

First, note the bizarre organization that published the “study”.  That is a head-scratcher.  Couldn’t get Medrxiv to take it?   Second, note the corresponding authors.  They are not infectious disease researchers or epidemiologists.  As far as I can tell they are economists.  Now economists do a lot of good research, but you gotta believe there is an agenda at work here.  Third, just look at the primary description of the results, look at carefully by age group.  That is all I needed to see to know this is garbage.  Apparently CV-19 is so smart that it can tell how old a person wearing a mask is and decide whether or not to infect them based on age.  When you see that difference in effect by age group, you know some very serious confounding variable is at work.  Fourth, look at the confidence intervals, always a dead give away about how good a study’s results are.  Not just wide, but Pacific Ocean wide.

So what was this study that so clearly shows how effective masks are, in a galaxy far, far away?  According to the researchers it was a “A randomized-trial of community-level mask promotion in rural Bangladesh during COVID-19 shows that the intervention tripled mask usage and reduced symptomatic SARS-CoV-2 infections”.   Really.  Now why these guys picked on poor Bangladesh to make a laughing stock of in the research world is beyond me, but they decided to see if they could encourage mask-wearing in rural areas of the country, because of course they were already convinced mask-wearing would limit CV-19 transmission.  I would encourage you to go to Worldometers and look at Bangladesh and tell me if those case curves look like masks made much difference, or really look at any country.  Nothing like promoting an intervention because you are sure it will work.

In any event they supposedly randomly chose certain of 600 villages to receive all the encouragement to wear masks and other villages were not so fortunate.  Then they supposedly tested to see if masks reduced cases.    The actual intervention was providing free cloth or surgical masks, modeling how to use and wear them, and giving people weekly reminders about mask-wearing.  There was a bewildering complex set of variations of the intervention, none of those variations appeared to have any differential effect.  The control group of the experiment got none of those interventions, but you can safely assume that there were other national and local mitigation efforts going on at the same time.   More confounders.   And here is an important sentence:  “Neither participants nor field staff were blinded to intervention assignment.”  In other words, the villagers and the people running the experiment knew which arm they were in.  Control and intervention villages were as close as 2 kilometers, or a mile and a half–think there might be any discussion among people in different villages.  All that affects behavior and results.  You are beginning to see what a pathetic design this was.

The study period was supposedly November 2020 to April of 2021, but the intervention lasted 8 weeks as the intervention was rolled out in waves.  We are never told if there were different results during different waves.  It might be important to look at the overall case curve in the country during that period to see if that might have influenced the outcome and maybe should be taken into account.  If you do that you will see that transmission was very low during the entire study period except for the period near the very end, so different waves of intervention faced different spread parameters.  Another potential confounder.  Outcomes included proper mask wearing, physical distancing, seroprevalence and reporting symptoms consistent with CV-19.  Oh, so you are trying to measure multiple outcomes which likely interact with each other?  And you are measuring proper mask wearing and physical distancing by occasional direct observation by the staff running the experiment–I am sure that is a rigorous method to gather evidence; no risk that the observation is occurring at different times and places in different villages.  No effort at all to actually track which individuals were and weren’t wearing masks and then see how many of each became infected.  And surveying people about their symptoms which might be CV-19, if you could reach them, and then testing them for antibodies; that sure won’t have any selection issues associated with it.  All of these design and execution issues become important, because of the incredible weakness of the results.  Even with stronger results, the credibility would be undermined.

The claim is that mask-wearing went from 13.3% in control villages to 42.3% in intervention ones.  I would not call that a particularly successful intervention.  And if masks are supposed to be source control as much as protection for the wearer, you aren’t getting much of either at 42%.  Nonetheless, keep in mind that public mask-wearing was supposedly tripled in the intervention villages.  So maybe we will see three times fewer cases, since masks as we all know are as good or better than vaccines.  Physical distancing was 24% of the time in control villages and 29% in intervention ones, more potential confounding–was it the masks or the distancing that led to that staggering reduction in infections.

And now we get to the really fun part–the very selective cherrypicking of supposed results.  Reporting of symptoms was almost indistinguishable in the groups.  First of all, a very small number of the total villagers involved in the study were even reached.  Out of those 7.62% in the intervention group and 8.62% in the control group reported symptoms.  Statistically indistinguishable.  Then some of these people were tested for antibodies.  Think there might be selection issues in who agreed to be tested?  And we aren’t initially told the actually level of people with positive tests, just that the difference in seroprevalence was 9.3%, that sounds maybe pretty good, well not really.  But it is misleading, because only .76% of the symptomatic group was positive for antibodies in the control group and .68% in the intervention group.  That is a miniscule difference in light of all the confounders and in absolute terms and it certainly isn’t a three times difference, in fact the confidence intervals overlap.  You may be getting a sense of why I call this study complete and utter garbage.  You have no idea how representative or complete your sample of symptomatics is and you have no idea how representative those who agreed to an antibody test are and you really have no clue whether your “intervention” made any difference.  Now they highlight that the effect was higher with surgical masks, and fail to say that there was no effect with cloth masks.  And it appears that social distancing was actually more effective than masks, so there is a substantial confounding effect.

My favorite is the fact that the intervention showed absolutely no impact on those under 50 years of age and only the most marginal effect on older groups.  There is no possible explanation for this other than an extremely discerning virus, capable of intuiting the age of various mask and non-mask wearers.  And we have no data on whether the increased mask wearing in the intervention group was concentrated among older persons, it not, then the age stratification of supposed effect makes no sense.   The confidence interval, which as usual basically means that if you repeated the experiment 100 times you would be likely to get a result in the range 95 of those times, was so wide that it ranged from the intervention actually leading to more cases to a much higher positive effect.  Okay, I could go on and on, and nothing I say is going to keep the media and the public health “experts” oriented toward terrorism from claiming that the study shows masks work.  It doesn’t.  What it clearly does show is that cloth masks made zero difference, and surgical masks had a completely confounded and incredibly small potential effect only in the population over 60.


Join the discussion 14 Comments

  • Pianoman says:

    The other day I saw a 20-something woman crossing the sidewalk wearing a mask. There was nobody near her, she was OUTSIDE … and yet, she was wearing a mask.

    It’s religious at this point. It’s the 21st century lucky rabbit’s foot.

  • Jenne says:

    It would be interesting to know which 600 villages participated in the study, and follow up by collecting seroprevalence data from those villages and compare it to random seroprevalence from another 600 villages in Bangladesh.

    • Kevin Roche says:

      yes, one of the criticism of the study that I read elsewhere was that they did not properly establish a baseline anywhere of the prevalence before the intervention, which would be standard procedure if you are supposedly measuring a change

  • J. Thomas says:

    How do you compete with the propaganda bombs being sent to every local rag by agencies such as this? Do pediatricians care about children’s mental health and learning and development, or just the Covid boogeyman ! I guess the only silver lining is that it’s in the ‘Opinion” section.

  • Chuck says:

    Google “mask study from Bangladesh”… The results yield many news stories promoting this study as gospel for the almighty mask. At least I didn’t find the CDC promoting the study? Yet?

  • Thomas L Heller says:

    Lies, damn lies and statistics. Whether it’s so-called global warming, race relations, public education, crime data, efficacy of transfer payments (welfare), etc., the Left pushes its agenda by any means necessary. COVID was about public health for a brief time, but for over a year it’s nothing more than a political tool and becasue of that public health suffers. And just for clarity, the Left is not only inhabited by Democrats.

  • Bill in Seminole says:

    Off-topic but would be interested in your thoughts on COVID booster shots. Only a few weeks ago the news media was telling us that booster shots would be given to anyone that wants them, starting September 20th. Today, I read that the FDA has backed off (somewhat). I also have read where the vaccine-maker, Pfizer, believes a booster shot will be needed.

    Given the apparent overage of supply vs. demand, and Pfizer’s position, why is the FDA stalling?

    • Kevin Roche says:

      I am not a big fan of boosters right now. No data on effectiveness other than a couple of very early Israeli studies and some studies in very immunocompromised groups. And no data on any potential safety issues. The real question it seems to me is that if the vaccines are lessening in effectiveness against serious disease, it might be better to design a better vaccine instead of subjecting people to never-ending boosters.

  • Ann in L.A. says:

    Every single argument for masking or vaccinating all kids applies every single year to influenza. Japan once required mandatory flu vaccines for kids, and they estimated it saved one life for every 420 kids vaccinated. If we go down this road, vaccine shots would have to be required for kids every single year, and masks would have to be permanent.

    Source: doi: 10.1056/NEJM200103223441204

  • Jack Perry says:

    Out of curiosity, how would you blind participants &c. in a study on wearing masks? Is that even possible?

  • Dan says:

    “ Now economists do a lot of good research”. You are either joking or you don’t know that the FED is controlled by PhD economists who think Continually money is always good because that’s what their models say.

  • Don. says:

    Were there any mask effectiveness studies before C-19?

Leave a comment