I feel a bit odd writing this entry today, since our HHS secretary is busy tearing down the US vaccination system. But if you’re going to stay on the side of science and evidence, you have to go where the evidence takes you: it would be another betrayal of reason to pretend the Every Vaccine Everywhere works perfectly every time, because that isn’t the case. But I have to say up front - even though it should go without saying - that RFK Jr.’s own apparent positions (that vaccines are evil poisons at worst and at best are an unnecessary burden on Healthy Patriotic Americans) are spectacularly wrong-headed and absolutely will get people killed, many of them children.
But that said, vaccine effectiveness has to be evaluated in as clearheaded a way as possible, just as drug efficacy has to be. And there are plenty of confounding effects that you have to look out for! For example, if a vaccine or therapy is disproportionately administered to people who are in worse health, a cursory look at its efficacy can underestimate its effects (indication bias). And if such an intervention is disproportionately taken up by people who are in better health than usual, you can get an overestimate of its benefits through the “healthy vaccinee effect”.
Here’s a paper that examines both of these in the context of influenza vaccines. You can correct a bit more easily for indication bias (patients who are already sick or in poor health), but healthy-vaccinee bias is harder to deal with. The authors here found that the great majority of studies on influenza vaccine efficacy suffer from both problems, but note that overall “implausibly high mortality benefits of influenza vaccination have been observed, particularly in elderly persons”, which shows healthy-vaccinee bias at work. And this new paper, using statistics from Qatar, extends this to Covid-19 vaccination. (The paper, which is open-access, is also a good source of references on these issues).
Using that country’s national health system data, the authors match cohorts as closely as possible to generate data from people who received the vaccinations or did not. Qatar is small enough to be essentially a city-state, and 90% of its population are expatriates from a very long list of other countries. As the authors note, combinations of nationality, age, and sex are strong proxies for overall socioeconomic status there, which is a delicate way to put it. But all this let you do some pretty strong cohort analysis.
And even with great care taken to match the patients up (age and other demographics, underlying health problems, etc.), the first six months after vaccination showed a very large positive effect on mortality and morbidity. Too large! The vaccinated cohort had a 65% decrease in the likelihood of dying during that period, and as pleasant as it might be to think that this was due to the vaccination, it’s just not believable. Looking over the subgroups, this effect was particularly strong in older patients or those with other vulnerabilities to the coronavirus. At the same time, a strong protective effect against the severe forms of Covid-19 was really observed (which is good!), but this waned over a period of months, as has been seen in other studies.
The most likely explanation for the early-and-strong healthy vaccinee effect is lower vaccination rates among people who were seriously ill and/or near the end of their lives, and among elderly patients who were less mobile in general and did not go to the vaccination sites. Qatar’s population is mostly heathy working-age adults, so these groups really did seem to have a strong effect on the data. The authors believe that this is a big enough problem that such patients should probably be excluded from the data sets when trying to estimate vaccine effectiveness from observational data in general. It also argues that we should try harder to distinguish mild and severe forms of the diseases being vaccinated against, and to confirm the presence of the relevant pathogens by testing rather than just taking population-level data as it comes.
But at the same time, we have to keep in mind that the childhood vaccinations under attack from the current administration are a rather different case than the population-wide coronavirus or influenza vaccines. Children and infants are a less heterogeneous group than adults-of-all-ages, and diseases like measles are very hard to misdiagnose or to miss entirely. Likewise, the protective effects of the child vaccines tend to be very strong indeed. The statistics on childhood vaccination show unmistakeable strong benefits, so don't let someone try to talk you out of them because they read an article on the healthy vaccinee effect. Particularly this one!