What VAERS Can and Can’t Tell Us About COVID Vaccine Safety

Why this BMJ story needs a closer read

The striking claim in The BMJ’s new investigation is that US officials knew a COVID vaccine safety monitoring tool was flawed and blocked efforts to repair it. That framing is tailor-made for an antivaccine cover-up narrative. But the article’s account, as presented here, rests on a more complicated dispute: how two statistical methods were used to detect possible safety signals in a passive reporting system. David Willman’s article,

“US officials knew COVID vaccine safety system was flawed—but turned aside the FDA doctor who alerted them”, focuses largely on myocarditis after vaccination, especially among young men. It asks whether US agencies recognized that risk as quickly as other countries, including Israel. There’s a real question here about pharmacovigilance. The evidence described in the fragment, though, doesn’t by itself establish a deliberate cover-up or gross negligence. The reaction to VAERS figures is a useful example of how the framing can mislead. One widely shared post called it “Unbelievably negligent if true” that more than 90% of 1.1 million reports in the first year concerned mRNA COVID shots. Yet COVID vaccines were being administered at enormous scale, under extraordinary public and media scrutiny. CDC and FDA promoted VAERS, health professionals had specific reporting requirements, and the government also introduced V-Safe, which regularly asked vaccinated people about symptoms. More reports were therefore expected. The raw total can’t show whether a vaccine caused more harm; the relevant question is whether particular events appeared disproportionately, given exposure and suitable comparisons.

Unbelievably negligent if true: "The problem arose because, in the first year of the rollout, almost all the reports coming into VAERS—more than 90% of 1.1
million—were for the mRNA covid shots, dwarfing vaccines for all other diseases. This meant that analysing the safety of >> https://t.co/TsCAT6DSiK

— Bram Bakker (@bramkoers_tweet) September 17, 2026

VAERS is an early-warning system, not a way to prove causation. Anyone can report a health problem after any vaccination, which makes the database useful for generating leads but vulnerable to false positives and misuse. Antivaccine advocates have long treated its reports as proof that vaccines caused whatever followed. The author has written about that pattern for years, including [the continued misuse of VAERS at the end of 2021](https://sciencebasedmedicine.org/as-2021-shambles-to-a-close-the-misuse-of-vaers-by-antivaxxers-continues-apace/) and [how the tactic was rapidly amplified after COVID vaccine rollout](https://sciencebasedmedicine.org/covid-19-vaccines-as-dangerous-continue-apace-vaers-edition/). Further background is available in the [VAERS archive](https://sciencebasedmedicine.org/tag/vaccine-adverse-event-reporting-system/) and at [Respectful Insolence](https://www.respectfulinsolence.com/tag/vaccine-adverse-event-reporting-system/). Willman’s credentials make the story’s framing more striking. He is an investigative journalist who won the 2001 Pulitzer Prize for reporting on seven FDA-approved drugs later withdrawn over safety problems and deaths. That earlier work also covered how policies intended to speed HIV/AIDS drug approvals contributed to approvals that came too quickly. It’s easy to see why a potential failure to detect vaccine harms would catch his attention. But the BMJ article includes a disclosure that changes how I read it: the account is adapted from an upcoming book co-written by Willman and Peter Doshi, a senior editor in The BMJ’s Investigations Unit. Doshi isn’t listed as a co-author of the article. That raises questions about his role in shaping the reporting and why the connection appears in the competing-interests note. The author points readers to an earlier account of [Doshi’s work and influence at The BMJ](https://sciencebasedmedicine.org/tag/peter-doshi/) before turning to the article’s arguments in detail.

Earlier disputes over The BMJ

This isn’t the author’s first public break with the journal. Nearly three years earlier, he asked [“What the heck happened to The BMJ?”](https://sciencebasedmedicine.org/what-the-heck-happened-to-the-bmj-2023-version/) and had already used [that title twice](https://sciencebasedmedicine.org/what-the-heck-happened-to-the-bmj/). He describes the journal’s recent vaccine coverage as a sharp change, and frames the new article as another instance of The BMJ lending authority to claims that antivaccine groups can recast as evidence of suppression. The author argues that readers should keep the reporting environment in view when weighing the statistical dispute. That’s the key point: an unusually large pile of reports in VAERS is not, on its own, evidence of unusual danger. The article’s claims about how officials handled safety signals need to be assessed against the methods, the reporting surge, and the actual evidence about specific events.

Peter Doshi: What happened to The BMJ

How Doshi became central to the story

The BMJ once helped expose Andrew Wakefield’s fraudulent claim that the MMR vaccine caused autism. Brian Deer’s reporting laid out the evidence, and his accompanying commentary called Wakefield’s work “Piltdown medicine,” invoking the Piltdown Man hoax. That history makes the journal’s later vaccine coverage harder to dismiss as a simple change in editorial taste. The point here is that The BMJ’s record includes both a major contribution to correcting vaccine misinformation and editorial choices that now invite scrutiny. The shift, in the author’s account, runs through Peter Doshi. Mark Crislip criticized a 2010 article by Doshi and Tom Jefferson that cast doubt on Tamiflu’s effectiveness. Doshi joined The BMJ as an associate editor in 2014, while an assistant professor in the University of Maryland School of Pharmacy’s Department of Pharmaceutical Health Services Research. The author says Doshi was a senior editor by 2021, though he hasn’t established exactly when the promotion happened. The author traces Doshi’s vaccine-related controversies back further. In 2009, during the H1N1 pandemic, Doshi was scheduled to speak at a conference hosted by the National Vaccine Information Center (NVIC), an anti-vaccine organization. When asked about the event, he replied that his appearance didn’t imply endorsement. The author argues that Doshi later repeated anti-vaccine talking points while maintaining that he was “not antivaccine,” and says a pseudonymous epidemiologist had criticized Doshi’s 2006 claim that estimates of annual influenza deaths were “grossly inflated.” Doshi has also challenged the severity of flu and the effectiveness of flu vaccines, and has said that people who question vaccines are unfairly labeled anti-vaccine. The COVID-19 vaccine debate made the dispute more consequential. In January 2021, Doshi published a BMJ blog post asking for more details and raw data behind Pfizer and Moderna’s reported 95% efficacy. The author says Doshi used statistical reasoning that misrepresented the trial results and put vaccine efficacy at just 19%. He argues that this analysis helped seed a persistent misinformation claim about the vaccines, which he calls a “slasher myth.” Doshi’s academic credentials are in anthropology, East Asian studies, and the history and anthropology of science, technology, and society; the author argues that he is not a vaccinologist, infectious-disease specialist, or clinical-trials expert. The author also cites Doshi’s signature on a 2006 petition questioning whether HIV causes AIDS, his support for a claim that VAERS data were being hidden, and his role as an expert witness for plaintiffs challenging the University of California’s flu vaccine mandate. Taken together, these examples form the author’s case that The BMJ had ample reason to question Doshi’s judgment on vaccines. That context also shapes the author’s reading of The BMJ’s 2021 investigation into Ventavia, a contractor involved in Pfizer’s COVID-19 vaccine trial. The article alleged poor working conditions and that trial participants had been unblinded. The author describes those claims as exaggerated or false, and says the reporting presented them as part of a conspiracy. He argues that Doshi’s role as senior editor of BMJ Investigations helps explain why the journal published work by Paul Thacker, whom he characterizes as a conspiracy theorist and unreliable journalist. That’s the author’s interpretation; the fragment doesn’t establish who controlled the Ventavia investigation at the time.

The charges

The new BMJ investigation, written by David Willman with Doshi listed as a co-author—though the author suspects Doshi’s contribution may have been nominal—centers on how federal officials monitored reports of possible COVID-19 vaccine harms. The author accepts that there is a legitimate story to examine. His objection is that the article frames officials’ actions as negligent or malicious without establishing those motives. The investigation focuses on the CDC’s use of VAERS, the Vaccine Adverse Event Reporting System, and the FDA’s role in analyzing its reports. Willman writes that the CDC planned to use two methods to spot possible safety signals: proportional reporting ratios (PRRs), which look for adverse events reported more often than expected, and an empirical Bayesian method supplied by the FDA. The agencies were meant to share and discuss their findings. The article’s first citation for that claim led to a dead link. The author says the Internet Archive was temporarily unavailable when he tried to retrieve it. He considers the two methods reasonable in principle: both assess whether a particular event appears disproportionately often after a vaccine compared with other vaccines, though each has strengths and weaknesses. The central allegation is that the agencies relied on the FDA’s empirical Bayesian system alone during the vaccine rollout, despite warnings that the flood of COVID-19 reports had weakened its ability to detect signals. Willman cites a September 2022 letter from CDC director Rochelle Walensky saying the agency began PRR analyses in 2022, to corroborate the FDA’s approach. Walensky described empirical Bayesian analysis as “a more robust technique.” The BMJ investigation argues that the volume of reports compromised the FDA algorithm’s sensitivity for Pfizer and Moderna’s mRNA vaccines, and says officials knew about that limitation during the rollout. The article goes further: it says officials continued to rely on the algorithm and cited the absence of detected signals when reassuring the public about vaccine safety. It also alleges that FDA managers silenced a career drug-safety officer who had raised the concern and proposed an updated algorithm. The article concludes that the public received unjustified assurances and missed timely warnings about possible harms. That is a forceful charge. The author’s framing asks readers to distinguish the documented question of how the system performed from the investigation’s claims about why officials acted as they did. The officer at the center of the account is Ana Szarfman, an FDA medical officer with a PhD in microbiology and immunology. She worked for decades on drug-safety data mining and helped modernize the agency’s approach. The article says she alerted FDA official Peter Marks to the issue in early 2021. It also describes Szarfman’s long professional relationship with statistician William DuMouchel, who developed the Bayesian algorithm the FDA had used for more than 15 years. The alleged failure, according to The BMJ, was that the system did not flag potential links between mRNA vaccination and myocarditis, pericarditis, Bell’s palsy, tinnitus, and other reported conditions because of a methodological flaw that senior officials declined to correct.

The control group may matter more than the algorithm

The central problem described here is masking: when most reports in VAERS involve the vaccines being assessed, a vaccine can look unremarkable against a comparison group dominated by another vaccine with similar risks.

In the first year of the rollout, more than 90% of 1.1 million reports to the VAERS concerned the mRNA covid vaccines, far more than reports for vaccines against other diseases.⁴ So an analysis of Pfizer’s vaccine could effectively compare it with a pool made up mostly of Moderna reports. If both vaccines raised the risk of an event such as myocarditis by roughly the same amount, the observed and expected frequencies could end up similar. An automated system might then raise no alert.⁵

That’s a problem with the comparison group, at least as the argument is presented here. It doesn’t, by itself, show that the statistical method was defective. The distinction matters because the debate has often been framed as a contest between the proportional reporting ratio, or PRR, and an empirical-Bayes method. Statistician Jeffrey Morris argued that they address the same basic question: whether a particular adverse event is reported disproportionately often.

Here’s Morris’s explanation of how the methods differ:

The empirical-Bayes method is not some fundamentally different or defective type of analysis. It builds on the same observed-versus-expected reporting framework, but uses shrinkage to stabilize estimates, particularly when counts are small, and provides a posterior distribution for the underlying reporting ratio that can be examined statistically. That shrinkage is generally a statistical advantage. My white paper discusses this relationship in detail.

An important difference is how a potential signal is flagged. The commonly used screened PRR rule is essentially an ad hoc threshold: PRR > 2, ChiSq>4, with at least 3 reports. In contrast, FDA used the more stringent empirical-Bayes criterion EB05 > 2: the lower 5th percentile of the posterior reporting-ratio distribution must exceed 2, which is equivalent to requiring greater than 95% posterior probability that the underlying reporting ratio exceeds twofold. Not surprisingly, those criteria have different operating characteristics. EB05 > 2 is substantially more conservative: it should produce fewer false-positive alerts, particularly for sparse data, but at the cost of lower sensitivity and potentially more false negatives.

The practical tradeoff is familiar: PRR can flag more possible signals, but at the cost of more false positives. The empirical-Bayes threshold is stricter, so it may miss more potential signals while making the ones it does flag more likely to merit attention. The white paper goes into the methods in more detail.

A larger count of PRR alerts doesn’t establish that PRR is the better method, or that the FDA’s approach was flawed. That would require comparing the methods’ sensitivity, specificity, false-positive rates, and false-negative rates under realistic reporting conditions. Morris’s criticism is that the report highlights the number of alerts without providing that operating-characteristic analysis. FDA describes data-mining signals as prompts for investigation, not proof that a vaccine caused an event.

Morris laid out his broader critique in an analysis on Twitter. The embedded post begins by warning that the article runs together several distinct statistical issues:

There are several distinct statistical issues here that the article largely runs together.

First, the classical PRR and empirical-Bayes approaches are fundamentally addressing the same disproportionality question: is a particular adverse event reported disproportionately often…

— Prof Jeffrey S Morris (@jsm2334) September 17, 2026

The report also describes a presentation by FDA medical officer Ana Szarfman comparing a PRR algorithm with the empirical-Bayes method then in use. It says the presentation was made at a meeting on February 29, 2020; the figure caption dates the presentation to February 29, 2021. The comparison showed many more signals under PRR:

The chart is striking. But more signals are exactly what you’d expect from a more sensitive threshold; the chart alone can’t tell us how many would prove meaningful. A system that flags too many events can swamp investigators with leads that don’t hold up. A stricter system may produce cleaner alerts while missing real problems. Deciding where to set that balance is a legitimate scientific question. The argument becomes less persuasive when the number of signals is treated as proof that one side was right.

That framing also tends to cast Szarfman as a whistleblower ignored by a negligent FDA. The record described here is more complicated. She was invited to present her findings to Peter Marks, then director of the Center for Biologics Evaluation and Research (CBER), the FDA center responsible for approving vaccines. That doesn’t settle whether her concerns were handled properly, but it does matter when judging the claim that she was simply shut out.

There were also reasons to question some of the signals. Dr. Vincent Iannelli points to signals Szarfman identified for events covid vaccines don’t cause, particularly sudden death. Willman’s report says Szarfman later raised an alarm about a “strong signal” for “death and sudden death” that she and Bill DuMouchel had identified using an updated algorithm. She shared the findings with Richard Forshee, a deputy director for biostatistics and pharmacovigilance who worked under Marks. Forshee wrote to Marks:

But by July 2021, Szarfman again sounded an alarm over a “strong signal” for “death and sudden death” that she said DuMouchel had identified using the updated algorithm. She shared the findings with a colleague, Richard Forshee, a deputy director for biostatistics and pharmacovigilance, who worked under Peter Marks. Forshee expressed his doubts about the analysis to Szarfman and wrote to Marks: “Ana Szarfman called me . . . She said that she and Bill DuMouchel had found an increased risk of mortality following covid-19 vaccination using data mining methods.

“I am very concerned that whatever association they think they have identified is spurious based on the way the covid-19 vaccination program prioritized individuals and the required and stimulated reporting we are seeing with the covid-19 vaccines. . . . Please let me know how you would like us to proceed.”

Marks later raised the issue with the director of the FDA’s Center for Drug Evaluation and Research (CDER). In a letter dated 15 September 2021, he wrote:

On 15 September 2021, Marks wrote to his counterpart in the FDA’s Center for Drug Evaluation and Research (CDER): “One of the CDER statisticians, Ana Szarfman, has decided on her own to do vaccine analyses using VAERS as part of her work at FDA. She is, however, not doing this in collaboration with our [vaccine centre] statisticians, and quite to the contrary, has been asked to cease and desist, because the strategy that she is using could create erroneous conflicts that feed in to anti-vaccination rhetoric. This is creating an issue . . . This issue came up previously during the pandemic and . . . it seemed to go away, but it is now back. Can we catch up about this sometime?”

Marks may have been justified in worrying about misleading signals, or he may have dismissed Szarfman too readily; the material here doesn’t resolve that. What it does support is a narrower point: the choice of comparison group could mask a signal regardless of whether the analysis used PRR or empirical Bayes. The fight over algorithms risks obscuring that design choice—and the basic limit of VAERS data mining. A signal is a reason to investigate. It isn’t, on its own, evidence of causation.