During the rapid rollout of COVID-19 vaccines in 2021, the FDA relied heavily on data mining of the Vaccine Adverse Event Reporting System (VAERS) to detect potential safety issues. A key tool was the Multi-item Gamma Poisson Shrinker (MGPS), an empirical Bayesian method. However, internal analyses revealed that this standard approach suffered from a major limitation called masking, which obscured important safety signals.

What Is Masking in Vaccine Safety Data Mining?

Masking occurs when a high volume of reports for one product (or group of products) inflates the background rate or "expected" count for adverse events. This drowns out or hides disproportionate signals for specific vaccines or events.

In the context of COVID-19 vaccines:

• The three authorized vaccines (Pfizer, Moderna, and Janssen) generated an enormous number of reports in VAERS due to the scale of the campaign.
• When analyzing one vaccine (e.g., Pfizer), the massive reports from the others (especially similar mRNA vaccines) artificially boosted the baseline comparison.
• As a result, genuine disproportionalities for serious events associated with a particular vaccine became statistically invisible or weakened under standard methods.

Simple analogy: Imagine comparing the toxicity of hemlock to a baseline of "all other drinks." If the baseline includes a lot of arsenic reports, hemlock's relative danger appears lower (masked). Separate the baselines properly, and the signal emerges clearly.

How Safety Signals Are Measured: MGPS vs. RGPS

Standard Method (MGPS):

• Uses an empirical Bayesian approach (Gamma Poisson Shrinker) to compute an Empirical Bayes Geometric Mean (EBGM) and its lower bound (EB05).
• A signal is typically flagged as statistically significant if EB05 > 2.0 (indicating the event is reported at least twice as often as expected, after shrinkage to account for small numbers).
• It is designed to reduce false positives from multiple comparisons but does not adequately adjust for masking when one or a few products dominate the database.

Improved Method (RGPS — Regression-Adjusted Gamma Poisson Shrinker):

• Developed/enhanced by Dr. William DuMouchel (inventor of the original GPS/MGPS) and Dr. Ana Szarfman (FDA senior medical officer and data mining expert).
• Adds a regression adjustment layer on top of the Bayesian framework.
• This explicitly models and corrects for confounders, including masking effects from high-volume co-reported products.
• It automatically "unmasks" signals by better estimating the true expected counts without the inflation from dominant reporters.

RGPS was available within the same Oracle Empirica Signal software that the FDA already used for MGPS.

Outcomes Measured and Signals Uncovered by RGPS

In March 2021 (and in follow-ups through July 2021), Dr. Szarfman’s RGPS analyses of VAERS data identified 49 examples of extreme masking and uncovered approximately 25 new statistically significant safety signals (EB05 > 2.0) that MGPS had missed or under-detected. These included:

• Sudden cardiac death
• Acute myocardial infarction (heart attacks)
• Bell’s palsy
• Pulmonary infarction
• Embolism and thrombosis (non-site specific)
• Dementia
• Increased mortality / "Death and sudden death"
• Myopericarditis (internally acknowledged in some analyses)

Later analyses and related publications confirmed that masking was roughly eight times more likely with COVID-19 vaccines than with other vaccines due to the sheer volume of reports.

Why Standard Methods Fell Short During the Crisis

1. Reliance on Flawed "Gold Standard": FDA and CDC SOPs emphasized MGPS as a robust technique. While effective in normal times with diverse, lower-volume reports, it was overwhelmed by the unprecedented COVID-19 vaccine data flood.

2. Known Limitation Ignored: Masking was a documented issue in pharmacovigilance literature, yet officials continued using the unadjusted method. Internal records show awareness but resistance to switching or supplementing with RGPS during the rollout.

3. Focus on Narrative Over Detection: Concerns were raised that highlighting additional signals could "feed into anti-vaccination rhetoric." Dr. Szarfman was eventually told to "hold off" on creating and sending further RGPS reports.

4. Delayed Action: Even when myopericarditis signals were internally confirmed, public warnings and label updates lagged. Broader unmasked signals (cardiac, thrombotic, mortality-related) were not promptly investigated or communicated using the superior method.

In summary, the masking effect in MGPS created a statistical blind spot precisely when rapid, sensitive detection was most critical. RGPS demonstrated that better tools existed within the FDA’s own ecosystem to illuminate hidden risks — tools that could have provided earlier, clearer insights into potential vaccine safety concerns. The Senate report highlights this as a critical failure in pharmacovigilance during a public health emergency.

This episode underscores the importance of continually validating and upgrading safety surveillance methods, especially under high-stakes conditions with massive data volumes.

NOTE The data used in the research does not observe the meta drops / removals during COVID era crisis in the VAERS database when it comes to observing safety signals. November 2022 observed a massive drop of safety signal data. See article about that: https://deepdots.substack.com/p/undeleting-a-gigabyte-of-data-purged

References

1. US Senate Permanent Subcommittee on Investigations, Committee on Homeland Security and Governmental Affairs. Majority Staff Interim Report: Documents produced by the Department of Health and Human Services show that Biden health officials ignored safety system that can better detect signals for COVID-19 vaccine adverse events. April 29, 2026. Accessed May 23, 2026. https://www.hsgac.senate.gov/wp-content/uploads/Senate-PSI-Majority-Staff-Interim-Report-April-29-2026-FINAL.pdf

2. Harpaz R, DuMouchel W, et al. Signaling COVID-19 vaccine adverse events. Drug Saf. 2022;45(7):765-781. doi:10.1007/s40264-022-01186-z

3. Vaccine Adverse Event Reporting System (VAERS) Standard Operating Procedures for COVID-19. Centers for Disease Control and Prevention. January 29, 2021. Accessed via archived link in Senate report.

4. DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat. 1999;53(3):177-190.