Experts at CDC debate test‑negative design and urge stronger data links to measure vaccine effectiveness

At a relaunch of CDC Public Health Grand Rounds, researchers from government and academia debated how best to measure vaccine effectiveness (VE) and recommended near‑term investments in data linkage and sequencing to make estimates more reliable and timely.

"Vaccine effectiveness studies help us understand how vaccines perform in real world settings and how they protect people from outcomes such as illness, hospitalization, and death," said Dr. Carrie Reid, branch chief in the Influenza Division at CDC, opening the technical session and defining how VE should be interpreted at the population level.

Panelists outlined the principal study approaches used to estimate VE: randomized clinical trials, observational cohort studies and case‑driven approaches including the test‑negative design (TND). Each has tradeoffs. "Randomized trials…just do not have a key role in year‑to‑year routine surveillance," said Dr. Natalie Dean, an associate professor at Emory University, because trials require time, money and ethical equipoise.

The TND — in which symptomatic patients who test positive for the pathogen are compared with symptomatic patients who test negative — has provided decades of timely surveillance data and helps mitigate care‑seeking bias when properly implemented, the panel said. But critics warned about important limitations. "We should never select the controls based on another disease," said Dr. Martin Kuldorf of the HHS Office of the Assistant Secretary for Planning and Evaluation, arguing that controls who present with different illnesses can come from different background populations and introduce bias.

Dr. Emily Martin of the University of Michigan urged a practical, evidence‑synthesis approach: "Our goal is to generate a body of evidence that, when taken together, gives us information about vaccine performance against the pathogens circulating right now." She and others showed historical examples in which multiple methods produced concordant estimates and noted cases when results diverged — highlighting the need to triangulate findings across methods and populations.

On practical fixes, panelists converged on several near‑term priorities: (1) better linkage between electronic health records and state vaccine registries to capture vaccination status; (2) increased routine sequencing of clinical specimens so surveillance can detect mid‑season strain changes; and (3) increased use of large retrospective cohort analyses (for example, using Medicare/Medicaid and commercial claims) to answer rare‑outcome questions such as mortality. "Use the same registries and data feeds we employ for vaccine safety to support effectiveness analyses," Kuldorf said.

Audience members pressed on diagnostics and analysis. David Sheay noted prior problems when studies used broad outcomes such as pneumonia and urged specific PCR‑confirmed case definitions; panelists agreed that case definition and testing practices materially affect estimates and that prospective testing in surveillance networks remains important for strain‑level inferences.

Panelists also discussed messaging. When interim VE estimates change mid‑season, Reid said public health messages should be transparent and practical — compare the current season to prior seasons and other datasets, explain what is known and unknown, and emphasize actions such as vaccination and antiviral treatment where appropriate.

The session closed with organizers urging continued methodological work and infrastructure investment. A recording will be made available and the Grand Rounds series will continue quarterly.