To improve predictions, Minnesota seeks to learn from sketchy COVID-19 models

Minnesota is using a $17 million federal grant to learn from the pitfalls of COVID-19 forecasting in the last few years and to improve its predictions for the next outbreak.

Better estimates of cases of an infectious disease and how it spreads could improve responses and target them at high-risk regions or populations rather than the entire state, said Dr. R. Adams Dudley, a University of Minnesota health informatics professor who is co-leading the effort.

Better estimates also could prevent the damage that occurred during the COVID pandemic when wayward predictions undermined public confidence in the government’s quarantine orders and restrictions.

“Think about the public trust cost of being wrong,” Dudley said.

A consortium of Minnesota agencies announced last week that it is one of 13 research groups in the United States to receive federal funding to better prepare the nation for the next public health emergency. The group expects to make significant discoveries based on Minnesota’s unique experience with COVID and a new shared medical record-keeping system that doesn’t exist elsewhere.

Though COVID remains a concern — a revised state dashboard identified low but rising viral levels in wastewater this month — the Mayo Clinic and other prominent health care organizations stopped forecasting its spread months ago. A federally supported model called ENSEMBLE still predicts COVID hospitalization trends but not infections or deaths.

So the timing is ideal to learn from the pandemic and use that experience to create more precise tools, said Eva Enns, a U public health researcher who helped create Minnesota’s initial COVID models in 2020.

It probably won’t take another century for the next public health emergency to emerge, she said, because of environmental conditions and global mobility.

“It’s probably not going to be another COVID. It’s going to be something new,” she said. “But we need to debrief what happened while those details are fresh.”

Getting case numbers right

A first step will be surveying Minnesotans to get a better understanding of their daily movements and face-to-face interactions — a key data element in any calculation of how a disease spreads. The mobility estimates used in COVID predictions were crude, and didn’t account for differences by season or urbanization.

Once a survey is validated, it could be used monthly to detect shifts in public attitudes, which could affect how quickly a pathogen spreads. Everything from political leanings to bad memories about the COVID public health response could influence whether people wear masks or stay at home as directed in future events.

“Will there be more resistance because, you know, [COVID] lasted longer than people were told it would?” Enns said.

The research also will rely on the new Minnesota EHR Consortium, which links medical records from clinics and hospitals to identify trends. The consortium’s first project earlier this year was a near real-time dashboard of drug-related ER visits in Hennepin County.

Minnesota was among the first states to publicize COVID modeling estimates in the spring of 2020. Gov. Tim Walz based a statewide stay-at-home order on predictions that 74,000 Minnesotans could die if nothing was done. Sketchy information about the new coronavirus strain led to overestimates of the death risks; the state is now approaching 15,000 deaths after more than three years.

The modeling was wrong on the rate of deaths per COVID cases because the denominator of total COVID cases was vastly undercounted and mostly based on hospitalizations, Dudley said. Tests weren’t widely available early on to identify milder infections, and field reports to public health agencies were inconclusive.

“The number was wrong because we didn’t have a system for really knowing how many cases there were,” he said.

Analysis of Minnesotans’ complete up-to-date clinical records would solve that problem by identifying broad groups of patients who went to their doctors with the same suspicious symptoms, Dudley said.

The consortium also could serve as an early warning in terms of understanding usual illness patterns and alerting health officials to increases in unusual symptoms — like the loss of smell and brain fog symptoms that were early characteristics of COVID.

Even that solution has limits. COVID ended up causing asymptomatic infections, which didn’t show up in clinical records and further obscured the disease’s true mortality rate.

Predictive tools

Consortium leaders said their solutions need to be flexible. The next outbreak could involve another respiratory infection or it could have little in common with COVID, perhaps causing digestive problems or emerging from insects.

Despite the numerical misses, the early Minnesota COVID models succeeded at their primary purpose: giving state leaders estimates of how policies such as mask mandates or nursing home closures would affect viral transmission.

New predictive tools need to be developed so they are easily understood and actually answer the questions decisionmakers face in a crisis, said Kristen Sweet, an infectious disease manager with the Minnesota Department of Health.

“What is helpful for people and how do we communicate both the strengths and limitations of different analytic tools?” said Sweet, also a co-leader of the grant-funded partnership.

New predictive tools also will be tested by running simulations to see if the tools would have identified COVID patterns sooner.

“We’re going to go back and pretend we know nothing more than we did in March 2020,” Dudley said, “and see if it would have worked and how fast it would have worked for COVID.”