COVID metrics arbitrary, backward facing

State, health-care workers and public need more information to prepare for shortages and help find solutions

By Peter Fisher [updated April 6]

All but five states have now issued a shelter-in-place order, or have such orders covering parts of the state.[1] They have done so not just to reduce the spread of the disease and the number of residents dying, but to forestall a surge of cases that would overwhelm the health care system in their state. In doing so, they have relied on epidemiological data and projections indicating when their health care systems will face a shortage of hospital beds, intensive care units, and ventilators without serious restrictions on travel, business operations, and social activities.

One of those projections, by the Institute for Health Metrics and Evaluation (IHME) of the University of Washington and updated April 5, shows the wide range of possibilities. They predict a peak day of April 26, with Iowa needing between 53 and 438 ICU beds, and 150 as the best estimate, compared to 246 available beds. They predict a need for between 42 and 367 ventilators, and a peak of 17 deaths per day (with a range of 0 to 100), with 263 to 711 total deaths in the state by August.[2] These predictions assume that Iowa takes more serious measures in seven days — a stay-at-home order and mandatory closure of all nonessential businesses. If the governor still sees no need for further restrictions by then, the situation could be worse.

So what is the Governor relying on that leads her to believe that halfway measures are adequate to protect our health care workers and our citizens, and to prevent such shortages? The “metrics” and “data,” referred to repeatedly but vaguely in press briefings, became clear to the public only on Wednesday, when the “Guidelines for Implementing Public Health Mitigation Measures” were obtained by the Iowa City Press-Citizen. While we now know something of how the guidelines work, much remains a mystery. We do not know why this scoring system was adopted, who developed it, what science is behind the measures and the scores, whether any other state or country has used anything like it, or whether it was vetted by any qualified epidemiologists at Iowa universities or hospitals. The matrix cannot be found anywhere on the IDPH website, and the underlying data have not been released despite repeated requests by members of the media and others.

What we do know is that the application of the model does not give much hope to the many groups pressing the governor for more aggressive measures, most recently the Iowa Board of Medicine.[3] Suppose you live in the southeast portion of Iowa, where the overall incidence of COVID-19 is the highest in the state. Even there you are not going to get a shelter-in-place order from the governor anytime soon, as long as she is wedded to the IDPH matrix.

The matrix consists of scores of 1 to 3 on each of four measures and is applied to each of six regions in the state. Only when the score reaches 10 is a shelter in place order justified under the IDPH guidelines. The southeast region, which includes Johnson, Washington and Scott counties, along with 12 others, scores only a 7. Getting 3 more points will require either an outbreak at two nursing homes plus another 100 cases, or an outbreak at 3 nursing homes. That’s the only way the score will get to the magic number of 10 because the region can’t get any more points out of the other two measures. Only one nursing home outbreak, but 500 more cases, or 1,000 more cases? Too bad; the score is still stuck at 9.

That is just how arbitrary and rigid this matrix is.

So how does it work? We were able to construct at least part of the scores for each region based on data by county for population by age and the number of COVID-19 cases reported as of April 3. We relied on images of the regional map published in the press, and translated those to actual county boundaries.

The scores in the regional map above are the official scores being used as of April 6, according to The Des Moines Register. The Iowa Department of Public Health is not promoting this information prominently on its website. Because of a lack of data being made available in some of the official metrics from which the scores are compiled, the origin of the scores in some cases cannot be determined.

First, a region gets a score of 1 to 3 depending on the percent of the population age 65 or older. For most regions, the score is 2 because the share is between 15 and 19 percent. The central region, No. 1, which includes the Des Moines metro area, is a little younger, and gets only 1 point here. The north central region, No. 2, on the other hand gets 3 points, with more than 21 percent of the population over 65.[4] None of those scores are going to change. It just means that central Iowa is going to have a harder time getting to 10, north central an easier time.

Second, the region is scored on the percent of cases needing hospitalization. Statewide, that percent in Iowa has hovered between 26 percent and 28 percent for the last several days.[5] This is substantially higher than national rates, but this may reflect nothing more than low rates of testing among those without substantial symptoms in Iowa. The more we test among those potentially exposed to the virus, whether or not they are symptomatic, the more the number of cases will rise and the lower the percent needing hospitalization. However, it seems unlikely that this factor will fall below 15 percent, so we can pretty much assume it is 3 points statewide, and probably in each region. (Hospitalizations by county has not been released by IDPH).

More to the point, why is this rate in the decision matrix at all? It is a characteristic of the virus; is there any reason to expect it to be more severe, and therefore more likely to require hospitalization, over time, or in one part of the state than in another? Surely the total number of people requiring hospitalization at the projected peak is the critical number; the historical percent is irrelevant.

The third measure is COVID-19 cases per 100,000 population. A number from 6 to 20 is worth 1 point, 21 to 49 is 2 points, 50 or more cases is 3 (the maximum). As of April 3, this measure ranged from 7 in the northwest (region 3) to 29 in region 5 (which includes Linn County) and 35 in the southeast, which includes Johnson County.[6] That gives regions 1-4 each 1 point, regions 5 and 6, 2 points. The latter regions could be well over 100 cases per 100,000 people soon, but no matter how high it goes the most they will get is 3 points.

Finally, the matrix assigns one point for each outbreak at a long-term care facility, up to a maximum of 3. As of April 3, the only such outbreak that had been reported in the press was at a Linn County care facility. Does an outbreak at one nursing home predict a shortage of ICU beds over the entire population in a region? Probably not.

The arbitrariness of the measures, the score cutoffs, and the maximum scores are not the only problems with the matrix. The scoring is entirely backward looking. This is particularly dangerous for a disease that spreads so easily and where, as we all know by now, cases and hospitalizations grow exponentially, everywhere that the disease has been found. Historical data in Iowa and in places where the diseases arrived much earlier is useful, but only because it allows one to determine the rate at which the disease is spreading. The fewer the number of days it is taking for the number of cases to double, the sooner we will arrive at a peak, the sharper that peak will be, and the more likely it is that we face a potential health disaster. Again, projected cases and hospitalizations is the goal, not history.

The governor has complained that the IHME projection does not take into account measures already undertaken in Iowa — the now-mandatory closing of schools and the mandatory closing of some kinds of businesses. But she has offered no alternative projections, no projections at all. The state should be planning for the worst-case scenario, not the best case, but she has not disclosed what she sees as either scenario.

The IDPH matrix is not science-based guidance for decision-making. It is an arbitrary scoring method divorced from all the epidemiological analysis and modeling that is taking place elsewhere. It tells us nothing about when or by how much our health care system will face critical shortages. It is an embarrassment.

[1] The New York Times.

[2] University of Washington, Institute for Health Metrics and Evaluation, April 1, 2020, update.

[3] Barbara Rodriguez, The Des Moines Register, April 3, 2020.

[4] These scores are based on the counties included in the six regions according to the map released by the Iowa City Press-Citizen, in combination with the data on population by age and county from the most recent five-year average of the American Community Survey.

[5] IDPH data for April 1-3 shows that on average the sum of deaths, current hospitalizations, and hospitalized but released equals the total number of cases in Iowa that have at some point required hospitalization, assuming that those who died were first hospitalized. This number divided by total confirmed cases equals 28 percent.

[6] Analysis by the author using cases per county as reported by IDPH on the afternoon of April 3, and county population according to the American Community Survey.

Peter Fisher is research director for the Iowa Policy Project. An economist and one of Iowa’s leading experts on tax policy, he has published several analyses of criteria used in indexes of state tax systems, available at Fisher is professor emeritus in the School of Urban and Regional Planning at the University of Iowa.