Our COVID Statistics are all Wrong

Management guru Peter Drucker gave us “what gets measured gets managed” (for better or worse). Governments bombard us daily with COVID-related statistics, but many of them are measuring the wrong things.

The most common statistic is “new cases per day”, but that has serious deficiencies. Firstly, it’s quoted as an absolute number, rather than as a proportion of the population. So in the early days of COVID, the US had huge daily numbers, but as a country of some 330 million, its absolute numbers dwarfed those of France and Germany, leading us to think it was much worse. At the time, their case rate per 100,000 population was actually lower than many countries in Europe. If anything, a better comparison would be between the US and the EU (450 million). Secondly, the new cases were never split out by severity: asymptomatic, mild, severe, and deadly. We only had the number of cases and the death count. That split is especially relevant now, as populations are being vaccinated, opening up, and new cases spiking rapidly with the Delta variant. While the case numbers are high, the proportion of severe and deadly cases are what we should be worried about. But is anyone tracking that? Are they tracking the proportion of each type of case, which can provide data over time to show an improved ability to treat cases. Thirdly, in Australia, the only good number of new cases per day is zero. That means our elimination strategy is either successful or it is not, with no room for nuance (and that’s quite aside from the unwanted side effects of that strategy).

The most important thing to measure is the R number – how contagious the virus is. In simple terms, it’s the average number of people anyone with COVID will infect (some people will infect no-one; others will be super-spreaders – the average is what is important). If R is more than 1, then COVID spreads and the number of cases keeps increasing; if R is less than 1, it fizzes (like the common cold or the seasonal flu). When COVID first came out, R was 2.5 which made it quite contagious, and at the time we didn’t know how to treat it, and were concerned about the capacity of the health system. The current Delta variant has an estimated R of between 3.5 and 4, and this is now poses significant additional challenges around the world. It seems to spread faster, and may also be more severe on average.

How to reduce R? If immunised, you are less likely both to get it and also to transmit it. Social distancing, mask wearing and hand sanitising also reduces the spread. COVID also spreads less outdoors, and in warmer temperatures. The common sense approach would be to

• set a goal of reducing R to below 1,
• develop strategies to achieve that,
• identify the metrics that show you are on track, and
• report them regularly.

In Australia, R is not being reported regularly (in some other countries, it is). Authorities may be tracking it, but if they are, they are not keeping us informed. Instead, we get a litany of fear-ridden statistics like number of exposure sites, and the number of people in isolation because they visited an exposure site. This comes straight from the How to Lie With Statistics playbook! We can’t directly manage or impact those numbers, so what’s the point of reporting them and using them as a measure of success? I’d like to think the government has clear targets that determine when they choose certain measures, and when those measures can be removed. If they are, they are not sharing them with the public!

Only recently, additional information was added to “new cases per day”: how many of those are linked to an existing outbreak, and what proportion were infectious while in the community. While that extra information is helpful, it has taken five lockdowns for the government to start telling us, and we are still in the dark as to the metrics of success and how to get there.

We have a similar problem with vaccination rates. Our government’s huge bet on the AZ vaccine has backfired badly. This is another case of bad measurement, irresponsible reporting, and poor social marketing. The risk of the clotting side effect is less then miniscule, and yet a single death amongst millions vaccinated makes front page news. This has led to mass reluctance to vaccinate, and policy on the fly regarding vaccination priorities and eligibility. If we want to reduce R, surely we should prioritise the vaccination for people who come into contact with many people from across the city (e.g. retail, transport) as they have the greater risk of spreading the virus across a large geographic area. We rushed to vaccinate the most vulnerable, but with a zero-COVID strategy, how does that help?

Here are some of the things we don’t report: number of deaths per year from the seasonal flu (about 1,000), number of severe side effects from medications taken regularly by the masses (a lot more than you think), number of deaths in hospitals from incorrect dosing (ditto). We must be careful not to stray here into the logic flaw of whataboutism which can lead to distractions. Rather, the numbers that we do choose to report need a healthy dose of context and nuance to avoid the kind of mass fear we are now experiencing.

I’ve been following the patterns of graphs here since the start of the pandemic. The spikes in new cases per day in well-immunised populations (like Israel and UK) are not such a concern – instead, we should look at “case fatality rate” which has dropped significantly in those places, and remained roughly steady in non-immunised populations. Singapore have recently had a shift in policy to start treating COVID (assuming enough people are immunised) like the flu, and focus on “medical outcomes” rather than pure case numbers (note: they are still applying suppression measures as well). We don’t count cases of mild flu, so don’t count cases of mild COVID.

The only way out of this mess is mass vaccination that turns COVID into just another flu. When countries all start doing that, measures like new cases per day will be meaningless. At all times, we need to be measuring the right things to deliver the outcomes we are seeking, and communicating clearly to the population.