Denominator Problem

Wasted, sacrificed for a new nirvana
Nighttime sends us on our way
--Icicle Works

It seems people at Stanford are among the few asking critical questions related to the accuracy of the 'official' COVID-19 assessment. Earlier we shared the 'evidence fiasco' thoughts of Stanford's John Ioannidis. Today two more Stanford professors of medicine voice their concerns in this WSJ piece.

They suggest what amounts to a massive 'denominator problem' in reported COVID-19 fatality rates statistics. As of this writing, the Johns Hopkins COVID-19 tracking site is reporting 802 deaths in the US from 55,243 total confirmed cases--a 1.45% fatality rate. Many people are using such fatality rates for future projections. If they suppose that 100 million Americans ultimately get infected, then 1.45 million would die using the above fatality rate.

The Stanford profs argue that those estimates are deeply flawed. The true fatality rate depends not on deaths from confirmed positive cases, but on the number of people who get infected. And the total number of people infected with the COVID-19 virus is likely to be far higher than reported cases indicate.

This is because of selection bias in the testing process. Testing for COVID-19 infection has largely been limited to those people who show symptoms of the virus, or who think they might have the virus. As the Stanford profs demonstrate using data from novel population samples from China, Italy, Iceland, and the US, prevalence of COVID-19 infection ranged from 0.9%-2.7% of all individuals in the sample. Applying that prevalence to the US population at large suggests that the number of people that might already be infected with the virus could be more than 10x the reported number of reported cases.

This would make the denominator of the fatality rate calculation much larger than the one currently employed--and the calculated fatality rate much lower. Using our data above, disease prevalence that is 10x the reported case level shifts the decimal point one place to the left for a fatality rate of 0.145%--that's only slightly higher than the flu.

The Stanford profs sense the true death rate may be even lower. The first Chinese COVID-19 cases surfaced in Wuhan in late 2019. Although the first reported US case was reported in mid January, tens of thousands of people travelled from Wuhan to the US prior to that. Assuming a highly transmissible virus that drives infection doubling rate of every three days, and an epidemic seed on January 1 in the US, the researchers estimate 6 million Americans infected by March 9. Assuming a two week lag between infection and death, and a COVID-19 death count of 499 on March 23, then the US mortality rate would be about 0.01%--about one tenth of the mortality rate from the flu.

The researchers suggest that if the true infection and mortality rates approach their estimates, then the epidemic is limited in scale. Measures that focus on reducing risks for older adults and at risk individuals, and on re-allocating health care resources to care for critically ill patients are sensible. Universal quarantines that impose large economic and social costs are not.

Their bottom line: Similar to Dr Ioannidis, we need accurate prevalence and incidence data in order to solve our denominator problem so that we can make intelligent decisions here.

Antibody Studies

Shiver and say the words
Of every lie you've heard
--Echo & the Bunnymen

'Antibody studies' are finally starting to roll in. The studies seek to estimate the percentage of people who have already been infected by COVID-19 by checking for the presence of serum antibodies in large samples (n = 100s or 1000s of individuals).

Results are streaming in. NYC, Boston, Santa Clara, LA, Miami--among others locations (hoping someone aggregates these studies soon on a website).

All suggest similar. The prevalence of COVID-19 among the population is much higher than official reported infection counts--orders of magnitude higher. Estimated prevalence from the antibody studies ranges from 4% to over 20% in some jurisdictions. Projecting these results suggest that tens of millions of Americans have been infected with COVID-19.

It should be noted that COVID antibodies may take 2-3 weeks to develop in the bloodstream after infection, meaning that the infection rate measured in a particular antibody study likely under-reports true prevalence at a single point in time. This is why antibody tests for prevalence need to be repeated to better understand incidence (rate of change) as well.

With total infection counts much higher than expected, fatality rates are much lower than reported--and certainly far lower than initially forecast. Alex Berenson believes COVID-19 death rates are likely to settle in the 0.25% - 0.40% range (deaths/# infected basis).

5/ Add all this up, and the COVID death rate will likely settle into the 0.25%-0.4% range (1 in 250 people infected to 1 in 400). Far from 20 times higher than flu. Meanwhile, the median age of death is 78-80. And unlike the flu, #SARSCoV2 is basically not dangerous to children.

— Alex Berenson (@AlexBerenson) April 24, 2020

It should be acknowledged that the Stanford folks were on top of this from the beginning. It began with Dr Ioannidis' call for prevalence and incidence testing (along with his analysis of the Diamond Princess population sample with conclusions and projections largely in-line with what we're seeing). Other Stanford professors weighed in as well, arguing that COVID infection rates were likely to be far higher than initially reported thus pushing fatality rates far lower--and that, consequently, draconian lock-down measures were likely to do far more harm than good. They echoed the call for prevalence studies.

Now, with the antibody data streaming in, another Stanford doc suggests that policymakers must focus on the data and fundamental biology, rather than clinging to woefully inaccurate hypothetical projections, to thoughtfully remove restrictions and restore order.

Although many people will not welcome the findings, antibody studies help to awaken reasoning minds.

COVID Mortality Risk

With a little perseverance
You can get things done
Without the blind adherence
That has conquered some
--Cory Hart

Another important paper by John Ioannidis and his Stanford colleagues (full paper here). The researchers analyze COVID -19 mortality data from 11 European countries, Canada, and 12 'hot spot' US states as of April 24th. The primary objective was to estimate risk of COVID-19 death using 65 as the over/under age.

Results indicate that people < 65 yrs old have very small risk of dying from COVID-19. Moreover, death for people under 65 with no underlying conditions is "remarkably uncommon." For people in many countries, risk of dying from COVID-19 roughly equates to the risk of dying from driving to and from work each day. The researchers conclude that strategies that protect the high risk elderly are prudent moving forward.

A few nuggets from their Discussion section:

The researchers suspect that the peak of the 'epidemic wave' may have already been reached in many countries by May 1. While they acknowledge the possibility of a second epidemic wave next fall or spring, they note that influenza presents similar uncertainties.

They highlight issues with death count accuracy. "Even though mortality is an unambiguous endpoint, attribution of death to a specific cause is often challenging and definitions of 'COVID-19 death' vary across countries and sometimes even change within countries over time." They note that their age-stratified data sample from Canada and the UK do not appear to include deaths that occurred outside of hospitals. They were added to the UK sample past the deadline for the paper's submission.

Moreover, "different countries and US locations differ on the threshold of including deaths at care homes." For instance Belgium reports more than half of its deaths as coming from nursing homes although > 90% have not had laboratory confirmation. "New York City and some other US locations have also started counting in more recent counts 'probable deaths' without any COVID-19 laboratory confirmation, a debatable practice at best [emphasis mine]."

The researchers also comment on outlier cases--i.e., mortality of people less than 65 with no reported underlying conditions. While such cases were observed these deaths were "remarkably uncommon." They note the possibility that these people may have had underlying comorbidities that had not been diagnosed. Nonetheless, these small number of cases do not invalidate the study's conclusion that "for healthy, non-elderly people, the risk of dying from COVID-19 this season has been infinitesimally small." Unfortunately, they note "this is in stark contrast from many news stories that focus on the demise of young people and the panic and horror that these widely reverberated stories are causing."

Personally, I find these stories despicable and indicative of journalistic malpractice.

Finally, the researchers discuss the large fraction of COVID-19 deaths coming from nursing homes. Data show 42-57% of European deaths coming from care homes. Similar evidence is surfacing in the US. The researchers note that "nursing homes and hospitalized patients (nosocomial infection) appear to account for a lion's share of COVID-19 mortality."

I increasingly suspect that this will instigate a flurry of CYA actions by policymakers in weeks ahead. As analysis of the data reveals that most COVID deaths come, not just from the elderly, but from elderly confined to care facilities, then the major policy error of widespread lockdown may become clear to a whole lot of people.

Evidence Fiasco

With a boulder on my shoulder
Feeling kinda older
I tripped a merry-go-round
With this very unpleasing
Sneezing and wheezing
The calliope crashed to the ground
--Manfred Mann's Earth Band

A Stanford MD and professor of biomedicine discusses the 'evidence fiasco' associated with the COVID-19 pandemic. His central premise is that we lack essential data on infection rates that enables effective decision-making. Draconian countermeasures adopted by many countries may be doing more harm than good.

As these pages have suggested, extant data on number of people infected and how the epidemic is spreading are unreliable. Limited testing underestimates the positive case count; number of true infections may differ from counted cases by orders of magnitude. Three months after the virus outbreak, no country has reliable data on the prevalence of the virus in a representative sample of the general population.

That is mind numbing.

The evidence fiasco creates extreme uncertainty about predicting how many will die from COVID-19 because case fatality rates are meaningless when selection bias causes patients with disproportionately severe symptoms and bad outcomes to be tested. Currently, reported death rates from positive cases are far higher than they would be if more tests were administered (the denominator in # deaths/# total cases goes up).

The author suggests that the Diamond Princess cruise ship situation presents an interesting example of a single, closed population where all of its and passengers were quarantined and tested for COVID-19. The author does not include all of the data in his discussion, so I am drawing from reports here. About 3,400 crew and passengers were quarantined on the ship off the coast of Japan for about two weeks. During that time, 696 tested positive for an infection rate of 20%. Of those cases, 410 (59%) were asymptomatic. Seven deaths were attributed to COVID-19, translating into a 1% case mortality rate and a 0.2% death rate for the population. It is also important to note that this population was skewed disproportionately toward elderly people who are much more susceptible to severe COVID-19 effects.

Projecting the Diamond Princess mortality rate onto the age structure of the US population, the author estimates the death rate among people infected with COVID-19 to be 0.125%. However, because the small sample size widens statistical confidence intervals, he estimates that the true death rate ranges from five times lower (0.025%) to five times higher (0.625%). If other uncertainty factors, such as frequency of chronic diseases for tourists and the possibility that some infected passengers in the sample might die after the quarantine period ended, then a reasonable case fatality ratio estimated from the Diamond Princess situation might vary from 0.05% to 1% for the US population at large.

The results suggest a large range of severity possibilities associated with this pandemic. If the true population fatality rate is indeed close to 0.05%, then it would be lower than the seasonal influenza death rate. Locking down the world based on such a death rate would be completely irrational given the economic, financial, and social consequences associated with this countermeasure. Indeed, concerns that the lock down approach constitutes an overreaction to the COVID-19 pandemic are increasingly being raised.

The author's recommendation is straightforward. The best information for good judgment here comes from data on exact levels and trends in prevalence and incidence of epidemic activity. Prevalence of the infection can be understood by testing a random sample of the population for the presence of the COVID-19 virus. I would think tagging those cases as symptomatic/asymptomatic would add further utility. Incidence of the infection can be understood by repeating the exercise at regular intervals to estimate the rate of new infections.

Meanwhile we are making decisions that cost trillions of dollars without the benefit of essential information, which raises the probability that current measures to counter the pandemic will be worse than the disease itself.

Evidence Man

"Let's play another game."
--Frank Dulaney (Body of Evidence)

Nice follow-up piece with Stanford MD and prof John Ioannidis. These pages discussed Ioannidis' mid-March warning of the 'evidence fiasco' surrounding COVID policymaking.

Not surprisingly, he was chastised by the public health establishment for breaking ranks from the party line. Fortunately, Dr Ioannidis did not bow to institutional pressures for compliance and remained on course. Since then, he has authored multiple academic articles on early COVID data analysis and need for better evidence-based policymaking. He has also collaborated with several Stanford colleagues to perform one of the early antibody studies on COVID infection prevalence.

One of the things I find most interesting is that most of Ioannidis' early conclusions--many drawn from analysis of the unique Diamond Princess cruise ship population sample--still hold today.

In the WSJ follow-up piece, Ioannidis notes that, although he enjoys models, "they're very, very low in terms of how much weight we can place on them and how much we can trust them." While models might offer some intuition about situation's early shape, "depending on models for evidence, I think that's a very bad recipe."

He laments that "there's a sort of mob mentality here operating that they just insist that this has to be the end of the world, and it has to be that the sky is falling." Rather than focusing on data and evidence, the mob favors "speculation and science fiction."

Well said, Dr Ioannidis. However, these pages have argued that games of COVID dissonance should be expected. We might also expect that such dissonance will dissipate as the body of evidence continues to grow.

Nineteen

It's hard times befallen
The Soul Survivors
She thinks I'm crazy
But I'm just growing old
--Steely Dan

For those unaware, the nineteen in COVID-19 stands for the year when the first case of this particular strain of coronavirus was detected. The first reported case in Wuhan, China was mid-November of last year, which suggests the virus was circulating freely for at least a week or more in the locality--given its incubation period and asymptomatic nature.

Back in March, bio-medicine folks at Stanford suggested that the virus entered the US far before the first reported domestic case in mid-January. After all, they observed, "tens of thousands of people travelled from Wuhan to the US in December." Assuming that the virus was highly transmissible and that the number of infections doubled roughly every three days, the researchers estimated that "an epidemic seed on Jan. 1 implies that by March 9 about six million people in the U.S. would have been infected."

The bio docs proceeded to argue that those high infection levels, if accurate, would have translated into a much lower COVID-19 mortality rate than what was being reported.

Not surprisingly, officials dismissed their logic, and stuck to the narrative that there was no chance that infection rates were that high that early in the US.

However, evidence continues to grow that the Stanford folks were on the mark. Sero-antibody studies have revealed that COVID infection levels are orders of magnitude higher than official counts. In fact, some people who have tested positive in serology studies, like these people from the Seattle area, are claiming that they displayed clinical symptoms of the virus in December. My sense is that we'll hear more of these stories.

It seems quite likely that the virus was circulating in the US for months before things were shut down. And that the virus has come and gone in tens of millions.

Publish or Perish

Six o'clock, TV hour
Don't get caught in foreign towers
Slash and burn, return
Listen to yourself churn
--REM

Following up on yesterday's post, Stanford prof John Ioannidis has written a paper on the harms of exaggerated information related to COVID-19 and of countermeasures grounded in such information. He includes recommendations for improving evidence-based decision-making related to the pandemic.

It has been accepted for publication in European Journal of Clinical Information. The peer review process was almost certainly expedited in this case to facilitate timely release of a meaningful contribution to the extant problem.

pdf of manuscript here.