Is it really likely that China contained COVID?

Why I think not

This past week there have been a lot of good thought pieces on, essentially, epistemology, or how we know what we know. Particularly, I want to point to Zvi Mowshowitz and Mike Hind (whom I might respond to directly later).

In this vein, I want to talk a little bit about China’s claims to have ~5,700 deaths from 3 January 2020 to 7 February 2022. No, seriously, that’s what the World Health Organization reports.

Note that those colors are TOTAL deaths, not deaths per 100,000 or anything. Is it really feasible that China is in the same total deaths band as Japan or South Korea? Just going by total population, that sounds nuts.

According to graphs on the site, by April of 2020 China had effectively zero deaths until a little blip in summer of 2021. So, out of ~1,400,000,000 (1.4 billion) people in the country where the disease originated and spread before people realized that it was a new thing, on’y ~141,050 cases resulting in ~5,700 dead, or 1 death in 245,614 population. Compare that to the US numbers of 893,970 reported deaths deaths in a population of ~331 million, or about 1 in 370. China’s reported death rate is three orders of magnitude lower than the US. That is what we are asked to believe.

The point here isn’t to say “Grrr, look how bad the WHO and China are!” but rather to highlight some of the problems with data and how we know things, and go through my logic of why those numbers probably are just not true. I think people have developed a strong bias towards trusting piles of numbers presented as data without considering how and why that data came into being, among other problems.

Disclosure: I have worked with data driven systems in industry and academia for around 20 years, and taught about it a bit at the university level. I have a bone to pick with the culture and norms surrounding how data is lionized today; it is possible that when fully revealed the bone will lead to the classification of a new species of megafauna.

Let us begin.

The Case for China’s Successful Containment

I want to briefly outline the arguments for why China is to be believed. Partially this comes from sources like the Zvi above, not because I think he firmly believes it or anything, but rather because he is about the only person who really has outlined why he thinks it is likely that China is to be believed regarding COVID deaths.

1. China tried really hard, and wasn’t afraid to implement really draconian measures that other countries wouldn’t dare, e.g. welding people into apartment buildings. If something would help to stop the spread, the Chinese government wasn’t going to let something as trivial as human rights stop them from doing it.

2. If the Chinese government had failed to stop COVID, we would have noticed. Given the impressive exponential spread of the disease it would be really hard to cover up if everyone was getting it.

3. People outside of China have connections inside China, and those inside do not report higher numbers of deaths or cases.

4. …. Actually that’s about it. I would personally add “Wouldn’t foreign observers see if people were getting and dying of COVID all the time?” but only because that seems to be the implied argument of those who don’t make one of the other three explicitly.

There may be some other reasons for believing China was successful, but I honestly have not heard any outside of those four. I am going to address those in a bit of structure, hopefully before this goes off the rails into an exegesis around how to lie with statistics.

The Case Against Believing China’s Successful Containment Claims

This will be broken into two parts: Why those four arguments are pretty weak, and then why lying about deaths is actually a lot easier than people seem to think, far easier than being accurate in fact.

Argument 1: As to argument 1: “China wants it, and will do whatever it takes to get it, therefor they will get it” there is a huge unspoken assumption here that needs to be questioned: Containment of COVID was possible if only we wanted to do it badly enough.

Is that really true? Is it really possible to contain a novel upper respiratory infection inside a very densely packed and relatively poor country before reliable tests and vaccines are available? (Not to mention the fact that the vaccines don’t prevent transmission and therefore cases, even the Chinese ones so far as I can tell.)

We know that COVID existed within the populace of China before it was identified as a new disease instead of just a local flu. We don’t know exactly how long it was between release and official recognition, or at least I haven’t been able to find that out, but it seems that a few weeks at least is reasonable. Wuhan itself apparently has about 8.5 million people living there, people who would have been passing the disease around themselves for a while before anyone knew what was going on. Is it believable that in a population of 8,500,000 people at the epicenter and before anyone knew what to look for less than 1.6% of the local population caught the disease¹? Wuhan could easily account for China’s entire multi-year case count by itself.

Travel in China is limited, but Wuhan is a pretty major city, and people are going to be going back and forth via trains all the time, carrying the disease along with them while sharing it among the cars, before anyone even knew to take precautions, draconian or otherwise. COVID 19, which in many cases is asymptomatic and in most cases presents as a bad case of flu and so wouldn’t obviously change people’s behaviors, had the time and opportunity to spread quite widely before it was identified.

This is the situation China faced. A new disease with airborne transmission (although that wasn’t known at the time) had been spreading throughout the country starting in a city of millions (although the source might not have been known at the time) for some number of weeks. The infected present as having life threatening respiratory infections, or the flu, or a minor cold, or nothing. Tests to determine whether the infected have the new disease or one of the other common upper respiratory infections don’t exist yet, nor do effective treatments or vaccines that we know of. Oh, and also, you have about 1,000,000,000 people living in cities within 100 miles of your coast, so crazy high population density to deal with.

Even if locking up sick people the moment they display symptoms works, is it really reasonable to think that only 150,000 people of over a billion caught COVID over 2 years? Is it reasonable that under 6,000 died? Is there some other intervention that I am not thinking of that keeps less than 1.6% the population of Wuhan catching COVID even with its head start? That’s just one city, ignoring all the spread to other cities in the time between release and identification of COVID 19, compared to the total number of reported cases over two years.

Argument 2: Let me come back to this one, because it is pretty much the entire later half of the piece. Let me just point out that, unlike say smallpox that leaves tell tale marks, COVID 19 has no obvious external signs that allow for easy identification of victims without a test, along with asymptomatic cases, especially in children. Would we really notice if China experienced even a super bad flu season without them telling us?

Argument 3: True, people inside oppressive regimes are often the only good source of information about what is going on inside those countries. See Aleksandr Solzhenitsyn for example. Indeed, early in the pandemic we in the west got word of doctors getting disappeared after talking about a possibly novel disease that was filling up hospitals and the images of people getting welded inside their apartment buildings from individuals before the media reported on the images. Reports from friends and family inside China have (anecdotally at least) not featured large numbers of people getting COVID or dying from it.

But why would they? As mentioned above in 2, for most people COVID looks just like the flu, perhaps a worse one than usual. With COVID having no symptoms to distinguish it from other flus without a test, observers can’t tell who has it or not. Death is a pretty obvious symptom, but sorting out just what killed someone is a bit difficult. In the US we probably over count COVID deaths; note that the CDC reports the number of people that died “with COVID,” not from. How do we know if someone died with or from COVID? We test them (or apparently in the early pandemic just assume? Things were sketchy then.) So again, if your neighbor, who was probably elderly to begin with, dies with flu like symptoms, are you going to know it was COVID instead of some combination of flu, old age, and the half dozen other ailments of the elderly?

I suspect probably not. Without people getting tested and being told they have COVID they can’t really know if they have it, and observers can’t know others have it without either seeing the test results or being told. If you lived in China and knew they forcibly locked people inside their homes if they caught it, and thought you had COVID, would you tell your neighbor your thoughts?

On top of that, even with the US’ much higher death rate of about 1 in 370, most people do not personally know someone who has died of COVID. Humans have a Dunbar’s Number of 120-150 or so, less than half the death rate, so while some US people might know a few people who died of COVID, most others won’t know any. You might know a person who knows a person who died, but knowing someone close enough to be sure if they are not inclined to tell just anyone is not terribly likely.

It strikes me then as rather unlikely that anyone could report from anecdotal evidence a different situation than what the government is reporting. Recall that in the US there are wildly varying estimates of the COVID death rate given by survey respondents, and we publish the numbers here all the time.

As a further point, China has already demonstrated a great deal of willingness and skill in censoring information people can share digitally. How many people with access to COVID case numbers that contradict the government are going to publish them? How long with they live if they do? You could perhaps get around this by calling people and speaking by phone directly, all while hoping no one is listening in on your conversation. That does not seem very likely to me, considering that the people with access to the real test numbers are probably already under scrutiny. It isn’t as though Joe Average is going to see the real numbers on the news and call up his relatives in the USA to let them know of the discrepancy.

Argument 4: Is it likely that lots of foreign nationals are working in Chinese hospitals, double checking the case numbers and reporting back to anyone? Roll that around in your head a little bit, and ask yourself whether the CCP is likely to allow foreign observers around any areas they don’t want outsiders to know about, and then ask yourself why anyone would pay people to hang around Chinese hospitals to track random health outcomes in the first place. No, all data on countries’ health outcomes are reported by the countries, except in countries that can’t track those things themselves. In that case maybe some NGOs do some reporting, or it just doesn’t get reported for areas outside the major cities, etc. Do you think it likely that the CCP allows a lot of NGOs to report information that might not be to the CCP’s liking?

So that’s the summary… here’s the long bit

Argument 1 is really the only one that I think might hold water, but only because I just don’t know enough about how those sorts of things work. There might be some way that a country could have nipped COVID’s spread in the bud, while that bud was already all up inside the country and had been for some time. I am actively allowing for failures of imagination on my part here.

I cannot assign a high probability of that being the case, however. We humans have a strong bias towards “We know how to solve X, if only we would try hard enough!” claims, where X is hunger, poverty, drug abuse, crime, pollution, scarcity, whatever… insert problem here. In every single case, however, it turns out we don’t have the solution, and when tried what solutions we have either don’t work at all or come with such stark costs that they are worse than the disease. We already do most of the things that are obvious fixes, and all the good ideas left are the stuff that is a good idea but looks like a bad one; everything else is bad ideas. When there is little knowledge of what is going on and what can be done, it seems very unlikely that we are going to pick the good idea that looks like a bad one by lucky chance.

We humans also have a strong anti-market bias, although in this case I would phrase that as a strong pro-top down bias. We really want problems to be solved by someone coming in and telling everyone what to do, and we really like solutions to that effect. Leaders that say “Look, I can’t do anything about this problem. You guys are going to have to do the best you can,” don’t get to be leaders very long, even if they can’t do a damned thing about the problem except make it worse. We constantly are electing leaders who promise to solve X, Y and Z, and when they fail spectacularly either make excuses for them or toss them out of office, only to elect someone who promises exactly the same thing². Humility and a good estimate of what people in power can accomplish is not what we look for in leaders.

Those two biases together make me very skeptical that there actually are some unknown solutions to contain COVID that only require a strong leader to implement, as opposed to people really wanting to believe that those solutions exist. When people really want something to be true, but there is no clear explanation of how it could be, I am inclined to doubt its existence more strongly. I disbelieve claims of proper dinosaurs still running around in out of the way places more strongly than I disbelieve claims of undiscovered species of deer, if only because people really want the former to be true and are likely to distort their thinking in favor of it.

It is arguments 2-4 that I think most obviously demand lowering the probability that China’s stated numbers reflect reality. Arguments 2-4 are reasons to believe that the solution necessary for Argument 1 exists and was successfully executed (two very different things!) If it was raining the sidewalk would be wet, after all, and if the sidewalk looks dry, well, Q.E.D.

Yet maybe there is a roof over that sidewalk, and maybe it is a whole lot easier to cover up COVID cases and deaths than it is to prevent them.

So how easy is it to simply lie about how many COVID 19 cases and deaths there are?

Answer: pretty easy, and easier than actually knowing how many cases and deaths there really were.

Let’s get back the to only way to know you have COVID: testing. Let’s look at how that chain works, and then consider how that information gets collected and disseminated to the end result that the WHO reports “China reported X COVID cases and Y COVID deaths.”

Imagine you are a Chinese citizen, and you are feeling pretty sick. You fear you might have caught the ‘rona, but how can you be sure? You have to go to a hospital or other clinic, tell the staff you think you might have COVID, get tested, etc.

Do you do that? Are you alright with the possibility that you might be forcibly quarantined, missing work and other activities? The more draconian the containment measures, the higher the costs for people to get tested in the first place. Besides, maybe it is just a cold, and you wouldn’t want to get a false positive and suffer for nothing. Perhaps it is best to just take a few days off work if you can, or just grin and bear it. (Let’s call this decision to not even go in for testing Failure Point 1, or FP1 because I already know I am not typing all that out again.)

Let’s say you do go to the doctor and get tested. It comes back positive, and you get locked up. Does the order come with a little note saying “Please contact everyone you have ever met and let them know you have this disease we are definitely succeeding in keeping contained such that no one gets it,” or a note that says “If you tell anyone you won’t ever have internet again. Or a job. Or we will send you to a reeducation camp. Or we will just kill you and make your entire family disappear. Whatever. Love, CCP”? It seems that if you want to keep people quiet that’s one way that has reliably worked for China. Let’s call this communication of testing results to third parties FP2.

But wait, what if your doctor says “Nope, test came back negative. You just have the flu.” You go home, maybe take some flu medicine and drink some soup, and live life as normal. Whether or not you have COVID or just the flu that is probably going to work in the vast majority of cases, so what do you know? If all the doctors got a little memo saying “By the way, none of your patients better get COVID. Love, CCP” how many are going to start telling patients they have it? Even if the Hippocratic Oath drives doctors to actually test their patients and honestly tell them the results on the sly, how many of those patients are going to open their mouths (see FP2)? So yea, doctors just saying “Nope, no COVID here, everything is fine,” is FP3.

But wait! What if the patients die in a hospital? That’s hard to pass off as the flu! I dunno… let me test this patient… nope, test came back negative. See? I have a test right here that says negative. No COVID here, thanks, just other causes.

Now, let’s say the doctor does do the test and it comes back positive, and he tells the patient, and the patient goes off to quarantine. We have dodged the first three failure points so we might get some good data. But that data needs to be collected. The test results are probably reported to the local government officials who pass it up the chain of command, either by the testing lab or the doctor’s offices, probably both. Is it likely those officials also have orders to the tune of “Oh, go ahead and make the numbers for your region publicly available online for major international organizations”? I am going to go with no, and in fact those numbers probably have the Chinese version of “TOP SECRET GOVERNMENT INFORMATION” written all over them, and everyone touching them knows sharing them out would be a very bad life decision.

This has two implications. Firstly, regional officials have a fair bit of leeway to massage their numbers before passing them up. If your bosses say “You had better contain COVID, or else!” those officials are going to make sure those numbers say what needs to be said before handing them up. This sort of willing blindness bedevils all harsh management styles; when everyone is scared to disappoint the boss, the boss simply never hears disappointing things, true or not.

The second implication is that the numbers have nothing like peer review³; no one outside the government chain can look at the numbers and notice errors or fraud. This is of course by design, but it is worth noting that in most countries more transparency means hiding things is vastly more difficult. Who outside the government is going to look at the numbers and say “You know, this just doesn’t add up or match what I am seeing on the ground”? Even if they were suicidal enough to go against the CCP that way, there is nothing they could really look at, no using the reports’ own numbers to show discrepancies.

So let’s call that non-transparent information pipeline FP4.

Next, let’s say all that testing got done perfectly and passed on up to the national reporting service data base, and the national leadership is looking at it and saying “Hmmm, yup… lots of cases down there… deaths going down here, but cases way up…” using the numbers for all the purposes governments might want to. Is the CCPs next step to say “Alright, let’s just upload all this raw data to the World Health Organization web site so they can see how well we are containing this”?

No. No of course not. If you want people to think you are containing the disease better than you are, you simply misreport the numbers! Who is the WHO going go to to check? Who is going to challenge your claims? With what data? All the “official” numbers for countries are self reported. If the USA reports 900,000 deaths and China 5,700, who is to say otherwise? Is the WHO going to say “Bullshit, that number is crazy. Whip out all the corpses from the past two years so we can test them for COVID.” If some group said “Let’s go to China and randomly test a few thousand people to see if they have had COVID so we can estimate the population rate” how long do you think that would go on before things got disappeary? Failure point 5.

Government self reporting is the ultimate failure point. It would be like giving students a test, asking them to grade it themselves, then accepting their score without asking to see the exam. Oh, you got an A, good job. Everyone got A’s? Well, ok, well done you!⁴

It goes both ways too. If the CCP wants everyone to think they have heroically saved their people from the scourge of COVID unlike every other country, they can report whatever ridiculously low number they want. If the US government wants everyone to panic and drive polarization and support for authoritarian measures, they can report much higher numbers than reality (although with the limitation that many groups can see and generate data to contest.) All government performance metrics, from GDP to COVID cases to police shootings to inflation rates, are self reported. We hope they are accurate, and barring that hope that third party sources can check the numbers a bit to keep those in power from lying too much. In China’s case, the CCP is the only party checking anything, and the only party that matters. We would do well to remember that.

Speaking of GDP…

We have the 5 Failure Points suggesting that lying about case and death numbers, along with just not knowing, is really very easy for the CCP. There is something else that suggests that draconian containment measures might not have been done enough: reported Chinese GDP didn’t really go negative in Q1 of 2020 compared to other places, and not at all in the summer of 2021, their second wave of cases and deaths, remember. We see a -10% rate of GDP growth for Q1 2020 then 11.7% growth in Q2, compared to -9% for Q2 in the US, which was followed by -2.8 and -2.4% the next two quarters. China never shows negative GDP growth for the rest of the series.

One might argue “Sure, their GDP was pretty stable, but that’s precisely because they were so good at locking down and controlling the pandemic!” Yet, that ignores the central trade off everyone else in the world pays for lockdowns, namely economic activity grinds to a halt. Other nations had less severe lockdowns, quarantines and other rules precisely because the economic impact was too great; indeed those we had turned out to be a lot worse than most expected. Yet somehow China had only a slightly worse first bout with the disease (Q1 for China, as the US only had noticeable cases in Q2) than the US, and then bounced back immediately with the best quarter in years, never to go negative again. Sure, anything is possible, but if that doesn’t raise your eyebrow it is possible nothing ever will again.

Probabilities are good, right?

Let’s assign some:

• Probability that there exists some strategy to contain a highly infectious airborne disease after it has been at large in the very densely packed population for who knows how long, and despite having very little information on the disease at the time of decision actually managing to identify and execute that strategy: 2%

  • Probability of above, but given that a lot more information on the nature of the disease was available to decision makers than was given to outsiders and foreign governments: 5-10%

• Probability that a government with highly centralized power, that has demonstrated a willingness to censor and oppress its people without regard to normal concepts of human rights and international approval, as well as demonstrated ability to censor and control information within its borders with the aid of international companies, upon deciding that it wants to claim success in containing the disease via very low case and death numbers will be able to do so without outsiders demonstrating otherwise: 75%

  • Probability of above, given that most nations don’t care much, and the World Health Organization was accused of being in that same government’s pocket before the pandemic: 90%

So there’s the logic for me. I think it is possible, though very unlikely, that China actually only has 5,700 COVID deaths after 2 years. I just find it much, much more likely that they are simply lying about the whole thing, because it would be really easy for them.⁵

---

Subscribe for more hard hitting analysis! Next week, I delve into whether or not my five year old really doesn’t know what happened regarding the spilled milk currently puddling around her feet.

And while I am at it, I wanted to say that I am not on Twitter and only use Facebook to keep up with the local SCA groups, so if you wanted to share this with a friend who might find it interesting, I would appreciate it. I am really uncomfortable with self marketing in general, and combined with a deep distaste for the main social media sites I pretty much rely on people clicking comments from ACX, Zvi or Arnold Kling’s blogs for traffic. It’s good because it encourages me to post good comments, but bad because it would be kind of cool to get more readership and comments. Because I love the comments like a big blue monster loves cookies.

Thanks for reading!

1. ~141,000 cases over two years in China all together compared to the 8.5 million people of Wuhan.

2. How’s that War on Poverty going? Or that War on Drugs? I guess that it has only been 4-5 decades so far, so we must be patient.

3. For lack of a better phrase.

4. In fact, it would be like having teachers in charge of teaching students things, then having the teachers make the tests to see how well the students learned what the teachers were supposed to teach them… which is what we do of course.

5. Oh, I didn’t get into the fact that China is hardly an island, with a REALLY long land border that covers some really wild places. How porous that border is, I have no idea, but I would bet there is a fair bit of petty commerce that goes on across it.