That paper about NPIs again

This is a follow up to my previous post about a paper that had some very surprising result about non-pharmaceutical interventions (NPI) and how strict lockdowns seemed to have less impact than light lockdowns.

The crux of this entire thing is that they feed delta(log(cumulative)) into a linear regression model for the log of the growth rate (g) but there is no linear relationship between delta(log(cumulative)) and g when g is moving around. There is if you use delta(log(daily)) but they didn’t. The result is that it is heavily biased towards NPIs which happen earlier and of course everywhere starts off by trying light NPIs and switches to heavier later. Hence the paper’s surprising results.

What’s new in this post is that I’ve fired up R and recreated the linear regression from the paper. The results are spectacular. I had no idea how biased this was.

The R notebook with several simulated epidemics is over here. The highlights are:

• 2 equally effective NPIs come out with an estimated impact on g of -0.2063621 and -0.0646044 respectively (with the later one losing out)
• An NPI that increases growth by 5% looks better than an NPI that immediately stops the epidemic!

That’s right, with this broken methodology, an NPI that makes things worse beats an NPI that ends the epidemic, simply because it happens earlier.

Super-spreader event on Japanese flight with strong evidence of airborne spread

Japan’s National Institute for Infectious Diseases published an epidemiological investigation showing long range airborne transmission on a plane. It occurred in Mar 2020. It was published in Oct 2020, in Japanese only. Google translate makes it pretty readable.

Main takeaways:

• started off following the “2 row rule” but after finding a bunch of infections near the index case they expanded and expanded and eventually tested 122 out of 141 passengers.
• found 14 PCR positive passengers.
• found several others with symptoms who they did not test.
• confirmed that all positives were an RNA match for the index case.
• it travelled far - furthest infection was 16 rows in front of index case with another 4 infections 9 rows in front and one 6 rows behind. Also, some of the symptomatic, untested people were far from the index case.
• index case had a severe cough but did not wear a mask

I have translated the seating diagram published in the report from Japanese to English. I don’t know why there are 2 “3rd tests”, that was in the original Japanese.

Seating diagram of flight with test results etc

In the discussion they mention droplet and “マイクロ飛沫感染” - micro-droplet - infection in-flight. They also say that they didn’t have aircondtioning and ventilation information or pre-boarding passenger interaction information.

I often hear that “Japan understood this was airborne from the start” and it’s half true - some scientists here knew and the “3 Cs” guidance has been good but a lot of the response has been focused on droplets and fomites, cleaning and perpex barriers.

All of the information in this report was available in March 2020. It’s really disappointing that this was not published sooner and more broadly as it seems like it would have been strong evidence for airborne spread and also strong evidence against the safety of air travel. Especially given the full RNA analysis and almost complete test coverage os passengers

How not to calculate excess mortality

I got in a twitter argument with someone about COVID19 and they threw a surprising stat at me. South Korea had over 20k excess deaths this year. This made no sense to me. SK is maniacal about testing, their official COVID19 death toll for 2020 was 917. Did they miss 20x that many? Was there some other big killer? Is it a statistical blip? The answer is "none of the above".

The source was this WSJ article. It's pay-walled but the key information is this info-graphic which purports to show that many countries have vast quantities of excess death, above their official COVID numbers.

So the world has massively under-counted COVID19 deaths? Probably but the other key information is how they calculated excess deaths

Methodology: To analyze the pandemic’s toll, the Journal compiled weekly or monthly death data for 2020 and for 2015-19, where available. Most of the data was collected from national statistical agencies, either directly or indirectly through inter-governmental or academic groups. In a handful of nations, data was collected by health data organizations or local analysts. Epidemiologists use several methods to calculate excess deaths, adjusting for age composition, incomplete data and other factors. The Journal used a straightforward method, summing deaths for the portion of 2020 available and subtracting from that total the average number of deaths that occurred in the same span of each year from 2015-19. When the result falls below zero—when the 2020 death total fell below the average—some countries adjust the result to zero, boosting excess death totals. The Journal did not adjust in those cases. All totals are based on actual counts and comparisons. For some nations, the average was based on three or four recent years, typically 2016-19.

I have bolded the important part.

Unfortunately this straight-forward method is a fundamentally flawed methodology (did they not talk to an epidemiologist before publishing?). It ignores the fact that most countries have underlying mortality trends due to their demographics. Using their methodology South Korea has +24k excess death in 2020 but guess what, it had +21k excess death in 2019! This is what SK's recent excess deaths look like with WSJ's methodology. Here is the sheet if you want to explore.

As you can see, SK's mortality is rising pretty rapidly, presumably due to a population explosion in the 50s and 60s. I believe most countries are similar. This makes all of the numbers in the article somewhere between questionable and meaningless.

Applying their methodology to the whole world, we get an excess of 1.7M in 2020 and 1.3M in 2019. They did a subset of the world and got 2.8M which is also interesting, I don't know where that discrepancy comes from.

So while the world has surely under-counted official COVID death, WSJ's figures could almost be anything, an over-count or an under-count. What's bizarre is that they said "Epidemiologists use several methods to calculate excess deaths, adjusting for age composition, incomplete data and other factors" and then proceeded to just do it wrong anyway.

Addendum: New Zealand is quite similar. Here's the data

Here we go again

TL;DR: More testing means much highers numbers but slower growth. We are probably going to have to wait quite a while before any emergency is declared. In the end, the spread will probably be much greater than last time.

Things are not entirely the same as before but not entirely different either.

The good:

Testing is way up from before and testing policy has changed. You can get tested on demand privately, if you want to pay for it. It seems getting tested publicly is now easy too. I know of two recent stories where people with mild symptoms got tested and it was pretty easy. When doing contact tracing they now test all contacts, not just those with symptoms. Combined with the targeting of some high risk groups for mass testing and this certainly has a large impact on the number of cases discovered. The official goal of the govt now is to identify positive cases as soon as possible, including asymptomatic cases.

Even though the numbers are much higher than before, it does seem like the rate of growth is lower. This might seem odd but it makes sense. If you go from finding and isolating 10% of infections to 30% of infections your numbers triple on paper but you reduce the speed of the spread (10% and 30% are very made-up numbers).

Treatment is improving. The govt has approved a couple of medicines that help suppress the violent immune over-reaction. So we may see fewer serious cases and fewer deaths for the same number of infections.

The mixed:

This time round, it’s spreading in younger people. Some possible reasons:

• younger people get no symptoms or light symptoms and we refused to test them last time around. Maybe nothing has changed and it’s just that we are detecting it in young people.
• (not sure if this is actually true) older people are sheltering more than before
• (not sure if this is actually true) nursing and caring staff are being tested regularly so infections are not spreading into hospitals and care homes
• targeting of host clubs etc biases detection towards the young
• when you shut everything down, undetected COVID mostly survives in young people. It could take several weeks for the infection to spread from young social circles back to old social circles.

If they can keep the infection away from old people, it will keep the ICU wards empty and the deaths low.

Hospital capacity has been increased. Many more hospitals are now designated to take COVID cases. Obviously more preparation and available care is a good thing. The downside is that the govt clearly knows that it’s not going to win and is betting on higher capacity to let it drag this round out as long as possible.

The bad:

There’s no doubt that we are losing again. A few weeks ago I was hoping that maybe the spike was mostly due to the testing change. It wasn’t.

The “serious” (i.e. ICU/ECMO) number has stopped falling and has been consistently rising for a week. It’s still single digits per day but it’s growing. Those people going into ICU in the last week were infected 2-3 weeks ago.

Testing capacity is growing incredibly slowly. They started publishing the numbers a little over a month ago and it has grown from 28254 to 33030 in that time (column P). Tokyo’s all-time record is 4,507. Ireland tests about 4500/day and finds 20-40 new cases per day (Tokyo finds 250-300/dat currently). Ireland is 1/3 the size of Tokyo. The rest of Japan’s testing numbers are even worse.

The avg daily testing rate (Column Q) is about half of the official capacity. It seems likely that Tokyo is using much more than half of its capacity and will soon max out. Then they will have to get stingy on tests again, e.g. no more proactive testing of host clubs or maybe make it hard for mild cases to get tested.

The national govt really don’t seem to care or have any ability to help. In fact they seem actively harmful. No responsible govt in any country is running a domestic tourism campaign right now, they’re all watching the numbers as they carefully try to restore their economies. Japan’s govt seems to see everything as secondary to the economy and their campaign-funding lobby groups. I don’t think they put any value on quality of life or are paying attention to the fact that COVID has many non-fatal but extremely nasty and chronic outcomes.

Speculation:

Present

The current numbers are not good but the numbers don’t mean the same thing anymore. My thinking had been that the true infection rate is maybe 10x what was being detected. Japan’s Case Fatality Rate was about 5% while it seems like in reality, COVID-19 has a fatality rate of about .2%-.5% that implies that the real case numbers were 10x bigger. That assumes that the virus is just as fatal here as anywhere else and that Japan’s fatality numbers are accurate (Japan was extremely stingy with testing, the true number of deaths may be much higher).

The new testing policy changes that. We might be detecting 2x or even 5x as much as previously, especially in Tokyo where they proactively test some high risk groups.

So we’re closer to the bottom of the curve than we might seem. We’re also climbing more slowly.

Near Future

I expect the govt are going to try to squeeze as much economic value as they can out of this round until one of these happens

• ambulances cannot find hospitals for patients
• ICU/ECMO capacity runs out

What’s extremely dangerous here is that it takes about 2-3 weeks for someone to go from infection to ICU. This means that we have to shut down 3 weeks before capacity runs out. If we screw up, we may exceed capacity. Exceeding capacity for ICU means picking who we try to save and who we leave to die.

There are a couple of ways to screw up.

• Wait too long.
• Have a sudden spike. E.g. as Tokyo runs out of testing capacity the rate of spread may increase, causing a numbers to spike.
• The SOE (state of emergency) is not as effective as expected. People are tired and don’t want to stay home. Young people are already acting like there is no problem.

Also the extra capacity means that the consequences of a screw up could be much larger. Essentially we will be going much faster than before when we hit the brakes.

The SOE was declared on 2020-04-16. By that day 185 people had died (officially). By 2020-06-16 927 had died. Now it’s 996. That means that 75% of deaths occurred in the two months after hitting the brakes. If we keep going until we fill ICU capacity the death toll of the second round could be enormous.

We’ll probably see deaths start to move again soon but hopefully more slowly, given better treatment and younger patients.

In the end, I think the infection will spread much more widely than April. Given the reduced speed of spread, it might take a long time for the govt to declare an SOE. Maybe 4 weeks, as a crazy estimate that I will surely regret. I suspect Tokyo is soon going to start racing ahead of the rest of the country (moreso than it already is). So Tokyo might go into SOE earlier than that.

Until then we will have an extended period with a very large number of asymptomatic cases wandering around. That will make this round much more dangerous for people with preexisting conditions. This Bloomberg story about South Korea describes someone getting COVID-19 from a person in a neighbouring karaoke box! I was hoping I would be going out and doing fun stuff by now but I’m staying put for now.