Streetwise Professor

March 22, 2020

If Policymakers are Going to Crater the World Economy, They Should At Least Make That Decision Based on Reliable Data

Filed under: China,Economics,Politics,Regulation — cpirrong @ 6:51 pm

I’ve expressed considerable skepticism about relying on test data to craft COVID-19 (AKA CCPVID-19) policy responses. This note formalizes the basis for my skepticism. Testing data would provide an accurate measure of the prevalence of severe infection if (a) the tests had low rates of false positives and false negatives, and (b) testing was random. Neither condition is remotely correct. Meaning that the test-based statistics are an extremely poor guide for policymakers, and a particularly dubious basis for driving the world economy into a depression, at the cost of trillions of dollars.

So what should we look at? If this is a particularly prevalent, virulent, and deadly respiratory disease, it will result in elevated levels of hospital admissions or physician visits for respiratory illness, and elevated levels of death from respiratory causes. That’s what we should be looking at. Or more to the point, what policymakers should be looking at. Is this a particularly deadly and widespread disease? If it is, it will have measurable effects on mortality and hospital admissions.

The CDC does collect data on influenza. Unfortunately, many of the statistics condition on a positive influenza test. For example, hospital admissions with a positive influenza test. That is not helpful, because we are focused on something other than the influenzas the CDC tracks. But the CDC does report deaths from influenza and pneumonia. That is more useful, as a deadly new respiratory illness should lead to higher pneumonia death rates.

Through last week, these data demonstrate little elevation on a national or regional basis. There was a spike in deaths above the “threshold” level early in 2020 (where the threshold basically is at the 5 percent significance level above the seasonally adjusted baseline), but subsequently it converged almost back to the baseline:

It is particularly interesting to compare 2019-20 with 2017-18. Heretofore, 2019-20 compares very favorably to that year, and even to less extreme years 2016-17 and 2018-19. Through now, in other words, 2019-20 does not look at all unusual.

The CDC also tracks data on those seeking medical treatment for flu-like symptoms: these data do not require a positive influenza test, and thus should reflect people suffering flu-like symptoms caused by something other than the flu. These data show somewhat higher levels for 2019-20 compared to previous years (except for 2017-18, which was much higher), but not extremely so. The main worrying aspect to the 2019-20 data is that they do not appear to be declining as rapidly with the approach of spring as in prior years. But the data do not exhibit a huge spike–they are just declining less rapidly than in prior years.

Yes, these data are backwards looking. I can imagine scenarios, such as the late introduction of CCPVID-19 into the US, which would mean that the wave of deaths/illness would not be manifest in the data, as it is still to come. But there are indications that the virus has been on the loose in the US at least since mid-January, and given its existence in China no later than mid-November, it could have been present in the US prior to mid-January. If it is indeed highly contagious and deadly, it should be leaving tracks in the mortality data.

It would be highly informative to have such data for other countries. I am not aware of it in as accessible a form as is provided by the CDC. If anyone can point me to it, that would be greatly appreciated.

You might argue that I am whistling past the graveyard. All I can say is that the data that alarmists point to is highly unreliable (and inherently so), and the reliable data as of yet demonstrate nothing out of the ordinary on the dimension that really matters–people dying from respiratory ailments.

What I can say with considerable confidence is that policymaking is driven by flawed data, and that there are types of data that would be more informative, and which are not infected by (deliberate choice of words) the problems inherent in the flawed data that dominates public discourse, and apparently dominates public policymaking. Produce that data. Disseminate that data. Make sure policymakers are aware of it, and are aware of the deficiencies of the data we hear about 24/7.

Print Friendly, PDF & Email

19 Comments »

  1. The data is indeed flawed, primarily because of the variations in the availability of testing results.

    But it is also clear that the health care system is reaching crisis points in several parts of the country and that appropriate supplies of PPE are lacking everywhere.

    As I noted in a comment to your previous post, lacking reliable data, I am focusing for the time being on the daily death rate.

    It took Italy 18 days (Feb 21-March 9) to reach 366 deaths on 7,375 confirmed cases at the time.

    On the other hand, it took the U.S. 22 days (Feb 29-March 22) to reach 340 deaths on 26,747 confirmed cases.

    Thus, the U.S. is at the same point at which Italy’s death rate exploded, resulting in 4,827 deaths as of March 22.

    So, buckle up. The next couple of weeks are going to be interesting.

    Comment by Tom Kirkendall — March 22, 2020 @ 7:08 pm

  2. @Tom. I am going to follow up on death rates, and comparability across countries. There are very different reporting regimes. Virtually all of the dead have at least one, and usually many sources of co-morbidity. This is especially true of the aged. In Italy, if someone has multiple illnesses including a positive coronavirus diagnosis, and they die, the Italians report the death as a covid19 death. Conversely, the Germans report the death as due to the underlying health condition (e.g., heart disease).

    Based on this subjectivity and arbitrariness of categorization of cause of death, the best metric is overall death rates.

    Comment by cpirrong — March 22, 2020 @ 7:44 pm

  3. What you say about death rates is true. My old internist used to say that, in his 50-year career, he had never seen anyone die of the flu – it was always a more serious complication that arose after the patient’s immune system had been compromised by the flu. But in an imperfect data environment, the death rate is probably the most reliable metric.

    The following is the death info as of this morning (March 23) from Worldometer:

    It took Italy 18 days (Feb 21-March 9) to reach 463 deaths on 9,172 confirmed cases at the time.

    It took U.S. 24 days (Feb 29-March 23) to reach 458 deaths on 35,070 confirmed cases.

    Over the past 14 days (March 10-23), Italy’s deaths increased to 5,746 on 59,138 confirmed cases.

    Comment by Tom Kirkendall — March 23, 2020 @ 4:43 am

  4. The 2nd and 3rd links provided by Streetwise Professor are broken. The correct 2nd link is just missing the final “l” of “.html” and so the unbroken link is https://gis.cdc.gov/grasp/fluview/mortality.html.

    Using that link, note that the screen capture graph pasted into Streetwise Professor’s post has a vertical scale going from 4% to 11% of deaths (main ticks every 2% starting at 4%).

    To assist in the interpretation of the national death statistics, here is an annotated (by []) cut and paste of the “Disclaimer” pop-up window.

    “The seasonal baseline [blue curve] of P&I deaths is calculated using a periodic regression model that incorporates a robust regression procedure applied to data from the previous five years. An increase of 1.645 standard deviations above the seasonal baseline of P&I deaths is considered the “epidemic threshold,” [black curve] i.e., the point at which the observed proportion of deaths attributed to pneumonia or influenza was significantly higher than would be expected at that time of the year in the absence of substantial influenza-related mortality. Baselines and thresholds are calculated at the national and regional level and by age group.”

    This leaves the red line as the actually reported deaths per week as a percentage of total deaths per week. Note that the last data point on the red curve (Week 10) is based on only around 77% of the expected category and total deaths – because of administrative delay in registrations and processing. Also, that website does not post any death information relating to the immediately preceding 2 weeks, because of (until later) substantially incomplete reporting.

    Note also that Streetwise Professor’s screen capture paste is of deaths accumulated over all age groups. Separate plots are available on the website for age groups <18years, 18..64years and 65+years. Interestingly, the age category 18..64years shows more worrying 2019/20 deaths than the other categories (in fact more worrying in all years except 2016/17). The interpretation of these different plots is difficult.

    This means that seriously interested readers should follow Streetwise Professor's 2nd link (corrected as above) and then play around with the data display options.

    Best regards

    Comment by Nigel Sedgwick — March 23, 2020 @ 4:47 am

  5. And here is the NY Times daily tracking model, which uses a logarithmic scale: https://www.nytimes.com/interactive/2020/03/21/upshot/coronavirus-deaths-by-country.html

    Comment by Tom Kirkendall — March 23, 2020 @ 6:06 am

  6. Perhaps your colleagues at JHU could point you in the right direction? I haven’t looked in detail to see what sources they using for their stats.

    Also, surely the epidemiology community are all over this?

    Comment by David Mercer — March 23, 2020 @ 7:14 am

  7. Watts Up With That has a chart they are updating daily to compare the most interesting cases (and which will allow us to see if state-wide lockdowns have any appreciable effect). Most fascinating of all is the light blue section at the top, which bounds the normal flu death rate. So far, no country has exceeded it, and we are months into this thing. If only Italy exceeds it, this will go down as one of the most mishandled events in world history: https://wattsupwiththat.com/daily-coronavirus-covid-19-data-graph-page/#001

    Comment by Josh Postema — March 23, 2020 @ 9:17 am

  8. Whatever conclusion I reach I shall hold with low confidence because the data are incomplete, biased, and plagued by artefacts (and maybe even lies).

    If I’ve said that already on this blog, apologies.

    Meantime the British papers carry stories saying that Boris has been forced into draconian measures by his cabinet, so

    (i) It’s good to hear that we still have cabinet government,

    (ii) But it’s a pity that Boris’s principled belief in personal liberty is not more widely shared. (As some wag said, it’s noteworthy because Boris has so few principled beliefs.)

    Comment by dearieme — March 23, 2020 @ 9:22 am

  9. FWIW, my gut feel is that the real data is way worse than what is being reported. Its clear is that governments across the globe are manipulating their data, which means only one thing.

    Comment by David Mercer — March 23, 2020 @ 10:43 am

  10. I don’t think it’s clear that governments are manipulating data. That seems too impractical and it would take just a single whistleblower (who would be praised endlessly) to stop the whole conspiracy. South Korea has had the most sophisticated testing (and I see no reason they’d lie about their numbers), and their mortality rate for this is 0.6, which puts it in the “bad flu seasons” range, although with far fewer young people and children dying.

    Comment by Josh Postema — March 23, 2020 @ 10:50 am

  11. @Josh-I never said they were manipulating the data. I am saying they are looking at the wrong data.

    Comment by cpirrong — March 23, 2020 @ 1:08 pm

  12. @Josh–Thanks for this. I look at WUWT regularly but missed this one. I’ll review.

    Comment by cpirrong — March 23, 2020 @ 1:09 pm

  13. @Nigel. Thanks. I’ll fix the links.

    Comment by cpirrong — March 23, 2020 @ 1:11 pm

  14. “I never said they were manipulating the data. I am saying they are looking at the wrong data”

    Sorry, I was responding to David. I should have referenced him.

    Comment by Josh Postema — March 23, 2020 @ 1:32 pm

  15. @Josh–I thought that might be the case but wasn’t sure. No problem.

    Comment by cpirrong — March 23, 2020 @ 3:44 pm

  16. @Nigel-Re the 18-64 flu+pneumonia death chart. Yes, it is above the “threshold” for parts of 2019-20, but the most recent data is right on the baseline. Not what you would expect from a galloping virus: that should lead to divergence from the mean (i.e., the baseline) not reversion to the mean.

    I’m also a little skeptical about the adjustment for variation used to create the threshold. There are large excursions of the actual deaths above the threshold in every year 2015-2020 except for 2016-2017. If that was a real confidence level, you would expect to see such divergences only 1 year out of 20. That is, the threshold should almost be an envelope of the observed data, with only a few instances of breakout.

    Moreover, the 2019-2020 excursion is smaller than 2015-2016 and 2017-2018, and about the same as in 2018-2019.

    My eyeballing of that data suggests that the seasonal adjustment is inadequate. The spikes above the threshold occur about the same time every year, and disappear about the same time. If they are using a kernel smoother (which the graph suggests), they are over-smoothing, i.e., they need to reduce the bandwidth.

    My guess is that they are using a constant bandwidth, but you need a tighter bandwidth in the peak seasons than in the off peak seasons. They way they are doing it is assigning too much weight to spring and fall death rates when calculating the average for the winter.

    Alternatively, or maybe also, they are assuming that the variance is constant throughout the year.

    Through experience (e.g., electricity load and price data) I know there are empirical challenges to seasonal adjustments. In particular, effectively your true number of observations is closer to the number of years in your sample, not the number of days or weeks.

    Comment by cpirrong — March 23, 2020 @ 6:35 pm

  17. Craig,

    LOVED your last piece on this. However, I think you have this one wrong.

    Your initial piece has it right where we have an ethical choice to make for the population here as the financial impact may wind up doing more data than nCoV ultimately does. However, straying from this point and talking about the death rate here is fundamentally flawed. The disease itself reduces and or eliminates your lungs ability to pump excess fluid out by targeting ACE2 receptors in said lungs. There are MANY unknowns to this virus that we have yet to fully characterize that show the risk of letting it take further “life-share” as a potential mistake.

    I do not think it is a coincidence that you had half of China’s pork population get decimated and then nCoV develops. I believe that the subsequent spike in duck farming may have been the intermediary host from bats as ducks are the only known species to jump coronavirus strands from delta -> gamma and now potentially beta. What that means is we have one bad ass mofo virus that isn’t done “growing-up”.

    Be safe.

    -JPM

    Comment by Jake — March 23, 2020 @ 8:37 pm

  18. @Jake: we’d like to do our bit to help our local Chinese take-away to survive. We’d also like to eat, seeing as we are told by Good King Boris to “shield” ourselves at home for three months.

    To the point: dare we order crispy aromatic duck?

    Comment by dearieme — March 24, 2020 @ 6:07 am

  19. Day One, infected Person Zero infects three others. Day Two, infected Persons 1, 2 and 3 each infect three others. Day Nineteen, 1.1 billion have been infected. The lockdown would have worked back around Day Seven. Now it’s utterly pointless.

    Comment by Michael van der Riet — March 24, 2020 @ 11:45 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress