Journal of the Plague Year III – Of models and black boxes

The Imperial College pandemic model was never peer-reviewed – until now. There are parallels between epidemiological models and catastrophic hedge fund “black box” investment strategies. Victor Hill has been scribbling in his Plague Year journal again.

All about “R”

The fate of nations is now determined by an obscure variable which only the medical cognoscenti had heard of three months ago. I mean the R-value (or R-zero) of the novel coronavirus that causes the malady that we call Covid-19. This is the reproduction rate of a pathogen; or, if you prefer, a measure of how contagious it is. If a pathogen has an R-value of four (which has been attributed to the novel coronavirus if left unchecked) then everyone who catches it will spread it to four others within x days and the resultant geometric sequence of contamination would engulf much of the population in short order.

It is now accepted wisdom in both liberal western and authoritarian societies that the task of governments is to drive this R-value below one by restraining normal social inter-action. Once well below one, the virus will only perpetuate itself in ever decreasing numbers of cases.

The problem is that policy driven by the observation of an obscure conjectured variable is not as scientific as it might first seem. For a start, the R-value is not a metric that can be measured in real-time in the way that we can measure the temperature of a patient using a thermometer. In fact, we can only approximate “R” ex-post facto (i.e. with a lead time of at least three weeks, and probably longer) based on the number of new cases being recorded (which is not the same as the actual number of new infections out there, which only God knows – and he’s not telling).

Second problem: at what level of societal organisation does one approximate the R-value? In the prevailing climate it is assumed that it is scientifically valid to compare R-values between nations. So the UK approximates its R-value at 0.9 (or whatever) and France approximates its own at 0.6 (or whatever). Ergo, France is ahead of the curve relative to the UK. Or, Ms Sturgeon asserts that Scotland has a higher R-value than England and that therefore (logicians will spot the inappropriateness of that conjunction) English people should be vigorously discouraged from crossing the Esk or the Tweed into Bonnie Scotland.

One only has to think about this for a few moments to realise that all this is highly questionable. By definition, the disease is transmitted by people who have contracted it. All those who have escaped contamination have an R-value of zero. Those who unfortunately have contracted it have an R-value which is a function of the inherent transmissibility of the disease and of how much they put themselves about in society. An infected nurse in a hospital is likely to transmit the virus to many more people than an asymptomatic child. So the R-value is widely heterogeneous within any given community.

People in rural areas who live in houses with gardens probably have a much lower R-value than people who live in flats in densely populated cities like London or Birmingham. An analysis by St. Andrew’s University concluded that prolonged and close contact is necessary in order to transmit the virus and the risk is highest in enclosed environments.

And yes, of course, there is a socio-economic and even an ethnic dimension to all this (ethnic minorities tend to live in multi-person households). But, surely, we knew that anyway from the longevity statistics. The rich live longer than the poor, and have done so throughout human history – though there is no conspiracy theory implied in that proposition.

Clearly the R-values in hospitals and care homes – and airports, by the way – is much higher than in the community as a whole. But the idea that one can aggregate transmission rates on a national level is crude. It would be like arguing that you could compute the national IQ by averaging out the IQ of each and every citizen – and then arguing that my nation is more intelligent than your nation.

The importance of being Neil

As we know, the Johnson government was converted to the prevailing lockdown orthodoxy by the output of a computer model developed by Imperial College London signed off by Professor Neil Ferguson around 16 March. The computer model on which Imperial College relied – which estimated 500,000 fatalities if the virus were just allowed to take its course without mitigating policies – has never been fully put into the public domain. But some of the code from the model was leaked last week to a programmers’ website called GitHub – though it should be noted that this was a refined programme, and not the original model. And I’ve just read an analysis of that code by a computer scientist (and apparently, an ex-Google employee). It makes for sobering reading.

Sue Denim (a pseudonym, methinks) observes that Imperial’s revised codebase is split into multiple files for legibility and written in C++; whereas the original 13-year old programme was “a single 15,000-line file that had been worked on for a decade”. This is considered extremely poor practice by computer scientists. Sue writes:

Due to bugs, the code can produce very different results given identical inputs…This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all…

Another computer scientist has suggested that the model produces different outputs depending on what kind of computer it’s run on because there was no regression testing – a process whereby software engineers test the quality of their coding by running multiple inputs. Sue concludes:

On a personal level, I’d go further and suggest that all academic epidemiology be defunded. This sort of work is best done by the insurance sector. Insurers employ modellers and data scientists, but also employ managers whose job is to decide whether a model is accurate enough for real world usage…Academic efforts don’t have these people, and the results speak for themselves. If anyone in SAGE or the Government had shown the code to a working software engineer…alarm bells would have been rung immediately. Instead, the Government is dominated by academics who apparently felt unable to question anything done by a fellow professor.

It has become normal amongst epidemiologists, along with economists and climate scientists, to refer to the output of mathematical models as evidence of a particular theory. This is a classic category error. Models are developed in order to test academic theories against known facts which are objectively observed – their output can never be described as factual. And yet, depressingly, Prime Minister Johnson referred to the Imperial College model twice in his most recent address to the nation last Sunday evening (10 May).

Professor Ferguson’s modelling led to the cull of six million animals during the foot-and-mouth epidemic of 2001 – with British Army snipers shooting cows as they grazed in fields. In the early 2000s Professor Ferguson predicted up to 136,000 deaths from Mad Cow Disease; later he predicted 200 million deaths form bird flu and 65,000 from swine flu. Yet the final death toll from all of these was in the hundreds. This alone should have given the government pause for thought.

SAGE (the Scientific Action Group for Emergencies) – the secret conclave which formally adopted the Imperial College findings – meets in private, publishes no minutes and until pressured to do so did not even release the names of its members. Labour went into convulsions when it emerged that the Prime Ministers’ Chief Advisor, Mr Cummings, had attended one of their meetings – apparently because he is not a scientist. (Even though he predicted a future pandemic in May last year.)

But the real scandal is that the worst economic downturn for 300 years was precipitated, not by a pandemic, but by politicians being panicked into shutting down large parts of the economy on the basis of weak science and elitist groupthink. Of course, social distancing is to be encouraged – that is common sense. But shutting down schools when it’s not even clear if children can transmit the virus (apparently the young have ACE2 receptors which block it)? And persecuting sunbathers?

Professor Ferguson resigned from SAGE after somebody alerted the Daily Telegraph about his crepuscular transgressions. Let’s just say for now that there is more to that than meets the eye.

Routes of transmission

Why are we in the UK doing so badly in the so-called war against Covid-19?

As I write (Thursday lunchtime) the UK has had nearly 230,000 cases of coronavirus (the fourth highest in the world) and 33,186 fatalities (the second highest). Morbidity rates should best be compared by reference to deaths per million. On that basis the UK ranks well below Belgium (768) and others at 489. But that is much worse than Germany’s 94 deaths per million.

No doubt analysts will pore over the data for years to come in search of answers to this question. The problem right now is that we understand much less about the virus than the mainstream media admits. We still don’t even know if most people contract the virus through breath (breathing in droplets containing the virus) or through touch (transferring droplets form their hands to their mouths or eyes).

But one thing seems clear. Countries which did a lot of testing in the very early stages of the pandemic (late January/ early February) have lower mortality rates. Crucially, this is also true of regions within countries. In Groningen in the north of the Netherlands the overall death rate has increased by just four percent; whereas in Brabant in the south of the country (neighbouring Belgium) it has doubled[i].

Matt Ridley believes that Britain’s failure to increase testing in mid-March and to limit testing to people already in hospital was a much bigger mistake than to delay the lockdown. By testing more people in the early stages countries like South Korea and Germany managed to stop the virus from spreading within the healthcare system. In contrast, it now seems that many of the early cases in the UK spread within hospitals and doctors’ surgeries. Many healthcare workers – not a criticism of our hard-pressed nurses – then spread it to elderly patients who were in hospital for other reasons who were then sent back to their residential care homes and infected their peers. Let’s recall that the policy was to empty hospitals of malingerers so as to free up capacity for the expected influx of Covid-19 sufferers.

But it may be that in protecting the NHS the British state ended up creating a localised epidemic amongst the elderly in care homes which now account for 40 percent of all Covid-19 deaths in the UK. Indeed, this week the Care Quality Commission (CQC) claimed that patients were sent from hospitals to care homes when they were known to be infected with the virus[ii]. So the venerated NHS was actually responsible for the grim care home figures.

If Covid-19 is largely a hospital-borne infection then there is some hope that the epidemic may burn out more quickly than we feared. Anecdotally, most people are doing their best to steer clear of hospitals right now – even if they have chest pains (this is a rotten moment to have a heart attack). That might be unwise – but it is most certainly not irrational.

The other major transmission route is, of course, air travel. The virus spread from a Chinese provincial capital to every corner of the globe within three months. In contrast, it took the Black Death several years to travel from Asia to Europe. The reason Covid-19 spread so quickly is: modern air travel. In 2019 an average of 14,000 people travelled from China to the USA every single day. The virus was undoubtedly brought to the UK on an aeroplane. But, while President Trump banned flights from China from landing in the USA on 02 February (with some exceptions), the UK authorities have never made any attempt to close the country’s air borders. They have not even tested or temperature-checked new arrivals.

Only this week has the UK Government proposed that, starting from next month, new arrivals in the UK will have to endure two weeks of quarantine. This inevitably opened them to the accusation of shutting the stable door after the horse has bolted. It is going to be vigorously resisted by the airline and hospitality industries. What would make more sense would be to test everybody boarding a flight to the UK and to prevent those who are infected boarding the aeroplane at all. Passengers on Emirates are already being tested at Dubai airport and being given the result within ten minutes.

When Genius Failed

When Genius Failed (2002) is the title of a fascinating book by Roger Lowenstein. It concerns the collapse of the hedge fund Long Term Capital Management (LTCM) further to the Asian, and then the Russian, financial crises. The fund had been founded by legendary trader John W Meriwether who had made a fortune while at Salomon Brothers in the 1980s and early ‘90s. He put together a team of the finest financial minds on the planet including Nobel laureates Myron S Scholes and Robert C Merton, the originators of the Black-Scholes-Merton options pricing model. Together, they developed a model – or rather a black box since its workings were secret – which predicted bond price movements based on yield differentials or spreads.

For four years the fund made money hand over fist – but then came the Asian Financial Crisis (which kicked off in July 1997) followed the next year by the Russian Financial Crisis (August 1998). The fund collapsed. (Technically, it was bailed out by the Federal Reserve Bank of New York and a consortium of financial institutions in September 1998.)

Essentially, the flight to quality which ensued both crises caused bond spreads to widen to unprecedented levels (that is, statistically improbable levels of 20 or more standard deviations). The model couldn’t cope. It blew up. A lot of people lost a lot of money.

Even when models work they only tend to work for a short time – because the underlying variables keep changing in the real world. That applies as much in epidemiology as in investment. Caveat emptor.

***

By rights I should be in the Languedoc right now, sipping chilled rosé on a sunlit terrace overlooking the pool. But here I remain in chilly Norfolk, watching pigeons cavort around the pond from my study window. And I was planning a trip to America in late summer – to visit friends and write about the presidential election. I can’t really see that happening now.

To be honest, the thought of negotiating an airport departure lounge – even if the lockdowns are relaxed entirely – mingling with the nervous multitude wearing masks and latex gloves – fills me with dread.

I have even asked myself if I shall ever fly again. I suppose, when the Saint Greta becomes World Leader, we shall all be allocated a lifetime allowance of air miles – and I shall be told in no uncertain terms that I have already exhausted mine.

[i] See: https://www.spectator.co.uk/article/we-know-everything-and-nothing-about-covid

[ii] See: https://www.dailymail.co.uk/news/article-8302039/Hospitals-probed-sending-elderly-care-homes-despite-KNOWING-coronavirus.html