The early responses of European countries to COVID-19 show why calls for “evidence-based decisions” are essentially meaningless.
“Evidence-based policy” has become a mantra, a condition of rational action, a bright line that separates decisions that have science and facts on their side from those that are merely political, emotional, or irrational in motivation. The COVID-19 pandemic is proving no exception, as claims that policy decisions must be based on science and evidence are rampant. On one level such claims are entirely obvious; of course we want evidence to guide our response.
Accordingly, governments make reference to scientific evidence (as well as expert judgment) to justify their actions (or, more often, inaction) and to explain their changes in policy as responding to changes in the available evidence. But on another level, the claim of evidence-based policy is a shield behind which politics are hidden and responsibility is evaded.
To understand the role of “evidence” in policy-making, especially during times of crisis, it is worth first considering the concept of evidence itself. The ideal of evidence-based policy can be traced back to the technocracy movement of the beginning of the twentieth century, but it has been most fully articulated and implemented through the aspirations of evidence-based medicine.
Evidence-based medicine depends on the recognition of hierarchies of scientific evidence in medical practice, with randomized controlled trials widely considered to be the “gold standard” of evidence in support of clinical decisions as diverse as prescribing drugs, choosing surgical procedures, and recommending diets or various therapies.
Even in medicine, however, this ideal is inherently problematic. A doctor has access to the results of randomized, controlled trials about a drug or procedure, but must still make judgments in light of considerations that include the uncertainties revealed by the trials, the particulars of any given patient (how, for example, might the drug interact with other treatments the patient is receiving?), and even the cost of the treatment.
TWO TYPES OF EVIDENCE
What type and amount of information are sufficient evidence for taking action? This is not something that comes prepackaged or predetermined. What is considered evidence, and how evidence is considered, depends on many factors. One way to think about evidence is to divide it into two general groups, which we will call “off-the-shelf” and “pragmatic.” Off-the-shelf evidence may come from formal scientific studies whose results were generated in the context of an experiment such as a randomized, controlled trial. In the case of COVID-19, it mostly comes as observational data (about rates of infection, number of deaths, and so on) whose implications must be extrapolated from one context to another. We may somewhat simplistically understand such evidence as prepackaged facts, available for deployment on behalf of decisions.