Motivated reasoning during a pandemic

Motivated reasoning is a familiar experimenter bias. An experimenter bias is any influence a researcher may have on the results of his or her research, derived from either interaction with participants or unintentional errors of observation, measurement, analysis, or interpretation (APA Dictionary of Psychology).

To put it simply, because you want something to be true, you unconsciously steer your enquiry in such a way as to reach just that conclusion. When this happens, conclusions or interpretations are shaped by their desirability instead of purely by the available evidence.

Unconscious bias is potent and can influence research at all stages: data gathering, selection of samples, classification, and interpretation of results. And unfortunately, motivated reasoning is ubiquitous in science.

An interesting possible case I came across today was a study about mortality in children and young people due to COVID-19.

How rarely do children die of COVID-19?

The paper ‘COVID-19 deaths in children and young people in England, March 2020 to December 2021’ looked at all 185 deaths in the under-twenties in England who had tested positive for SARS-CoV-2 in the 100 days before their death. Which of these people actually died of their infection with the virus?

The authors conclude that, reassuringly, the number is very low. And that’s of course the conclusion we all want to hear! Satisfying.

But should we believe it? The fact that the paper’s conclusion is clearly desirable should put us on high alert. Not only is it obviously desirable that few young people die of COVID-19, the conclusion also fits a narrative of SARS-CoV-2 as ultimately a mild virus we can live with without doing much about it. That narrative quite clearly has a strong magnetic effect on people. And scientists are people too.

Sabina Vohra-Miller points out that the authors quite conveniently removed some young people from their final dataset.

1- The ableism.
2- The authors of the study - without any data - removed >50% of deaths bc they *felt* wasn’t due to Covid-19 despite death certificate having that as cause.
3- THEN, they removed deaths due to MIS-C post Covid.
What a joke of a ‘study’.https://t.co/REVy2XiLSG

— Sabina Vohra-Miller (@SabiVM) November 19, 2022

Notably, the authors removed half of the deaths because they judged that COVID-19 wasn’t the cause of death, even though this was the cause specified on their death certificates. And they also, again conveniently, excluded deaths due to MIS-C (Multisystem inflammatory syndrome in children), even though this is a well-known consequence of an infection with SARS-COV-2 in children. These are the kinds of researcher interventions that are potentially influenced by unconscious biases.

Of course the authors have rationalised these decisions to themselves and to their reader. They are convinced they made the right call. But it’s hard to deny that the conditions here are ideal for experimenter bias: a desirable outcome combined with a moment of personal judgment by the researchers themselves.

Perhaps the problem arose already in the way the study was framed. The authors asked themselves how rare deaths of COVID-19 in children and young people are. They concluded that they were indeed very rare, even with the emergence of new SARS-CoV-2 variants. But it seems clear that with this research setup, the researchers had already primed themselves for the welcome conclusion.

Bias of societal proportions

As I said, motivated reasoning is ubiquitous, and you should always be very careful with scientific studies that reach a conclusion that is obviously desirable.

But I fear that the bias is common also in the non-scientific world. Because people want the pandemic to be over, they will interpret the situation around them in a way that satisfies this desire. Looking away at the right times, or describing what you see or hear in terms that are acceptable to you.

Here I have in mind the GP who will not test her patients for the virus, because community spread is allegedly low (it isn’t). Or the school teacher who attributes the increase in exhaustion or distractedness of his pupils to stress or mental health problems due to a situation at home. The employer who demands an explanation of a shrinking workforce without wanting to entertain that Long Covid could have something to do with it. Or people who have lingering symptoms of COVID-19 and who convince themselves it’s not that bad because they believe the virus is mild (again, it isn’t).

Through this type of cognitive bias, a society can quite successfully convince itself everything’s alright. Even when it isn’t.