The coronavirus pandemic has posed some serious challenges for scientists – inherent in the novelty of the infection itself (biology and epidemiology), in the need to explain difficult research to an apprehensive public and in the desire to inform governmental policy without unduly influencing decisions that are in the end political ones. There have been some great successes in rising to the challenge – for example in the development, evaluation and delivery of vaccines supported heavily by NIHR and the NHS and in the fields of infectious disease and genetic epidemiology.
More dismaying has been the way that differences of opinion have been handled. Elements of the press have of course behaved shamefully – what else do we expect – but it has been a real disappointment to see some members of the academic and clinical communities joining in what has been at times more like a childish shouty argument than a serious discussion about a difficult subject.
Against this background, my eye was caught by the release of a document from a group called the Health Advisory and Recovery Team (HART), COVID-19: an overview of the evidence. The trailers on their website were encouraging-
“HART is a group of highly qualified UK doctors, scientists, economists, psychologists and other academic experts…Our core aim is to find the common ground between the Government and groups that are concerned about COVID-19 restrictions. The ambition is to bring all sides together and to widen the debate in order to formulate an exit strategy that benefits everyone in society…Our research has identified a need for public policy to reflect a broader and more balanced approach across a number of key areas…We must now bridge the gaps in policy areas where there is obvious disagreement. We should try to avoid bias and follow the rapidly emerging evidence to guide us. If we listen and communicate clearly and respectfully with each other we can find the common ground”.
I restricted my reading to the area I know most about, the consequences of the pandemic and its management for the psychological wellbeing (mental health) of the population. That is to the chapters on COVID policies and harm to children (actually, including young adults), the psychological impact of the Govts communication style and restrictive measures, and the benefits and harms of wearing masks.
On a (very) superficial glance this looks like a scientific review of the evidence; lots of expert-sounding people have authored sections and there are nearly 300 references cited over 55 pages. A more detailed read gives quite a different picture. One notable section (suggesting that wearing a mask is psychologically harmful) isn’t referenced at all. Newspaper articles are a cited source of evidence and often do no more than recycle anecdotes, and it doesn’t help that there are website links that don’t work.
Where properly conducted research is cited there seems to have been no attempt at systematic identification or reviewing of its findin. Primary research studies are either selectively chosen or selectively reported. Two examples: First, The Prince’s Trust, with a comment on the “devastating impact” of the pandemic, noted that an online survey of theirs found a quarter of young people reporting that they felt unable to cope with what was happening in the pandemic. What was hardly mentioned by the Trust’s press releases (and not at all in the HART report) is that there have been only slight changes in recent years in levels of anxiety (53% in 2018, 55% in 2019, 54% in 2020, 56% in 2021) and depression (38% in 2018 and in 2019, 39% in 2020, 41% in 2021). The same applies to reports of increasing rates of self-harm and thoughts about self-harm: “research shows that suicidal thoughts have increased dramatically” refers to a single study reporting a rise from 8.2% to 9.8%, 12.5% to 14.4% in younger adults. Bizarrely the original study isn’t referenced, but instead a synopsis of it published in a different journal.
This isn’t to say that reports of not feeling able to cope, or of ending your life, are trivial. But there is considerable uncertainty about their interpretation, which doesn’t get an airing here. For example, it isn’t clear what the link is between mental health and feelings of not being able to cope if the one changes sharply but the other doesn’t. The meaning of “suicidal thoughts” can be difficult to interpret without a lot more detail than a questionnaire can provide, and the link between such thoughts and suicidal acts or suicide itself is not straightforward. One thing we do know is that a decline in young people’s mental health and increase in self-harm rates is a decade-long phenomenon, associated with social, educational and vocational inequalities and loss of community resources, not something that can be attributed to the pandemic or its management.
The authors describe this report as a rigorously and widely researched document and we have a pretty good idea what rigour in reviewing evidence should look like. After all, David Sackett and colleagues published Evidence based medicine: what it is and what it isn’t in the BMJ in 1996, the Cochrane Collaboration began its work in 1993, and the Journal of the American Medical Association launched Its series on step-by-step critical appraisal in 1993. Set against such guidance the HART overview doesn’t come anywhere close. The parts I have reviewed are more like personal statements of opinion dressed up as science, with conclusions expressed in over-certain and melodramatic ways – crisis, devastation and catastrophe are part of the vocabulary of journalism, not critical appraisal.
Perhaps all this doesn’t matter too much? The report was written to influence a decision about managing the pandemic in the UK and that decision has now been made.
So as a single incidence it will probably disappear from view, but more generally the bias and tone of pieces like this do need to be challenged. They are bad news for several reasons. Mixing polemics with science just strengthens the arm of critics who want to represent academics as opinion-mongers who dress up their own prejudices in pseudoscience. It oversimplifies the public policy dilemmas posed by the pandemic, which are more complicated than lockdown-don’t lockdown. But perhaps most importantly it muddies the waters so it is hard to see what the real problems are; anybody sensible thinks that the pandemic is likely to have mental health consequences and we need an adult discussion about what they are and what the inevitable trade-offs are when implementing a response to the risks posed by the virus. At the moment the HART Group really isn’t contributing helpfully to that discussion.