Tag: health

The Great Suicide Deception – Part IV – What purpose is served by spurious statistics?

Dan Waugh-Regulus Partners May 2024

This is the fourth and final article in our series on attempts by state bodies to claim widespread suicide mortality associated with problem gambling. In the first three articles we demonstrated why estimates prepared by Public Health England and the Office for Health Improvement and Disparities were irretrievably flawed; we examined the conduct of PHE and OHID, including evidence of bias and inappropriate behaviour; and we considered the role played by the Gambling Commission, the Advisory Board for Safer Gambling and others in either propagating the PHE-OHID claims or withholding concerns about their reliability. We conclude by addressing the wisdom of attempts to boil down a matter as complex as suicide to any single factor.

It has long been understood that people with gambling disorder are at elevated risk of death by suicide. The DSM-5 (the American Psychiatric Association’s ‘bible’) comments on elevated rates of suicide ideation and attempts among people in treatment for gambling disorder (and makes similar observations in relation to a large number of other mental health conditions, including alcohol use disorder). Concerns in relation to gambling disorder and self-harm – and what might be done to prevent suicide by people with the disorder – are entirely valid.

It is also widely accepted that suicidality is a complex matter. In their 2016 meta-analysis of 50 years of suicide research, Franklin et al. made the following observation: 

“…any individual with nearly any type of mental illness (i.e. internalizing, externalizing, psychotic, or personality disorder symptoms), serious or chronic physical illness, life stress (e.g. social, occupational, or legal problem), special population status (e.g. migrant, prisoner, nonheterosexual), or access to lethal means (e.g. firearms, drugs, high places) may be at risk for [suicidal behaviours and thoughts]. A large proportion of the population possess at least one of these risk factors at any given time, with many people possessing multiple factors.”   

Understanding that people with a gambling disorder are at elevated risk of suicide is helpful when it comes to devising self-harm prevention strategies. For example, Hakansson & Karlsson (the Swedish researchers relied upon by PHE-OHID) conclude their 2020 study with the following recommendation:

“The findings call for improved screening and treatment interventions for patients with gambling disorder and other mental health comorbidity.”

It is questionable however whether studies of discrete associations between any single activity or human characteristic and death by suicide should – by themselves – be used to justify state controls on that activity.  By way of illustration, a 2021 study on the prevalence of suicidal behaviour in a group of patients with behavioural addictions (Valenciano-Mendoza et al.) found: 

“the highest prevalence of suicide attempts was registered for sex addiction (9.1%), followed by buying–shopping disorder (7.6%), gambling disorder (6.7%), and gaming disorder (3.0%).”

These findings may be useful for addressing risk of self-harm within population groups suffering from these mental health conditions. They do not – by themselves – justify bans on sex, shopping or playing video games. A 2017 study of young adults in England (aged 20-24 years, n=106) by Appleby et al., found that four deaths by suicide were linked to ‘gambling problems’; and this has been used to suggest that 250 deaths by suicide each year are ‘gambling-related’. The study also found that 44 of those who had died “had a reported history of excessive alcohol use. Illicit drug use was reported in 54 (51%)”; seven “were reported as experiencing problems related to being a student” (including five experiencing “academic pressures”. One might therefore estimate (using the same methodology as for gambling problems) that around 3,200 suicides are related to illicit drug use; 2,625 to excessive alcohol use; and 440 to academia. Such findings should prompt concern and policy responses; but it is questionable whether these should extend – for example – to complete bans on advertisements for beer or universities.

Some activists have called for coroners to assess, as a matter of routine, the possible involvement of gambling in deaths under investigation – the Bishop of St Albans has doggedly pursued a Private Members Bill to mandate this. At first blush it seems to be a reasonable suggestion. The problem is that it places an additional requirement on already over-burdened coroners; and risks distortion if other known factors are not also investigated with the same degree of rigour. The presence of Adverse Childhood Experiences (‘ACEs’) is a well-documented antecedent of suicide with one study (Dube et al., 2001) finding that as many as 80% of suicide cases analysed had a history of ACEs. There are also well-documented associations between relationship breakdown and self-harm. The practicality and wisdom of asking coroners to probe into every corner of the deceased’s life should be carefully considered.

Those determined to produce figures on the prevalence of gambling-related suicide should first set out a clear operationalised definition of what this term means. How is the relationship to be characterised (e.g. does the individual need to have gambled in the prior 12 months? Does he or she need to have a diagnosis of gambling disorder?) and to what extent is there evidence of causal contribution to death (e.g. was gambling disorder a significant factor or a minor factor?). Finally, they should be required to contextualise their findings by reference to other risk factors.

Running through some of the institutional responses to PHE-OHID is the idea that unreliable estimates of mortality serve a valid purpose pending the production of more robust statistics – something along the lines of ‘fake it until you can make it’. The chair of the Gambling Commission’s Advisory Board for Safer Gambling (‘ABSG’), Dr Anna van der Gaag, for example has written that: 

“Good research, especially if it is on an under-researched area like this one, tends to begin and end in a different place, prompting challenge, replication, debate, and the research in this important area is no different.”

It is a view that overlooks four important points. First, the PHE-OHID work on the cost of gambling harms is riven with errors (including mathematical mistakes) and should not be considered “good research”. Second, the ABSG specifically called for “action” as a result of the PHE estimates – with no suggestion of the need for caution or refinement. Third, rather than welcoming challenge, the ABSG has engaged in ad hominem disparagement of those attempting to apply scrutiny to the PHE-OHID claims (likening this, without substantiation, to the activities of Big Oil). Fourth, it is questionable how far we should trust ‘better research’ if those responsible for it have propagated or tolerated misinformation in the past. As we saw during the Covid pandemic, the production of misleading statistics may in fact set back the cause of harm prevention by undermining trust in authority. 

Suicide risk among people with a gambling disorder is a legitimate issue and warrants an intelligent response; but this is unlikely to be achieved through the publication of spurious estimates of prevalence. As the US economist, Professor Douglas Walker has observed; 

“If researchers continue to offer social cost estimates, they should estimate costs that are measurable. But for other costs such as psychic costs that cannot be measured…let us identify them without providing spurious empirical estimates. Offering methodologically flawed cost estimates does not improve our understanding nor does it promote sound policy…In areas where research is still quite primitive, perhaps no data would be better than flawed data.”

Coda

We are aware that some individuals and organisations will resent this series of articles on PHE-OHID (not least the OHID researchers themselves). Our intention in writing them has not been to hurt or insult – but to shine a light on the way that some statistics are created and the distortive effect that ‘bad statistics’ can have on government policies. The application of scrutiny to research is an important part of the scientific process; and where state bodies are concerned, an important part of the democratic process too. It is entirely consistent to be concerned about a particular issue (e.g. risk of self-harm in a gambling context) and at the same time to believe that research into that issue should be conducted with honesty, openness and in accordance with scientific principles. In this way, we may hope to reduce the stigma associated with self-harm (such that gambling firms and other businesses gain the confidence to openly confront it); and that, over time, we may apply greater intelligence to the prevention of suicide in a gambling context and more generally. 

A Very Public Deception: On the manufacture of mortality statistics in gambling

Part II – Why did public health get things so badly wrong?

n the first in this series of articles, we examined the problems with claims made by state bodies – specifically Public Health England (‘PHE’) and the Office for Health Improvement and Disparities (‘OHID’) that up to 496 deaths by suicide each year in England are associated with ‘problem gambling’. We demonstrated that the basis for these claims is irretrievably flawed. Analysis of the Swedish dataset upon which they rely concluded that “gambling disorder did not appear to be a significant risk factor for the increase in suicide” (Karlsson, 2023). PHE and OHID researchers overlooked critical research findings and clear warnings about the advisability of their approach. While gambling disorder has long been recognised as a risk factor for self-harm, the estimates published by PHE-OHID are categorically unsound.

Read Part One: Lost in Translation?

In this second article in the series, we attempt to understand why PHE and the OHID persisted in following such a clearly problematic approach in the face of strong evidence of its unsuitability; we examine a number of issues of governance; and consider whether officials may have deliberately misled policy-makers and the public.

The Tobacco Road: why did PHE make such unsound claims?

In May 2018, at the conclusion of its review into gaming machines and social responsibility, the British Government’s Department for Culture, Media and Sport asked PHE to “conduct an evidence review of health aspects of gambling-related harm to inform action on prevention and treatment”.  More than three years later, in September 2021, PHE responded with the publication of five reports on the subject. One of these reports (‘The economic and social cost of harms’) claimed annual costs of £1.27bn a year associated with ‘problem gambling’ – with roughly 50% attributable to deaths by suicide.

It was this rather speculative document, rather than PHE’s more robust quantitative review of evidence from NHS Health Surveys, that officials chose to emphasise – prompting Britain’s Gambling Commission to surmise that PHE’s goal was, “to ensure gambling is considered as a public health issue.”

The Gambling Commission had already been given a glimpse of what “a public health issue” would entail. In a draft press release (seen by the Commission), PHE officials called for:

“a public health approach to gambling…similar to how we tackle tobacco consumption or unhealthy food consumption…”.

In the summer of 2022, the PHE researchers (now transferred to OHID) spelt out what this tobacco-style offensive would involve. Their paper, published in the Lancet Public Health, contained 81 measures for state intervention in the gambling market. The list included prohibitions on: all gambling advertising and marketing (including at racecourses); all in-play betting; and the sale of wine, beer and spirits in bingo clubs and casinos. It also included limits on the number of people permitted on a website at any one time, annual tax increases above the rate of inflation and even ‘plain packaging’ for all gambling products (no colours, logos or images permitted on playing cards, gaming machines, National Lottery tickets and so on).

There were other indications that PHE’s endeavours were not entirely objective – or morally neutral. In 2020, for example, its project leader stated that “more research is required to support advocacy and action” against gambling – hardly a statement of impartiality or scientific rigour. Meanwhile, documents made available under the Freedom of Information Act (‘FOIA’) reveal that PHE had agreed to be part of a research group set up by the activist charity, Gambling With Lives (‘GwL’) during the review period – an engagement it failed to disclose within its report.

Thank you for reading CIEO. This post is public so feel free to share it.

Share

Why did OHID publish its report…and did officials mislead?

In January 2023, the Department of Health and Social Care (‘DHSC’) withdrew the PHE report and published an updated set of cost estimates – this time in the range of £1.05bn to £1.77bn a year (underpinned by a choice of 117 or 496 deaths). OHID described the decision to review PHE’s work as “a standard approach for previously published reports ”; but this seems to be untrue. The decision to re-examine the PHE cost estimates alone (none of the other four reports was reviewed – despite the presence of errors) was taken in July 2022 and announced to Parliament shortly afterwards. We have found no evidence that reviewing state agency reports within ten months of publication is a “standard approach” or that any such policy exists.

Disclosures made under FOIA reveal the true reason for review. On 26th July 2022, an unnamed DHSC official circulated a memorandum, stating:

“We are going to need to make changes to two of the evidence review reports as an error has been spotted, and as it’s a change to results, its [sic.] probably what you would classify as a major change.”

Given that the PHE report contained quite a few errors, it is difficult to know which particular mistake prompted re-examination; but the decision was certainly not part of a “standard approach”. This raises the possibility that OHID may have deliberately misrepresented the grounds for review.

The Gambling Commission and the Advisory Board for Safer Gambling were both told by OHID researchers that “nothing in the report has changed substantially”; but this is incorrect. In fact, every single line item in the OHID cost estimate differed from the PHE version – in some cases substantially. Its estimate of direct costs to the Government was £234.1m lower than PHE’s – a reduction of more than one-third. This was masked by the introduction of a new area of intangible costs, relating to depression and several revisions to the suicide calculation. OHID’s estimates were also based on a ‘harmed population’ 59% smaller than in PHE. As chart 1 (below) shows, the claim that ‘nothing changed substantially’ appears misleading.

In August 2022, the then Health Minister, Maggie Throup MP advised Parliament that the PHE report would be reviewed and that the calculations underpinning its estimates would be published. The review however, has never been made public and – according to disclosures made under FOIA – no such document is held by the DHSC. Contrary to the minister’s pledge, the PHE calculations have still not been released. To do so would reveal a number of errors, such as the fact that PHE’s suicide figure was based on a 21% over-statement of the population prevalence of ‘problem gambling’.

The mystery of the OHID expert panel

OHID was at least prepared to admit – with a heavy dose of understatement – that its estimates were “uncertain”. It relied on a study of hospital patients in Sweden with a clinical diagnosis of gambling disorder (among many other health issues) to estimate the health risks for people in England with no diagnosed mental or physical health conditions whatsoever. In consequence, OHID leaned heavily on the opinion of its expert panel of health economists and academics who, it is claimed, approved the approach.

There are, however two problems where this opinion is concerned. The first is that one member of the expert panel, Dr Henrietta Bowden-Jones of the NHS had publicly criticised the PHE-OHID methodology. At a fringe meeting of the Conservative Party Conference in September 2022, Dr Bowden-Jones stated: “we cannot extrapolate from Swedish studies, from Norwegian studies – it doesn’t work”.

CIEO is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Subscribed

The second issue is that the meeting of the expert panel – to discuss the most significant matter in the OHID report – is entirely undocumented. In February 2023, the DHSC admitted that:

“there was no agenda or papers shared before the meeting or minutes circulated afterwards”.

It is difficult to understand how this panel of experts might have been expected to review OHID’s work without access to any documents; and why officials did not consider it necessary to record the panel’s deliberations on this critical point.

Why did public health get things so badly wrong?

Inappropriate behaviour?

The task attempted by PHE-OHID was always going to be challenging, given the dearth of actual data available. This does not explain or excuse the large number of errors and omissions made by researchers and officials:

  • PHE and OHID ignored warnings by Karlsson & Håkansson about the representativeness of the sample in the 2018 Swedish study (upon which they relied);
  • PHE and OHID ignored findings in the 2018 study of high rates of mental and physical health comorbidities.
  • PHE and OHID ignored the follow-up study by the Swedish researchers (Håkansson & Karlsson, 2020), which found that risk of suicide attempt was significantly mediated by the presence of other disorders.
  • PHE and OHID ignored the opinion of Dr Anna van der Gaag, chair of the Gambling Commission’s Advisory Board for Safer Gambling, that the PHE calculation was likely to be inaccurate.

A large number of issues with the PHE-OHID reports were brought to the attention of its Director-General, Jonathan Marron in July 2022 and again in September 2023. On both occasions, Mr Marron promised to investigate. Last year, he wrote that he would provide “a proper explanation” for the errors and methodological flaws; but more than seven months later, none has been forthcoming. In what may well be a breach of the Civil Service Code, OHID officials resorted to ad hominem disparagement of their critics – including one national news media outlet – rather than engage constructively.

What is particularly disturbing about the PHE-OHID scandal is not the fact that researchers (presented with an unenviable task) made so many mistakes; but that state officials proved so unwilling to confront them – responding with hostility to legitimate scrutiny.

Next week, in our third article, we will consider the behaviour of others in positions of political or moral authority who variously connived in the deception or turned a blind eye to it. We will reflect on what this means for their future involvement in research and policy-making.

Dan Waugh

May 17th 2024

Regulus Partners