Tag: suicides

The Great Suicide Deception – Part IV – What purpose is served by spurious statistics?

Dan Waugh-Regulus Partners May 2024

This is the fourth and final article in our series on attempts by state bodies to claim widespread suicide mortality associated with problem gambling. In the first three articles we demonstrated why estimates prepared by Public Health England and the Office for Health Improvement and Disparities were irretrievably flawed; we examined the conduct of PHE and OHID, including evidence of bias and inappropriate behaviour; and we considered the role played by the Gambling Commission, the Advisory Board for Safer Gambling and others in either propagating the PHE-OHID claims or withholding concerns about their reliability. We conclude by addressing the wisdom of attempts to boil down a matter as complex as suicide to any single factor.

It has long been understood that people with gambling disorder are at elevated risk of death by suicide. The DSM-5 (the American Psychiatric Association’s ‘bible’) comments on elevated rates of suicide ideation and attempts among people in treatment for gambling disorder (and makes similar observations in relation to a large number of other mental health conditions, including alcohol use disorder). Concerns in relation to gambling disorder and self-harm – and what might be done to prevent suicide by people with the disorder – are entirely valid.

It is also widely accepted that suicidality is a complex matter. In their 2016 meta-analysis of 50 years of suicide research, Franklin et al. made the following observation: 

“…any individual with nearly any type of mental illness (i.e. internalizing, externalizing, psychotic, or personality disorder symptoms), serious or chronic physical illness, life stress (e.g. social, occupational, or legal problem), special population status (e.g. migrant, prisoner, nonheterosexual), or access to lethal means (e.g. firearms, drugs, high places) may be at risk for [suicidal behaviours and thoughts]. A large proportion of the population possess at least one of these risk factors at any given time, with many people possessing multiple factors.”   

Understanding that people with a gambling disorder are at elevated risk of suicide is helpful when it comes to devising self-harm prevention strategies. For example, Hakansson & Karlsson (the Swedish researchers relied upon by PHE-OHID) conclude their 2020 study with the following recommendation:

“The findings call for improved screening and treatment interventions for patients with gambling disorder and other mental health comorbidity.”

It is questionable however whether studies of discrete associations between any single activity or human characteristic and death by suicide should – by themselves – be used to justify state controls on that activity.  By way of illustration, a 2021 study on the prevalence of suicidal behaviour in a group of patients with behavioural addictions (Valenciano-Mendoza et al.) found: 

“the highest prevalence of suicide attempts was registered for sex addiction (9.1%), followed by buying–shopping disorder (7.6%), gambling disorder (6.7%), and gaming disorder (3.0%).”

These findings may be useful for addressing risk of self-harm within population groups suffering from these mental health conditions. They do not – by themselves – justify bans on sex, shopping or playing video games. A 2017 study of young adults in England (aged 20-24 years, n=106) by Appleby et al., found that four deaths by suicide were linked to ‘gambling problems’; and this has been used to suggest that 250 deaths by suicide each year are ‘gambling-related’. The study also found that 44 of those who had died “had a reported history of excessive alcohol use. Illicit drug use was reported in 54 (51%)”; seven “were reported as experiencing problems related to being a student” (including five experiencing “academic pressures”. One might therefore estimate (using the same methodology as for gambling problems) that around 3,200 suicides are related to illicit drug use; 2,625 to excessive alcohol use; and 440 to academia. Such findings should prompt concern and policy responses; but it is questionable whether these should extend – for example – to complete bans on advertisements for beer or universities.

Some activists have called for coroners to assess, as a matter of routine, the possible involvement of gambling in deaths under investigation – the Bishop of St Albans has doggedly pursued a Private Members Bill to mandate this. At first blush it seems to be a reasonable suggestion. The problem is that it places an additional requirement on already over-burdened coroners; and risks distortion if other known factors are not also investigated with the same degree of rigour. The presence of Adverse Childhood Experiences (‘ACEs’) is a well-documented antecedent of suicide with one study (Dube et al., 2001) finding that as many as 80% of suicide cases analysed had a history of ACEs. There are also well-documented associations between relationship breakdown and self-harm. The practicality and wisdom of asking coroners to probe into every corner of the deceased’s life should be carefully considered.

Those determined to produce figures on the prevalence of gambling-related suicide should first set out a clear operationalised definition of what this term means. How is the relationship to be characterised (e.g. does the individual need to have gambled in the prior 12 months? Does he or she need to have a diagnosis of gambling disorder?) and to what extent is there evidence of causal contribution to death (e.g. was gambling disorder a significant factor or a minor factor?). Finally, they should be required to contextualise their findings by reference to other risk factors.

Running through some of the institutional responses to PHE-OHID is the idea that unreliable estimates of mortality serve a valid purpose pending the production of more robust statistics – something along the lines of ‘fake it until you can make it’. The chair of the Gambling Commission’s Advisory Board for Safer Gambling (‘ABSG’), Dr Anna van der Gaag, for example has written that: 

“Good research, especially if it is on an under-researched area like this one, tends to begin and end in a different place, prompting challenge, replication, debate, and the research in this important area is no different.”

It is a view that overlooks four important points. First, the PHE-OHID work on the cost of gambling harms is riven with errors (including mathematical mistakes) and should not be considered “good research”. Second, the ABSG specifically called for “action” as a result of the PHE estimates – with no suggestion of the need for caution or refinement. Third, rather than welcoming challenge, the ABSG has engaged in ad hominem disparagement of those attempting to apply scrutiny to the PHE-OHID claims (likening this, without substantiation, to the activities of Big Oil). Fourth, it is questionable how far we should trust ‘better research’ if those responsible for it have propagated or tolerated misinformation in the past. As we saw during the Covid pandemic, the production of misleading statistics may in fact set back the cause of harm prevention by undermining trust in authority. 

Suicide risk among people with a gambling disorder is a legitimate issue and warrants an intelligent response; but this is unlikely to be achieved through the publication of spurious estimates of prevalence. As the US economist, Professor Douglas Walker has observed; 

“If researchers continue to offer social cost estimates, they should estimate costs that are measurable. But for other costs such as psychic costs that cannot be measured…let us identify them without providing spurious empirical estimates. Offering methodologically flawed cost estimates does not improve our understanding nor does it promote sound policy…In areas where research is still quite primitive, perhaps no data would be better than flawed data.”

Coda

We are aware that some individuals and organisations will resent this series of articles on PHE-OHID (not least the OHID researchers themselves). Our intention in writing them has not been to hurt or insult – but to shine a light on the way that some statistics are created and the distortive effect that ‘bad statistics’ can have on government policies. The application of scrutiny to research is an important part of the scientific process; and where state bodies are concerned, an important part of the democratic process too. It is entirely consistent to be concerned about a particular issue (e.g. risk of self-harm in a gambling context) and at the same time to believe that research into that issue should be conducted with honesty, openness and in accordance with scientific principles. In this way, we may hope to reduce the stigma associated with self-harm (such that gambling firms and other businesses gain the confidence to openly confront it); and that, over time, we may apply greater intelligence to the prevention of suicide in a gambling context and more generally. 

A Very Public Deception: On the manufacture of mortality statistics in gambling

Part II – Why did public health get things so badly wrong?

n the first in this series of articles, we examined the problems with claims made by state bodies – specifically Public Health England (‘PHE’) and the Office for Health Improvement and Disparities (‘OHID’) that up to 496 deaths by suicide each year in England are associated with ‘problem gambling’. We demonstrated that the basis for these claims is irretrievably flawed. Analysis of the Swedish dataset upon which they rely concluded that “gambling disorder did not appear to be a significant risk factor for the increase in suicide” (Karlsson, 2023). PHE and OHID researchers overlooked critical research findings and clear warnings about the advisability of their approach. While gambling disorder has long been recognised as a risk factor for self-harm, the estimates published by PHE-OHID are categorically unsound.

Read Part One: Lost in Translation?

In this second article in the series, we attempt to understand why PHE and the OHID persisted in following such a clearly problematic approach in the face of strong evidence of its unsuitability; we examine a number of issues of governance; and consider whether officials may have deliberately misled policy-makers and the public.

The Tobacco Road: why did PHE make such unsound claims?

In May 2018, at the conclusion of its review into gaming machines and social responsibility, the British Government’s Department for Culture, Media and Sport asked PHE to “conduct an evidence review of health aspects of gambling-related harm to inform action on prevention and treatment”.  More than three years later, in September 2021, PHE responded with the publication of five reports on the subject. One of these reports (‘The economic and social cost of harms’) claimed annual costs of £1.27bn a year associated with ‘problem gambling’ – with roughly 50% attributable to deaths by suicide.

It was this rather speculative document, rather than PHE’s more robust quantitative review of evidence from NHS Health Surveys, that officials chose to emphasise – prompting Britain’s Gambling Commission to surmise that PHE’s goal was, “to ensure gambling is considered as a public health issue.”

The Gambling Commission had already been given a glimpse of what “a public health issue” would entail. In a draft press release (seen by the Commission), PHE officials called for:

“a public health approach to gambling…similar to how we tackle tobacco consumption or unhealthy food consumption…”.

In the summer of 2022, the PHE researchers (now transferred to OHID) spelt out what this tobacco-style offensive would involve. Their paper, published in the Lancet Public Health, contained 81 measures for state intervention in the gambling market. The list included prohibitions on: all gambling advertising and marketing (including at racecourses); all in-play betting; and the sale of wine, beer and spirits in bingo clubs and casinos. It also included limits on the number of people permitted on a website at any one time, annual tax increases above the rate of inflation and even ‘plain packaging’ for all gambling products (no colours, logos or images permitted on playing cards, gaming machines, National Lottery tickets and so on).

There were other indications that PHE’s endeavours were not entirely objective – or morally neutral. In 2020, for example, its project leader stated that “more research is required to support advocacy and action” against gambling – hardly a statement of impartiality or scientific rigour. Meanwhile, documents made available under the Freedom of Information Act (‘FOIA’) reveal that PHE had agreed to be part of a research group set up by the activist charity, Gambling With Lives (‘GwL’) during the review period – an engagement it failed to disclose within its report.

Thank you for reading CIEO. This post is public so feel free to share it.

Share

Why did OHID publish its report…and did officials mislead?

In January 2023, the Department of Health and Social Care (‘DHSC’) withdrew the PHE report and published an updated set of cost estimates – this time in the range of £1.05bn to £1.77bn a year (underpinned by a choice of 117 or 496 deaths). OHID described the decision to review PHE’s work as “a standard approach for previously published reports ”; but this seems to be untrue. The decision to re-examine the PHE cost estimates alone (none of the other four reports was reviewed – despite the presence of errors) was taken in July 2022 and announced to Parliament shortly afterwards. We have found no evidence that reviewing state agency reports within ten months of publication is a “standard approach” or that any such policy exists.

Disclosures made under FOIA reveal the true reason for review. On 26th July 2022, an unnamed DHSC official circulated a memorandum, stating:

“We are going to need to make changes to two of the evidence review reports as an error has been spotted, and as it’s a change to results, its [sic.] probably what you would classify as a major change.”

Given that the PHE report contained quite a few errors, it is difficult to know which particular mistake prompted re-examination; but the decision was certainly not part of a “standard approach”. This raises the possibility that OHID may have deliberately misrepresented the grounds for review.

The Gambling Commission and the Advisory Board for Safer Gambling were both told by OHID researchers that “nothing in the report has changed substantially”; but this is incorrect. In fact, every single line item in the OHID cost estimate differed from the PHE version – in some cases substantially. Its estimate of direct costs to the Government was £234.1m lower than PHE’s – a reduction of more than one-third. This was masked by the introduction of a new area of intangible costs, relating to depression and several revisions to the suicide calculation. OHID’s estimates were also based on a ‘harmed population’ 59% smaller than in PHE. As chart 1 (below) shows, the claim that ‘nothing changed substantially’ appears misleading.

In August 2022, the then Health Minister, Maggie Throup MP advised Parliament that the PHE report would be reviewed and that the calculations underpinning its estimates would be published. The review however, has never been made public and – according to disclosures made under FOIA – no such document is held by the DHSC. Contrary to the minister’s pledge, the PHE calculations have still not been released. To do so would reveal a number of errors, such as the fact that PHE’s suicide figure was based on a 21% over-statement of the population prevalence of ‘problem gambling’.

The mystery of the OHID expert panel

OHID was at least prepared to admit – with a heavy dose of understatement – that its estimates were “uncertain”. It relied on a study of hospital patients in Sweden with a clinical diagnosis of gambling disorder (among many other health issues) to estimate the health risks for people in England with no diagnosed mental or physical health conditions whatsoever. In consequence, OHID leaned heavily on the opinion of its expert panel of health economists and academics who, it is claimed, approved the approach.

There are, however two problems where this opinion is concerned. The first is that one member of the expert panel, Dr Henrietta Bowden-Jones of the NHS had publicly criticised the PHE-OHID methodology. At a fringe meeting of the Conservative Party Conference in September 2022, Dr Bowden-Jones stated: “we cannot extrapolate from Swedish studies, from Norwegian studies – it doesn’t work”.

CIEO is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Subscribed

The second issue is that the meeting of the expert panel – to discuss the most significant matter in the OHID report – is entirely undocumented. In February 2023, the DHSC admitted that:

“there was no agenda or papers shared before the meeting or minutes circulated afterwards”.

It is difficult to understand how this panel of experts might have been expected to review OHID’s work without access to any documents; and why officials did not consider it necessary to record the panel’s deliberations on this critical point.

Why did public health get things so badly wrong?

Inappropriate behaviour?

The task attempted by PHE-OHID was always going to be challenging, given the dearth of actual data available. This does not explain or excuse the large number of errors and omissions made by researchers and officials:

  • PHE and OHID ignored warnings by Karlsson & Håkansson about the representativeness of the sample in the 2018 Swedish study (upon which they relied);
  • PHE and OHID ignored findings in the 2018 study of high rates of mental and physical health comorbidities.
  • PHE and OHID ignored the follow-up study by the Swedish researchers (Håkansson & Karlsson, 2020), which found that risk of suicide attempt was significantly mediated by the presence of other disorders.
  • PHE and OHID ignored the opinion of Dr Anna van der Gaag, chair of the Gambling Commission’s Advisory Board for Safer Gambling, that the PHE calculation was likely to be inaccurate.

A large number of issues with the PHE-OHID reports were brought to the attention of its Director-General, Jonathan Marron in July 2022 and again in September 2023. On both occasions, Mr Marron promised to investigate. Last year, he wrote that he would provide “a proper explanation” for the errors and methodological flaws; but more than seven months later, none has been forthcoming. In what may well be a breach of the Civil Service Code, OHID officials resorted to ad hominem disparagement of their critics – including one national news media outlet – rather than engage constructively.

What is particularly disturbing about the PHE-OHID scandal is not the fact that researchers (presented with an unenviable task) made so many mistakes; but that state officials proved so unwilling to confront them – responding with hostility to legitimate scrutiny.

Next week, in our third article, we will consider the behaviour of others in positions of political or moral authority who variously connived in the deception or turned a blind eye to it. We will reflect on what this means for their future involvement in research and policy-making.

Dan Waugh

May 17th 2024

Regulus Partners

A Very Public Deception: On the manufacture of mortality statistics in gamblingA study of suicides in gambling, are we being told the truth? Part 1

Public Health England was closed down because it was incompetent and was too easily distracted by lifestyle issues when it should have been focusing on public health. It was more of an in-house lobby group than a serious scientific agency. It seems that closing it down and re-opening it under a new name (OHID) with the same staff was not enough to make the leopard change its spots.

Dan Waugh- Regulus Partners

In recent years, the claim that up to 496 deaths a year in England are associated with problem gambling has become a staple of the debate on gambling market reform. The estimates originate from a 2023 report by the British Government’s Office for Health Improvement and Disparities (‘OHID’) and have been used to support demands for a wide range of additional controls on consumers and the market. There is just one problem – they are based on junk science.

While it has long been recognised that people with gambling disorder are at elevated risk of self-harm, the specific estimates produced by OHID – accepted uncritically by many in Parliament and the news media – rely on a number of ‘flat-Earth’ assumptions.

In this series of articles, we examine the methods used (and errors made) in calculating these figures and consider the conduct of those who have propagated them. In this, the first article, we demonstrate why the OHID estimates are unsound. In subsequent weeks we will describe the behaviour of the public health officials responsible for their manufacture; consider the actions of other notionally responsible bodies; and ask what public benefit is served by the generation of spurious statistics.

The first state-sponsored estimate of gambling-related suicides in Britain appeared in September 2021 with the release of Public Health England’s (‘PHE’) report, ‘Gambling-related harms evidence review: the economic and social cost of harms’. It contended that, in England, 409 suicides a year were “associated with problem gambling only”. In January 2023, the PHE report was replaced (due to identification of errors) by an update from OHID. It offered a choice of either 117 or 496 suicides “associated with problem gambling”.

Both the PHE and OHID estimates were based on a 2018 study of the medical records of patients treated in Swedish hospitals between 2006 and 2016. Dr Anna Karlsson and Professor Anders Håkansson from Lund University found that patients in the dataset with a clinical diagnosis of ICD-10 ‘pathological gambling’ (renamed gambling disorder in the ICD-11) were on average, 15.1 times more likely to die by suicide compared with the general population. PHE applied suicide mortality ratios from this study to NHS Health Survey estimates of the prevalence of PGSI ‘problem gambling’ in England to produce a figure of 409 deaths a year.

In 2023, OHID repeated the exercise, using precisely the same information, and produced figures of either 117 or 496 deaths (the lower figure based on the application of the Swedish mortality ratios to the population prevalence of DSM-IV ‘pathological gambling’). In doing so they ignored critical information and clear warnings that their methods were unsound. The hospital patients whose records were analysed in the ‘Swedish study’ suffered from a wide range of diagnosed mental and physical health conditions (see charts 1 and 2, below). As a group, they were at elevated risk of self-harm, regardless of the presence or absence of gambling disorder. PHE-OHID thought otherwise – assuming that  health risks for hospital patients in Sweden with a wide range of illnesses were the same as for people in England with no diagnosed health disorders whatsoever. In other words, they made the ‘flat-Earth’ assumption that there is no association between mental and physical ill-health and risk of suicide.

In making this assumption, PHE and OHID ignored a clear warning from Karlsson & Håkansson. Their paper advised that the hospital patients whose records they had studied were likely to suffer from particularly severe and complex disorders:

“It is therefore likely that results may be skewed toward a population of individuals with more severe forms of GD [gambling disorder]. It is likely that this once again implies that this study sample might contain patients with higher mental health comorbidity, as well as individuals with more severe forms of GD, since these individuals are more likely to receive specialized psychiatry care”.

The PHE-OHID researchers also ignored findings from the follow-up to this study (the second in a series of five undertaken by the researchers from Lund University). Håkansson & Karlsson (2020) showed that comorbid health conditions were even higher within the group of patients who had attempted or completed suicide (see chart 3).

Professor Håkansson and Dr Karlsson showed that risk of suicide attempt was five times higher for patients with gambling disorder if they also had diagnoses of alcohol use disorder and drug use disorder. Of those patients who had made a suicide attempt, 70% had a diagnosis of alcohol use disorder or drug use disorder or both. The researchers at Lund University provided a range of adjusted odds ratios based on the presence of other diagnosed mental health conditions (see table 1). This study – which was published ten months prior to the PHE report – indicated that suicide risk for patients with gambling disorder was halved where no alcohol use or drug use disorders were diagnosed. Even before adjusting for other risk factors, these findings clearly demonstrated the inappropriateness of PHE’s approach.

A third study assessed the effect of socioeconomic factors on risk of suicide attempt. In the fourth study, a control group was used to identify discrete risks associated with gambling disorder. It concluded that:

“gambling disorder did not appear to be a significant risk factor for the increase in suicide and general mortality when controlling for previously known risk factors”.

This finding creates a dilemma for OHID and those who have propagated its claims. If one believes that analysis of the Swedish National Patient register by Karlsson & Håkansson provides a reliable basis for assessing suicide risk in England, then one must conclude that – contrary to PHE-OHID assertions – gambling disorder is not “a significant risk factor”. If on the other hand, one does not believe this is a suitable approach, then the PHE-OHID claims also cannot stand because they rely entirely on the mortality ratios from the first of the Swedish studies.

The fact that PHE and OHID got things wrong does not mean that underlying concerns about gambling disorder and self-harm are misplaced – or that gambling operators, treatment providers and policy-makers should ignore the issue. It has long been recognised that people with the disorder are at elevated risk of suicide, even if the precise nature of the relationship is complex. A number of recent inquests in England have determined that excessive gambling contributed to loss of life. Operators should do more to promote positive mental health and to address risk of self-harm among their customers and employees – whether gambling is involved or not. The PHE-OHID claims are, however, irretrievably flawed and should be disregarded by policy-makers. There is simply no coherent logic that allows them to stand.

In next week’s article, we will consider why PHE-OHID produced such obviously flawed findings and examine potentially serious issues of governance attending their publication.

List of abbreviations

DSM-III: The third edition of the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Health Disorders.

DSM-IV: A screening questionnaire published by the American Psychiatric Association within the fourth edition of its Diagnostic and Statistical Manual of Mental Health Disorders

OHID: the Office for Health Improvement and Disparities. Part of the Department of Health and Social Care.

PGSI: The Problem Gambling Severity Index. A screening instrument developed by Ferris & Wynne (2001).

PHE: Public Health England. A state agency, reporting to the Department of Health and Social Care. It was disbanded in 2021.

Dan Waugh is a partner at the global strategic sports and leisure advisory firm, Regulus Partners.