Primed Numbers: exercises in policy-based evidence-making?
Last month, the Racing Post revealed that the Gambling Commission had withheld for more than three years, evidence of widespread consumer opposition to affordability checks. The story raised a number of questions – from legal experts and others – about how evidence is used to inform regulation. In particular, it prompted speculation about why the market regulator was able to claim a consumer mandate for the imposition of checks when the weight of evidence tilted so clearly in a different direction. In this article, we examine the Gambling Commission’s use of consumer research on affordability checks, how survey responses can be stimulated to achieve the desired effects and why we should be wary of policy-based evidence-making.
The evidence trail starts in 2019 when the Gambling Commission asked the research firm 2CV to investigate consumer attitudes towards the prevention of ‘binge gambling’ episodes. The results of this survey indicated considerable antipathy towards hard interventions with more than 70% of respondents rejecting operator-imposed controls (with one-quarter selecting no action whatsoever).
The Gambling Commission omitted these findings when it published the results of the 2CV study and they were also excluded from the 2020 call for evidence on affordability checks, despite clear relevance. The results of the Gambling Commission’s ‘mini-survey’ in 2021 were even more stark. While three-quarters of respondents (most of whom were online bettors) agreed that operators should be required to take action to support vulnerable customers, 78% rejected proposals for affordability checks, citing consumer privacy, consumer freedom and their likely ineffectiveness in preventing harm. Just 14% of respondents stated that they would comply with checks while two-thirds said that they would feel uncomfortable being subjected to assessments by Credit Reference Agencies (‘CRAs’).
As the Racing Post has reported, these results were kept under wraps for almost three-and-a-half years and were deliberately excluded from the 2023 consultation on Financial Risk Assessments (‘FRAs’). In October last year, the Commission refused a request under the Freedom of Information Act (‘FOIA’) for the survey findings to be released, claiming that it would be too time-consuming to do so and that it saw “no outstanding public interest” in making them while the consultation was open. In February this year, the chief executive of the Commission, Andrew Rhodes, gave a personal pledge that the results would be published. The Commission however, failed to publish the survey results when plans for FRAs were announced in May; and it took a further FOIA request to finally force its hand.
By this point however, the Gambling Commission had released the results from a third survey (a self-selected online panel managed by the polling firm, Yonder); and this time the results were rather different. A large majority (78%) of respondents agreed that checks were “necessary to protect people from gambling harm” and just 6% disagreed. It is true that the 2023 survey related to a modified form of the affordability checks proposed in 2021 – but this seems an unlikely explanation for such a change in views on checks and the use of CRAs in particular (see chart). Is it plausible that consumer attitudes should have altered so dramatically in just two years?
| Survey | Sample size |
| Gambling Commission 2021 | 12,124 |
| Yonder 2023 | 1,000 |
There are two strong contenders for explaining why the Yonder panel yielded such different views from the 2019 2CV survey and the Commission’s own ‘mini survey’ in 2021. The first relates to sample composition. The 2021 survey was responded to by people interested in the issue of affordability checks – people who had sufficient skin in the game to take the trouble to respond. The Yonder panel by contrast, consisted of people who get their kicks (as well as some small financial compensation) from taking part in surveys – but who otherwise might have little interest in the policy (only 14% had personal experience of affordability checks). It is easy to be supportive of controls imposed on others.
The bigger issue however, is likely to be one of response priming. Prior to completing the 2023 survey questionnaire, panellists were exposed to ‘stimulus packs’, including ‘video stimuli’ to “maximise engagement and efficiency”. They were informed that the checks being consulted on were already Government policy and that they would “protect the most vulnerable while allowing everyone else to enjoy gambling without harm”. They were then shown a selection of newspaper headlines, with the explicit intention that these should affect (or stimulate) responses. Some of these headlines reflected concerns about checks while others warned about delays to implementation; but the overall impression was far from balanced.
One of the headlines told respondents that “Gambling addicts will die because of delay to reforms, government warned”; another that “‘There will be more people dying’: mother whose daughter took own life criticises gambling white paper”. In other words, survey participants were encouraged to believe that people would take their own lives if the checks were opposed. They were also exposed to headlines expressing opposition to checks (e.g. “MP urges racing to make its case against ‘crippling’ Gambling Commission proposals”) but such concerns will pale when juxtaposed against self-harm. Viewed in this light, the Yonder panel’s strong support for FRAs appears distinctly unsurprising. The second of the two ‘suicide headlines’ in the stimulus pack did not even relate to affordability checks but instead to advertising; and in fact rejected the idea that “even stricter versions” of checks would prevent loss of life (suggesting that survey respondents were not simply led – but actively misled). A third headline related to a comment piece in The Times written by a director at the activist group, Gambling With Lives; while the author of the fourth is understood to have connections with the same organisation.
Reviewing the stimuli to which the Yonder panellists were subjected, it is difficult not to be cynical about the Gambling Commission’s intent. Having failed on two occasions to obtain the desired response, the Commission left nothing to chance with the Yonder survey. Even if we ignore these issues and take the 2023 Yonder survey at face value, it does not explain why the Commission repeatedly refused to publish the results of the 2021 ‘mini survey’ (or the 2019 2CV study). The Commission’s selective use of evidence is difficult to justify – and so far it has made absolutely no attempt to do so. Its silence in the face of serious concerns about consultative process appears evasive rather than dignified. If it has reservations about the 2019 and 2021 surveys, it should explain what these are and allow people to make up their own minds about their value. Suppressing inconvenient evidence serves only to undermine confidence in the regulatory process.
In around six weeks from now, a pilot scheme to assess the viability of Financial Risk Assessments will commence; yet the Gambling Commission has still not published details of how this will be evaluated. The criteria for success have not been released and it is unclear how the integrity of the tests will be assured. Having spent the last three-and-a-half years agitating for checks, the Commission cannot be considered a neutral actor; and its reputation for impartiality has taken a battering of late. In the absence of a proper explanation for the withholding of evidence and a commitment to transparent evaluation of the pilot, the market regulator faces the threat of fresh controversy – and possibly legal challenge. It is unlikely that the incoming Culture Secretary, Lisa Nandy, will thank the Gambling Commission for such an avoidable headache.
Geoff Banks Online Reports 