Plain Numbers Project: Initial trials report

• Context
• The study
• Key findings
• Points to consider

Context

Poor numeracy affects approximately half of working-age adults (BIS, 2012), and this report cites evidence of increasing Consumer Vulnerability in the UK (defined by the FCA as someone who, due to their personal circumstances, is especially susceptible to harm, particularly when a firm doesn’t act with appropriate levels of care).

Plain Numbers, a social enterprise working on this project with The Bank of England, aims to enable more customers to make informed choices by making ‘seemingly small changes to the way that numbers and data are presented to generate substantial increases in comprehension’.

Noting the negative impact of poor numeracy on customers’ ability to understand the numerical concepts underpinning products or services, the study was undertaken to demonstrate what change is possible, and to encourage others to embed the Plain Numbers Approach.

The study

The research questions were:

The objective of the study was to test the effectiveness of the Plain Numbers Approach on customer comprehension across a range of markets and documents that had different strategic intents.

Five randomised controlled trials were conducted by Kantar Public, with ClearScore, Atlanta, Direct Line, Octopus Energy and Thames Water. The trials compared an existing document provided by each of the five partners (the control) with the Plain Numbers version (the intervention) that sought to communicate the same information in a clearer and fairer way.

Kantar Public recruited around 1,000 participants who were representative of the UK population for each trial, and randomly allocated them to see either the control, or the Plain Numbers intervention. The performance of each document was measured using a number of outcomes, to understand changes in participants’ comprehension, confidence in their own understanding, and likelihood to contact the company concerned.

Key findings

• When using the Plain Numbers Approach, the number of people who correctly answered at least four out of five comprehension questions doubled, on average across the five trials. The least impactful trial saw comprehension increase by a quarter and the most by over two and a half times.
• Using the existing documentation, the evaluators noted a difference between perceived and actual understanding. While at least six out of ten participants agreed that the document was ‘presented in a clear and fair way and I understand it’, fewer than four out of ten participants were able to demonstrate a reasonable level of understanding.
• Whereas applying the Plain Numbers Approach appears to close the gap between perceived and actual understanding: again, at least six out of ten participants agreed that the document was presented clearly and fairly, but this time between three and six out of ten (across the five trials) were able to demonstrate that understanding. This suggests the Plain Numbers Approach creates the conditions for genuine informed choice for vulnerable customers who struggle with numbers.
• The study notes that the results of the trials are not necessarily replicable in other contexts, as the treatment that generated the improved comprehension was very different in each trial.
• There were also some indications that those with the most challenging financial circumstances benefitted the most from the Plain Numbers Approach.

Points to consider

• Methodological strengths/weaknesses: The RCT methodology and analysis appear sound and the findings are tested for statistical significance, so we can have a good degree of confidence in the robustness of the findings.
• Generalisability/ transferability: This report is of significance to anyone with an interest in the financial literacy of the UK population. The study notes that the findings of the five trials are not necessarily replicable in different contexts, and further testing would be needed to test effectiveness in those situations.
• Relevance: The proportions of people experiencing low levels of numeracy and financial literacy are of significant concern to the financial sector and policy makers, making this practical report of particular relevance.