Evaluation Scotland Wales

evaluation

Improving numeracy to increase financial capability

Evidence type: Evaluation i

Description of the programme

National Numeracy, which seeks to improve maths skills in the UK, worked with the Money Advice Service as part of the ‘What Works’ initiative, to ascertain whether improvements in people’s numeracy enhances their attitudes and motivations in making financial decisions.

The target audience was working-age adults within and outside the workplace, and those at risk of falling into problem debt within the ‘struggling’ and ‘squeezed’ segments of society. Activity was split into two strands:

  • Virtual: adults were engaged directly via moneysavingexpert.com to improve their numeracy. Altogether 19,166 people registered on National Numeracy’s Challenge website as a result.
  • Delivery Partner: National Numeracy partnered with employment support organisation Prospects in the South West to engage long-term unemployed adults in improving their numeracy as part of face-to-face support to help them find employment. A variety of approaches was trialled, with various barriers being encountered, resulting in 498 unemployed adults being engaged in at least one activity.

The study

An outcome and process evaluation of the project activities was carried out, including analysis of qualitative feedback. The evaluation was carried out by National Numeracy, with support from an evaluation and learning partner appointed by the Money Advice Service.

Each online participant took part in a pre- and post-attitudinal evaluation test which gives credence to results showing an increase in numeracy and ability and improved attitudes towards finances.

For the delivery strand, National Numeracy and Prospects, their partner, were unable to gather enough outcome data by the end of the project to make a statistically valid evaluation of the outcome for these participants. Key findings were related to the process, the participants’ attitudes, and how to work with a delivery partner to effectively engage this group of adults.

Key findings

The key findings from outcome evaluation in the Virtual Strand were that:

  • People improved their Financial Capability Mindset as a result of engaging with National Numeracy’s provision.
  • The more that adults engaged with the numeracy provision, the more they improved their Financial Capability Mindset.
  • The higher an adult’s level of numeracy, the better their Financial Capability Mindset was likely to be.

Improvements in numeracy score correlated with improvements in Financial Capability Mindset.

The Virtual Strand also produced learning about the power of working with a well-known, far-reaching, trusted brand in engaging adults in improving their numeracy. This can work well if the activity is a visibly joined-up effort across multiple communication channels and is seen to be integral to the financial information offered.

The key findings from the Delivery Partner Strand were that:

  • A paper-based Quick Check tool is effective to triage numeracy levels and identify those who will benefit from improving their numeracy and financial capability.
  • There are a number of barriers to engaging the unemployed via a delivery partner, notably attitudes, and including adults who do not see the value of numeracy and who lack self-confidence.
  • The project showed that an unemployed adult who has been identified as being someone who would benefit from improving their numeracy, should first engage with attitudinal support to ensure that they begin to see the value of maths in daily life and their search for employment, and to increase self-confidence before they tackle skills training. This will improve the likelihood of them engaging and improving their skills.

Learning from the delivery strand should be treated with caution as not enough data was gathered to be statistically significant. The 184 pre-intervention and 60 post-intervention surveys indicated that participants’ financial capability mindset had got slightly worse. In the most striking change, the percentage of people who strongly agreed with the statement ‘I feel confident making decisions about financial products and services’ dropped from 25% to 10%. This could support a theory that less confident adults will initially lose confidence as they reflect on their skills, before their confidence grows in the longer term. If so, it would take a longer intervention and evaluation to assess this.

Points to consider

Methodological limitations:

  • There were no control or comparison groups, due to practical difficulties such as measuring skill levels.
  • In the Virtual Strand, only a post-intervention survey was used, as it was felt that a pre-intervention survey would have inhibited engagement rates.
  • In the Delivery Partner Strand, difficulties with getting unemployed adults to engage at all stages of the project prevented a statistically significant outcome evaluation being carried out. Learning from this strand came from process evaluation.

Relevance and transferability:

  • The project is relevant to others working in the same space but has limited value due to its partnership with Money Saving Expert, which would not be available to most organisations, and due to the lack of verified learning from the delivery strand due to lack of available data.

Full report

Improving numeracy to increase financial capability - full report

Key info

Activities and setting
The project sought to improve numeracy skills to enhance ability to make financial decisions. The study took place online and via a delivery partner.
Measured outcomes
Programme delivered by
National Numeracy
Year of publication
2018
Country/Countries
England
Contact information

Paul Milner, Paul@nationalnumeracy.org.uk