Evidence of impact: An overview of financial education evaluations

• Background
• Methods
• Findings
• Points to consider

Background

The Financial Services Authority (FSA) commissioned this literature review to inform the National Strategy for Financial Capability, which aims to improve the financial capability of the UK population. The FSA designed the review to meet two objectives:

• To describe the extent to which the empirical literature identifies any incremental impact of financial capability interventions on people’s behaviour and attitudes, paying particular attention to the evidence about the effectiveness of different delivery methods of financial education and the impact on key target groups; and highlighting how various schemes are delivering different aspects of financial capability.
• To create a set of plausible estimates of the proportion of people likely to respond to types of financial capability initiatives by making changes in their behaviour, and the extent of any changes they make to their personal finances.

Methods

The FSA commissioned the Personal Finance Research Centre (PFRC) at the University of Bristol to undertake the review. PFRC conducted this review over a three-month period, and included relevant reports written in English made available before April 2008, which were mostly published within the previous ten years. PFRC accessed literature through electronic and online searches, utilising existing knowledge and networks. The literature reviewed consists almost entirely of primary reports of evaluations, rather than data drawn from other literature reviews and discussion papers. In total, PFRC considered evidence from over 70 studies.

PRFC structured the analysis and summarised the outcomes in relation to the aims of the National Strategy, focusing on the several key target groups: school children; young people who are Not in Education, Employment or Training (NEET); students in universities and Further Education (FE) colleges; employees in their workplace and new parents. Where PFRC identified several outcome measures, they took the most robust ones.

Findings

• There was limited evidence to form robust assumptions, or draw wider conclusions, about how a particular scheme might impact on a population of interest.
• There is little robust evidence about the impact of initiatives on particular aspects of financial capability, or the most appropriate delivery mechanisms for use with particular groups.
• The majority of evaluations that have identified measurable outcomes appear to have focused on savings behaviour, but it is not clear whether this is because it is the easiest behaviour to change, or the easiest to measure.
• Overall, the evidence relating to improvements in terms of money management and planning ahead is stronger than improvements in the other financial capability domains.
• The literature indicates that while it is widely believed that financial education in schools is valuable, the outcomes are likely to be long-term and will therefore be difficult to measure during an evaluation period.
• The current evidence suggests that only the very youngest pupils gain financial knowledge through school interventions. Rather, it appears that the main impact of school financial education is to improve behaviour in adulthood.
• Strategies aimed at adults also appear to have some long-term benefits. The research highlighted how workplace financial education seminars are associated with an increased likelihood of saving for retirement. Improved recruitment and retention appears to be an additional, and perhaps unexpected, advantage of workplace financial education, indicating that employees value such provision.
• There has been no systematic testing of the delivery methods used to increase financial capability, which is a serious oversight.

Points to consider

• Relevance:
  • As the study is from 2008, other studies may have superseded it.
• Generalisability/transferability:
  • The evaluations that PFRC summarised varied widely, in terms of the aims and objectives of the project under evaluation, and of the methods used to identify outcomes and measure impact. Therefore, the authors stress that caution is necessary in interpreting and generalising from the results.
  • The study’s second objective was to create a set of plausible estimates of the proportion of people likely to respond to types of financial capability initiatives by making changes in their behaviour, as well as the extent of any changes they make to their personal finances. However the report finds that there is insufficient robust evidence to make this possible.
• Methodological strengths:
  • Although not a systematic review, the study is a comprehensive evidence review that includes both published and unpublished research and covers evidence from 70 evaluations.
• Methodological limitations:
  • The report looks only at the evidence of outcomes from specific financial education initiatives rather than more general policies aimed at understanding or changing financial behaviour.