Independent evaluation of the Community Links’ What Works project

• Description of the programme
• The study
• Key findings
• Points to consider
• Full report

Description of the programme

The Community Links (CL) ‘What Works’ project was a pilot project to evaluate the effectiveness of embedding financial capability support into existing services. The project was set up to test the hypothesis that financial capability training is more effective if it is underpinned by an on-going relationship, if there is an element of co-production and if it is delivered as part of a holistic approach.

CL services targeted a variety of groups with different demographics, with the aim of revealing useful attitudinal and other additional information. CL delivered training to ten frontline staff and external delivery partners who in turn delivered 12 financial capability training sessions to 87 participants. The project was spread across five existing internal programmes and four external programmes in Newham and surrounding London boroughs.

The target groups for CL services included east and north London residents, who were: 16 to 24 year olds not in employment, education or training (NEET); unemployed and/or low income working age adults on, or about to go on, Universal Credit; renting in the private or social sector and/or Black and Minority Ethnic (BAME).

The study

CL commissioned Mark Elliot and Sarah Thelwall to undertake the evaluation. The evaluation comprised of a process, outcome and economic evaluation. The evaluators employed both quantitative and qualitative methods, including pre- and post-intervention surveys with participants, focus groups with participants, focus groups and qualitative interviews with key training staff, and written reflections from frontline staff. The surveys and focus groups with participants were conducted either following the training, or one week later.

The study aimed to investigate whether embedding financial capability into existing, successful services produced positive financial capability outcomes for working age people and NEET young people, as well as which model is best for integrating financial capability into existing services.

Key findings

Embedding financial capability into existing, successful services produced positive financial capability outcomes immediately following the training and in the medium-term.

• The data suggests that the training increased the majority of participants’ willingness to seek, engage with and act on financial advice and guidance. The proportion of people knowing where to seek financial advice rose from 42% to 62%. This effect remained after three months.
• For some measures, such as ‘preparing for and managing life events’ the ‘very positive’ effect of the financial capability intervention that participants initially experienced had diminished after three months, and had returned to pre-intervention levels.
• For some indicators, such as feeling that they have improved confidence and skills in ‘managing their current financial situation’and‘dealing with debt’, the immediate positive effect remained with 62% of participants, at least in the medium-term follow-up.
• Some participants started saving to prepare for future financial shocks, and this remained higher than the pre-intervention proportion after three months. On the other hand, however, more participants reported not thinking about the future after three months, than in the pre-intervention survey.

The process evaluation generated some insights into what worked well with the intervention:

• Trainers’ familiarity with the material and personal use of the tools and techniques played a large part in the success of the delivery. Clients responded well to being involved in topic choice, and also to the personal experience of the trusted course leaders.
• Programme staff developed a library of modules for course leaders, who defined the delivery schedule for their programmes. The length of the sessions depended on the programme delivery schedule, client base and confidence of the frontline staff member delivering the training.
• Participants were more likely to attend the financial capability training if it was part of a wider training programme. Participants in more individualised programmes, such as Talent Match, saw the training as optional and were much less likely to attend.

The economic evaluation found:

• The project included a number of costs, which would not normally be associated with delivering such a project (such as additional set-up costs, or expert support for frontline staff).
• If repeated, the intervention could reduce costs by retaining a project co-ordinator role to implement the training, increase the number of days for a consultant to provide training to practitioners, and increase staff time to allow them to embed the training into delivery.

Points to consider

Methodological limitations:

• Fewer participant sessions took place as a result of organisational re-design. Therefore, the evaluation was completed with 87 participants, instead of the 277 participants CL aimed for.
• The limited number of participants was in part due to the fact that that Talent Match participants did not attend the financial capability training as it was optional. To build up a reasonably sized evidence base, CL ran additional sessions with other internal projects and with external organisations, across a wide geographic and demographic base. This affected the evaluators’ ability to analyse the data in terms of gender, age and income.
• CL was not able to run a control group for this project, thus it is difficult to attribute causality to the intervention.
• The quality of the frontline staff training was not assessed.

Full report

Independent evaluation of the Community Links’ What Works project - full report