Evaluation Scotland Wales

evaluation

What Works for You project evaluation

Evidence type: Evaluation i

Description of the programme

Community Housing Aid (CHA) piloted the What Works for You (WW4U) project, which aimed to provide support to two client groups with an identified need for support with financial capability; young people aged 16-25 and working aged adults recently released from prison. This pilot project took place over a 12-month period finishing in 2018, supporting 164 clients residing in and around Exeter. Client specific sessions took place twice a week for five hours, with each client supported over a five-week period in a community hub. The holistic support consisted of an initial assessment, one-to-one case work and assessment, guided learning through a wide range of resources and materials, peer support and group work.

The study

CHA commissioned SERIO, at the University of Plymouth, to undertake the evaluation. The evaluation examined the extent to which CHA’s ‘holistic wellbeing’ approach in its hub impacted on the financial capability of 16-25 year olds and ex-offenders referred to the project because they face homelessness.

SERIO, at the University of Plymouth, designed an evaluation that focused on the impact of the project using a pre/post design. The methodology had several interrelated strands, which were:

  • Monitoring data - a selection of survey questions from the MAS Financial Capability Outcomes Framework administered to participants at the baseline and at the end of the five-week project.
  • Qualitative interviews – 18 interviews conducted across a sample of two client groups (young people and ex-offenders) both prior to and following the project.
  • Observations – informal observations at three intervention sessions.
  • Case studies – four case studies of individual participants to chart their progression and highlight their individual successes and challenges.
  • Stakeholder interviews – interviews with four key stakeholders about delivering the project and its impact.

Key findings

Process Evaluation:

  • The evaluators identified that delivering a project with typically ‘hard to engage’ groups was challenging, particularly in terms of recruiting young people into the project. CHA addressed this by adapting the project delivery and introducing community outreach and group work elements.

Outcome Evaluation:

Note: the findings are limited by only have 40 follow-up survey responses. Percentages are therefore not reported here.

  • At follow-up, both young people and ex-offenders reported an increase in knowledge of how to access financial advice. Ex-offenders discussed how their understanding of financial services had improved following the WW4U programme.
  • Clients made improvements towards planning and sticking to a budget. At the baseline interviews, many ex-offenders expressed their inability to budget due to limited funds. Despite limited funds at follow-up, ex-offenders demonstrated confidence and skills in utilising budgeting strategies.
  • The evaluation showed greater improvements for ex-offenders compared to young people for measures taken to maximise income. The number of ex-offenders checking whether they were receiving all benefits, tax credit or pensions they were entitled to increased. Steps towards debt resolution was also observed at follow-up in the interviews with ex-offenders.
  • The study demonstrated modest improvements in money management. There was an increase in both client groups reporting to be organised with finances. Ex-offenders also reported improvement in feeling in control of their finances, with only minor improvements reported from young people.
  • Both client groups reported an increase in confidence towards approaching household income and expenditure, being able to talk to people who can give advice about money, and in negotiating with creditors or organisations they owed money to.
  • At follow-up, the ex-offenders reported considering longer-term goals, suggesting that they had broadened their consideration, and recognised the importance, of future long-term financial goals.
  • Through the pre-and-post questionnaires, the evaluators identified improvements in clients’ abilities to manage their finances day-to-day, along with a reduction in the extent to which they worried about their situation following the intervention. At follow-up, interviews with ex-offenders revealed how mental wellbeing improved for those who reported practical changes in their situation.
  • The evidence suggests the project made progress against the intended outcomes and had a positive impact on financial capability, with follow-up interviews and case studies supporting the monitoring data. The findings suggest ex-offenders benefited from the programme more than young people.

Points to consider

Methodological considerations:

  • The analysis was limited by the small sample size at follow-up.
  • As there was no control group, it was not possible to determine causation. Therefore, the evaluation could not draw definitive conclusions about the effectiveness of the programme.
  • Stakeholders were concerned that the data monitoring questionnaires may have impacted on the intervention itself, making clients less willing to engage in the project due to both the length and language of the questionnaire.
  • The project delivery changed for young people to be predominantly delivered off-site in one-off group work sessions in venues they were familiar with, meaning the hub setting could not be assessed.

Applicability/generalisability

  • The project experienced ongoing issues with engagement, with only 40 of the 164 clients providing follow-up data. The small sample size and analysis using unmatched data therefore limited the evaluators’ ability to generalise findings to wider populations.

Full report

What Works for You project evaluation - full report

Key info

Programme delivered by
Exeter Community Housing Aid
Year of publication
2018
Country/Countries
England
Contact information

SERIO – University of Plymouth serio@plymouth.ac.uk