Evaluation Scotland Wales

evaluation

IT’s all about money – evaluation phase

Evidence type: Evaluation i

Description of the programme

This project was evaluated through the Money Advice Service’s (MAS) What Works Fund (WWF). The project aimed to assess the effectiveness of financial capability activities for retired older people. Group and one-to-one sessions were delivered on a range of money management topics. The project built on a three-year older persons’ money management service funded by Comic Relief. Both projects were delivered by the same partnership; Citizens Advice North Liverpool, Age Concern Liverpool and Liverpool Central Citizens Advice (now Liverpool Community Advice).

Delivery took place in community settings including sheltered housing schemes, community centres, older people’s lunch or activity clubs and pensioner forums. One-to-one sessions were held in advice centres or by home visit. Topics covered in the sessions included: maximising income, saving money on bills, managing money and staying safe online, later life planning and dealing with financial difficulties. 700 individuals took part in the project.

The study

Praxis CIC were commissioned to undertake the evaluation, which ran from March 2017 to March 2018.

The research question for the evaluation was as follows:

To what extent can a project that combines digital and financial capability support and impact older people’s financial capability and their overall financial resilience?

The evaluation explored the following assumptions:

  • Many older people have limited financial means and struggle financially in later life.
  • Many older people find it difficult to access money management tools, products and services (including online tools and products).
  • Many older people are not prepared for the unpredictable financial challenges of later life.
  • Some older people are reluctant to, or do not know where to, seek help if they are in financial difficulties.
  • Older people are susceptible to financial scams, including online scams.

The evaluation adopted a mixed methods approach. This included a baseline survey of all participants before the intervention. A follow up survey was conducted eight to 12 weeks after the initial intervention through a telephone interview. Baseline surveys were completed by 577 participants, and 241 completed the post-intervention survey. The evaluation also consulted a small ‘control group’ of 26 older people who completed both the baseline and follow-up surveys, but did not participate in any learning activities. The responses from the control group were used as a comparison, to help assess the impact of the project.

Key findings

Outcomes evaluation:

The following trends and characteristics were identified from the baseline analysis:

  • Less than half the respondents had internet access, with a similar proportion reporting low digital confidence. 65% felt well organised in managing their money; this figure rose to 78% for those with internet access.
  • Across the cohort, there was an even split between those who were satisfied with their financial situation and those who felt limited by it.
  • Participants reported limited awareness of income maximisation options, with only 27% having checked their benefits entitlement in the preceding year.
  • Participants demonstrated some awareness of planning for later life issues. Around half the cohort were aware of power of attorney options, and some expected support from their children and wider family.
  • Low levels of financial difficulty were reported at the project outset. Half the respondents rarely ran out of money, and only 15% did frequently. Less than half the respondents thought advice would help them make better financial decisions, while a similar number felt they would sort out their own money issues, without seeking advice.

The analysis of the post-intervention survey responses identified the following outcomes, which could be partly attributed to the project:

  • More participants were checking their benefits entitlement after the sessions. 40% of participants checked their benefits post-intervention, compared to 12% pre-intervention.
  • Participants were more willing to make changes and less passive about their financial circumstances. In particular, participants were more willing to change energy suppliers after the course.
  • More participants were willing to seek help with money problems. This increase from around half of participants at the start, to 90% of respondents to the follow-up survey.
  • More than 50% of respondents to the follow-up survey identified local independent not-for-profit advice agencies as a first resort for support with financial difficulties.
  • Participants at the follow-up stage demonstrated high levels of awareness of financial difficulty warning signs.
  • Few people had made any new arrangements to either safeguarding their finances, or save money.
  • There was little evidence of change in later life planning by the post-intervention respondents.

Process evaluation:

  • Overall the project was successful in reaching its target number of older people, for both group and one-to-one sessions.
  • The project workers’ efforts to engage a range of community partner organisations working with older people was key to the recruitment of the target group.
  • The interest and involvement demonstrated by older people’s intermediaries helped to boost attendance and engagement levels.

Points to consider

  • Methodological limitations: The evaluation timescales were reduced to a year owing to initial staffing challenges, reducing the potential to explore results over a longer period of time. This limited the extent to which the evaluation could assess if changes has been sustained by participants.
  • Generalisability/ transferability: A reasonable number of survey responses were secured. Statistical significance testing was conducted, which increases confidence in the results, and means that the findings could be generalised from to some extent. The study may therefore be of interest to providers wishing to establish similar provision in other contexts.

Key info

Client group
Activities and setting
Group and one-to-one sessions in community venues and via home visits.
Programme delivered by
CAB North Liverpool
Year of publication
2018
Country/Countries
England
Contact information

dave@praxiscic.co.uk