Evaluation Scotland Wales

evaluation

An evaluation of the Family Fortunes programme

Evidence type: Evaluation i

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

The Family Fortunes (FF) programmes specifically targeted families with English as an Additional Language (EAL), who had children aged between 7-11, living in disadvantaged areas, and who wanted help and advice to make their money go further. It aimed to change families’ attitudes to, and knowledge about, money, and the ways they manage their money.

Six pilot programmes were developed and delivered in Birmingham with Smartlyte, a local learning delivery partner to test the delivery model. Based on this pilot work, Campaign for Learning (CfL) provided a series of one-day training events for 85 tutors and ongoing support to 22 FL tutors to deliver the FF programmes.

The 36 FF programmes that formed the fieldwork ran from June 2017 to March 2018; in addition, researchers evaluated 31 other Family Learning (FL) programmes, which acted as a control group. Courses ran for 10 hours, across a variety of geographical areas across England, and the majority took place in classrooms on primary school sites.

CfL provided a series of one-day training events and ongoing support to FL tutors to deliver the FF programmes. 85 tutors attended the training and 22 tutors delivered the FF programme. Courses ran for 10 hours, across a variety of geographical areas, and the majority took place in classrooms on primary school sites.

The study

The main research question was:

  • What are the changes in the financial capability outcomes (attitudes, knowledge, skills), and the financially capable behaviours experienced by parents with EAL and their children after their participation in the FF programme?

The secondary research questions were:

  • How does training Family Learning tutors impact on their confidence and ability to deliver effective FF programmes for parents with EAL and their children?
  • Which aspects of the FF programme work well, which might need further improvement and how it could be developed and enhanced?

The evaluation of the FF project was carried out by University College London,The Institute of Education (UCL/IOE) and used a mixed methods approach. The quantitative methods consisted of two surveys, with both parents and children, which were administered by tutors at the beginning (TIME 1) and end (TIME 2) of the course. The design included quasi-experimental features where parents and children on the FF programmes (the intervention group) were compared to parents and children on other FL programmes unconnected to money or mathematics (the control group). The evaluation also included classroom observations, FF tutor qualitative surveys, and parent interviews including follow up interviews between four to seven months after the FF programmes finished.

Thirty-six FF programmes (intervention group), delivered by 22 FL tutors, and 31 other FL programmes (control group), ran from June 2017 to March 2018. Overall numbers of parents attending FF courses, including the six in the pilot programme, exceeded 300. Almost all of these parents had children who either attended the course, or participated in FF activities at home, bringing the total number of beneficiaries to over 600.

Key findings

Tutors’ confidence and ability to deliver the FF programmes

  • The training and ongoing support gave tutors the confidence, knowledge and skills to teach effective financial capability programmes that improved financial capability outcomes for parents with EAL and their children.
  • Tutors felt well prepared, and were given the tools and resources to deliver the FF programmes successfully.
  • Two thirds of the tutors rated the training day as being ‘excellent’, while nearly a third assessed it as ‘very good’. Tutors found the on-going support, in the form of weekly webinars, very helpful.
  • Tutors rated the FF programme highly - the practical ideas, concepts and resources - and enjoyed teaching it.
  • Some tutors invited other practitioners into their classes to watch them teach, and some of these went on to run their own FF courses, which suggests that a model of cascading was working.

Parents’ financial capability outcomes (attitudes, knowledge and skills) and behaviours

  • 98% of parents with EAL thought their FF course had changed how they dealt with money.
  • Half of parents with EAL on FF courses experienced a positive change in their knowledge about at least one area of specific financial knowledge. The areas of the greatest changes were in their knowledge about debit and credit cards, and about the rate of VAT.
  • 45% parents with EAL on FF courses experienced positive changes in their confidence to find out financial information online.
  • 64% of parents with EAL on FF courses experienced positive changes in their money management behaviours in one or more of 8 behavioural indicators. The highest proportion of those who experienced positive changes in their financial behaviours was in the four areas of: being consistent and sticking to their original financial decisions (43%), saving up for a special item (41%), ranking decisions about expenditures in order of importance (38%), and planning ahead about what to buy (38%)
  • Further analysis also suggests that overall 71% of EAL parents experienced a change in at least one of the three financial capability areas (knowledge, confidence and behaviours) after attending FF programmes. The proportion of parents increases to 74% if we include positive changes in their confidence associated with being a role model for their children, and the frequency of discussing money matters with their children (please see 1.3.3). That is slightly below the programme target of 80% specified in the Theory of Change.
  • Interviews suggest that the FF programme was having a profound effect on developing a large number of parents’ knowledge and skills in money matters, and on their attitudes and behaviours, particularly in the three areas of planning, budgeting and spending behaviours. They also suggest that many families have been able to gain greater control of their personal finances.
  • Qualitative data also show that these changes were being maintained a few months after the courses had finished.
  • Data provide evidence to support a hypothesis that parents with EAL who attended FF programmes not only experienced positive changes, but they experienced greater change in their knowledge about specific areas of finance (e.g. differences between credit and debit cards; interest rates; VAT; calculating sales and discounts), compared to parents with EAL who engaged with other FL programmes.
  • However, the small sample size means that the evidence is not strong enough to conclude that the changes experienced by parents with EAL, who attended FF programmes, with regards to their money management skills and behaviours and digital skills relating to finance, are significantly different compared to those on other FL programmes.

Parents’ confidence, knowledge and ability to support their children’s financial capabilities

  • Almost half (45%) of parents with EAL talked more often about financial matters with their children after attending FF courses.
  • Parents with EAL who attended FF courses experienced greater changes in their confidence to be role models to their children about good money management skills, compared to similar parents in the control group.
  • Parents with EAL on FF courses experienced a slightly greater increase in the frequency of talking to their children about money matters, compared to those in the control group, but the evidence is inconclusive due to the small sample size.
  • Qualitative data revealed that many more parents wereinvolving their children more in money matters, such as by including them in decision making when going shopping, looking at financial websites together, and discussing the difference between needs and wants.
  • Many parents with EAL felt that the course had been of enormous benefit, not only to their child who physically attended the course, but also to other siblings, and in some cases, their partners.

Children’s understanding of money

  • Over three-quarters of all children at the start of FF (76%) and other FL programmes (70%) thought that learning how to spend their money well was very important, which meant there was no evidence for significant change over time as there was little room for improvement.
  • 64% of children with EAL said the programme helped them a lot to learn about money.
  • 48% of the parents agreed that their participation in FF courses had had a great impact on their children’s knowledge about managing their own money.
  • The greatest positive change happened in how often children with EAL decide to save up to buy something special (33%), the frequency of them checking their change (30%), and also how often they talked to their parents about the costs of things (30%).
  • There is some evidence to show that children’s mathematical calculations improved in the FF courses, and also that by the end of the intervention course, more were aware of the need to shop around to gain better value for money.
  • Overall 61% of children with EAL on the FF programmes experienced positive changes in at least one of the financial capability indicators (knowledge, confidence and skills) and 38% demonstrated changes in at least two indicators or more. This is lower than the target of 70% specified in the Theory of Change.
  • There is also some indication that EAL children from FF programmes experienced slightly greater changes in checking to see if they had the correct change after buying items, compared to children from EAL families on other FL programmes.
  • Many parents who were interviewed thought their children’s understanding of money, their attitudes and behaviours towards money, and their awareness of money issues, had developed and changed. This was a result of the course, and because of behavioural changes within the parents themselves, which included more of them acting as good financial role models. A few parents stated that they thought the course had had more impact on their children than on themselves, particularly in terms of saving.

Areas of the FF programme that worked well and areas that need to be developed

Areas of the FF programme that worked/are working well

  • The materials and resources, and Schemes of Work are excellent. Tutors found the ideas easy to teach and the learners were highly engaged.
  • 85% of parents rated the FF course as being either ‘amazingly helpful’ or ‘very’ helpful in teaching them how to manage their money more effectively.
  • Most parents (73%) thought the course should have been longer, and almost all (95%) said they would recommend it to another parent.
  • The most useful things that parents talked about learning from the course concerned learning to budget, or manage, their money better, plan more effectively, and spend their money more wisely.
  • Tutors were well-organised and prepared, and created a relaxed and comfortable atmosphere.
  • Tutors employed learners as a resource, to good effect, to share their own financial strategies.
  • Classes were more effective when children were able to join their parents.
  • The quality of the accommodation varied but, in the main was good, and this included a strong internet signal.
  • Provision was particularly effective for ESOL learners when classes contained additional support from a second practitioner for learners with less developed language skills.

Areas that needed/need developing

  • Despite the pilot project suggesting a high level of parental enthusiasm this was not replicated in some other areas of the country, and some FF courses struggled to get off the ground.
  • There was some confusion amongst parents and schools about what FF was about, and what it was trying to achieve. The title of ‘Family Fortunes’ was rather vague and information about its purpose and objectives needs to be made clearer. In the future a strapline, ‘Making Your Money Go Further’, is going to be added to the main course title, which will make the aims more transparent, and means that the programme is marketed on a more positive note, rather than based on a top down model of ‘advising’ people what to do.
  • Some schools were reluctant to release children, particularly as they were at KS2, for shared activities with parents as a part of the FF programmes. This situation is aggravated when courses run in the morning, at a time when schools place a greater emphasis on pupils working on core curriculum subjects, and when some children are involved in practising for national assessments. Where children did attend the FF classes attendance was sometimes sporadic. This was mitigated to some extent by supporting parents within the session to be able to deliver the FF children’s activities themselves at home.
  • Some courses had problems with technology, which is an integral part of the FF programme. CfL will develop additional guidance for tutors to conduct a pre-course digital audit, including assessing issues of connectivity.
  • The FF programme was designed primarily for learners whose English language levels ranged between ESOL levels Entry 3 and Level 1, and tutors were advised to include additional language support where learners had specific language support needs. Some courses included ESOL learners whose level of English was assessed at Pre-entry and Entry 1 and where a second practitioner was not present to provide additional language support, some parents experienced difficulties in using resources, accessing websites, understanding many of the concepts, and completing the surveys. In response to this, CfL will emphasise the target audience for the existing programme, and the need to provide additional language support in future tutor training. CfL’s has plans to create additional modules, activities and resources for parents who are assessed at Pre-entry to Entry 2.

Points to consider

Methodological limitations, relevance, generalisability/transferability and applicability

  • Although over 300 parents attended the FF courses in total, only 168 EAL parents and 109 children from EAL families provided the project with quantitative data by completing surveys at both time points. Similarly, only 117 EAL parents and 91 children from EAL families provided completed survey data from the control group, and this meant that the conclusions are not as robust and conclusive as they would have been with higher numbers.
  • Follow up interviews to assess the impact of the programme took place at different time points, and some parents may have needed more time to incorporate the full learning outcomes from the programme into their behaviours and daily routines.
  • Although the fieldwork was extended until March, the cut-off point for analysis was late February. This meant that parents from the autumn and spring terms could not be interviewed 6 months after their courses had completed, so missing one of the evaluation plan’s targets. Only six parents were interviewed from the summer term (over 6 months after completing their course).
  • Because FF programmes were self-selected, it seems possible that a larger proportion of parents on these courses, as opposed to learners attending general FL courses, were already thinking about financial issues because money was tight. Some had a greater incentive to develop financial management strategies, and this meant that a higher proportion of these parents had less room for new learning and improvement in terms of their financial knowledge and skills than parents in the control group.
  • Some children completed surveys at home and it is impossible to know how much assistance they might have received from their parents, or from other family members.
  • Although researchers used biographical data, classifying whether a learner was EAL was not always straightforward and, for instance, some parents designated as being EAL had a high level of spoken English. Moreover, although parents lived in areas of disadvantage (designated by post codes), some had highly-developed financial skills, and some ran their own businesses.
  • The short project lead in time between the training organised by CfL and the end of the school term put some FL tutors under pressure, and some did not have enough time to set up and run FF courses.

Key info

Activities and setting
A series of one-day training events delivered in the classroom.
Programme delivered by
Campaign for Learning
Year of publication
2018
Country/Countries
England
Contact information

www.campaign-for-learning.org.uk