Evaluation Scotland Wales

evaluation

Secondary Money Twist

Evidence type: Evaluation i

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

MyBnk deliver financial education to secondary school aged young people through the Money Twist (MT) and Sporty Money Twist (SMT) programmes. The Money Twist programme is delivered to Key Stage 3 and Key Stage 4 pupils in secondary schools, while the Sporty Money Twist is being delivered to a range of groups of young people outside of a school context, mainly, though not exclusively, within the National Citizens Service. In total, across MT 447 distinct programmes were delivered across 86 schools, reaching 16,860 pupils aged 11-16. For SMT a further 85 programmes were delivered reaching 3,501 pupils aged 14-18.

The study

In order to evidence the contribution MyBnk MT and SMT programmes have on the three key outcome areas, the following methods were deployed:

For Money Twist:

  1. Quantitative, pre-and post-delivery surveys of pupils at school settings.
  2. A follow-up survey 3-6 months post-delivery.
  3. Qualitative case study research (observations and focus groups at 5 schools).
  4. Surveys and interviews with teachers.
  5. Interviews with stakeholders.
  6. A control survey of pupils in schools not involved in the programme.

For Sporty Money Twist

  1. A quantitative post-delivery survey of participants in sporty settings.
  2. A follow-up survey 3-6 month post-delivery.
  3. Qualitative case study research (observations and focus groups).
  4. Surveys and interviews with trainers and staff from organisations where delivery took place.

In addition to the above methods, a short evaluative film has been produced which depicts the MT and SMT delivery, highlighting the central outcomes in this research report.

Key findings

Across all pupils The MyBnk intervention had a positive impact in all three outcomes.

  • For Outcome 1 - Increased understanding of the role in society: There was an average improvement of 14 percentage points in relation to ‘confidence to learn and talk about money’, an average of 24 percentage points improvement in ‘financial knowledge’ and an average 26 percentage point improvement in ‘understanding of the principles and roles of money in society’. Indicator increases ranged from 9 to 31 percentage points, and were statistically significant for both KS3 and KS4 MT pupils.
  • For Outcome 2 – Pupils have a better understanding of financial choices and decision making: 44% of pupils who would not have delayed financial gratification prior to MyBnk training now would (contributing to a 7 percentage point overall increase in pupils who would do this). 31% of pupils who would not have made a spending plan (and stuck to it) now would (contributing to a 7 percentage point improvement across all pupils). Pupils showed a 10 percentage point improvement in relation to understanding of personal money habits (ranging from a decrease of 6 percentage points to an increase of 14 percentage points).
  • For Outcome 3 - Young people have an improved understanding of financial concepts for their future: Pupils demonstrated an average 27 percentage point improvement in understanding financial services and an average 19 percentage point increase in understanding financial products and terminology. Improvements ranged from 18 to 34 percentage points and were statistically significant, although sample sizes were smaller for this outcome (due to fewer sessions delivered in this area).

Control group scores on these measures were largely similar to the pre-delivery MyBnk scores, indicating that MyBnk pupils, pre-delivery, have similar levels of understanding to the control group, but on the whole, higher after training.

Regarding process and delivery, teachers, pupils and trainers felt that the pace and varied delivery style worked well in gaining and retaining pupil focus during the sessions. There was further support for the teaching techniques such as quizzes, challenges and other interactive methods in supporting pupil understanding of new financial concepts. The expert nature of the MyBnk trainer brought an inherent additionality to the experience and pupils and teachers alike, spoke of the positive learning benefit of this ‘outsider influence’.

Points to consider

Methodological Considerations

  • The sample size was sufficient for a robust evaluation to demonstrate statistically significant results. The results obtained from the data provided are generalizable to the entire population who received the training because the sample reflected the range of contexts in which the programme has taken place, including school OFSTED rating, IDAC deprivation rating and school size.
  • To compare the effectiveness of MyBnk’s delivery compared to financial education delivered by teachers, the control group results were used as a proxy. However, responses in the post-delivery MyBnk survey were conducted immediately after session delivery, whereas the time between control group education delivery and survey is not known and could vary widely. This difference in timing limits the accuracy of this approach and in future, control data should be collected in comparable time frames.
  • Follow up survey periods ranged from three to six months, due to school administrative delays, therefore measurement of ‘impact’ across a single time period was not possible and this could affect results. More extensive follow-up analysis with a single time period and greater numbers would increase the usefulness of this element of the evaluation.
  • The combined qualitative and quantitative approaches to programme evaluation could work well for similar programmes and allow programme to programme comparisons such as using standardised measures including questionnaire templates and interview schedules.

Key info

Programme delivered by
MyBnk
Year of publication
2018
Country/Countries
England
Contact information

Substancewww.substance.netinfo@substance.net