Evaluation of Embedding Talk, Learn, Do in Wales

• Description of the programme
• The study
• Key findings
• Points to consider

Description of the programme

The Talk, Learn, Do (TLD) is a programme funded by the Money and Pensions Service (MaPS) and delivered by Campaign for Learning, an independent learning charity. The programme aims to test the feasibility of improving the financial capability of children through supporting parents to talk to and teach their children about money. TLD was originally trialled in Wales, and was evaluated to have had positive impacts. MaPS identified a need to further embed the programme in Wales to fill a gap in support for parents. The aim of the embedding project was to:

• Deliver at least 40 training sessions to practitioners.
• Deliver training to 500 practitioners (revised to 410 following discussions between Campaign for Learning and MaPS).
• Ensure at least 1,000 parents are supported.

The project’s delivery model differed from the initial pilot by offering training to practitioners from a wider variety of backgrounds, and including a wider variety of parents in a wider variety of settings. The project also included webinars to inform stakeholders and managers of practitioners. Although the majority of practitioners had been trained within the timelines of the project, delivery to parents was made extremely challenging due to covid-19 restrictions brought in from March 2020.

The study

Although the programme delivery to practitioners almost met targets, delivery to parents was disrupted; it is estimated that at least 191 parents participated in a session where at least some elements of TLD were delivered to them.

The evaluation comprised the following:

• Project monitoring reports compiled by Campaign for Learning.
• 309 completed ‘point of delivery’ (PoD) practitioner evaluation questionnaires.
• 70 responses to a practitioner follow-up survey.
• 40 qualitative interviews with practitioners.
• 20 stakeholder interviews.
• 15 responses to a paper-based ‘postcard survey’ of parents.
• 14 qualitative interviews with parents (including 4 children)
• A session observation which included 9 parents.

The study was conducted by Arad Research, an independent research company.

Key findings

The report lists extensive detailed findings. Overall, the authors concluded the following:

The evidence shows that practitioners working in a variety of settings and contexts have been successfully trained to deliver TLD. Practitioners and stakeholders indicate they have valued the training and reported that they intend to deliver TLD in future, as it meets a need among parents.

Parents indicated they engaged with TLD through practitioners with whom they already had a trusted and established relationship. They had positive views of the content and had started to put some of the ideas into practice with their children. However, delivery, especially to parents, was adversely affected by the Covid-19 pandemic which has limited the extent to which the programme has been embedded.

The project indicates it has had positive impacts on practitioners’ knowledge, skills and behaviours, including changes to their own financial capability. However, given the amount of time that has elapsed since the practitioner training, there is a risk that practitioners’ knowledge about TLD, and confidence to deliver it, could diminish over time. This suggests that further training and support may be needed to encourage and support organisations to embed TLD in future.

TLD is well-aligned with national-level strategies and organisation-level programmes and there are further opportunities to further align and embed these.

TLD could be further embedded through promoting its use within organisations, linking it to other programmes engaging parents, and designing new digital options and resources to enable ongoing delivery. At a national level, the programme aligns well with current strategic plans in Wales relating to tackling poverty and the new Curriculum for Wales. Ongoing dialogue between MaPS and its partners in Wales is needed to ensure that this potential is realised.

Points to consider

• Methodological strengths/weaknesses: The report discusses the limitations which include the following:
  • The PoD evaluation survey is administered immediately after the session, so views may not be sustained over time.
  • Practitioners who had more positive views may have been more likely to complete the follow-up survey and qualitative interviews. This was somewhat mitigated by included some practitioners who had not delivered TLD.
  • The parent postcard surveys were limited in sample size and should be considered indicative only of the views of the parents who attended these sessions, and cannot be generalised to the population of parents who have been supported by TLD.
  • The parent postcards were also administered immediately after the session and so we don’t know about the sustainment of outcomes. It was also a very short survey that didn’t go into much detail (or breadth) about the changes.
  • Additionally, some parents may have reported more positive findings as they were aware the practitioner might see their answers.
• Generalisability/ transferability: The study is specific to the TLD programme but there may be learnings for other interventions around supporting parents on any topic.
• Relevance: The intervention relates to supporting parents talk to children about money, which is important in developing financial capability
  • Applicable primarily to those involved in this particular programme although, again, there may be learnings for anyone delivering interventions to parents such as support agencies or educators.