Evaluation Scotland Wales

evaluation

An evaluation of smart phone apps and financial capability

Evidence type: Evaluation i

Description of the programme

This programme aims to develop mobile apps designed to enhance financially capable behaviours and help people from disadvantaged communities make better informed decisions about how to tackle debt, manage money day-to-day and prepare for life events. Four apps, packaged together under the title ‘Money Matters’, were provided to 240 working age members (16-65 years) of Derry Credit Union in Northern Ireland, from February 2017 and April 2018.

The study

The study sought to explore whether mobile apps can:

  • Increase personal and financial well-being and confidence;
  • Help those at risk of over indebtedness;
  • Help those excluded from mainstream credit make well informed credit decisions;
  • Improve basic financial skills, attitudes and abilities to meet financial commitments.

Researchers conducted a randomised control trial (RCT) of 500 participants divided between a control group of 260 (those who did not receive the intervention) and a treatment group of 240 (who received the ‘Money Matters’ app package). Participants were members of a credit union and as such many members fall into the struggling and squeezed segments of the population. Baseline survey results from before the intervention (June-August 2017) were then compared with follow-up surveys post-intervention (February-March 2018), to assess changes in financial mindsets and capabilities. App usage was assessed through usage statistics provided by the app developers as well as through additional questions in the follow up survey.

Key findings

  • Confidence analysing financial information: Access to mobile apps significantly increased the probability of being very confident when shown information about financial products such as a loan, credit card or store card by 7%, though this finding was of borderline statistical significance.
  • Attitudes towards financial planning and forward-thinking: Access to the mobile apps increased the probability of claiming to prefer ‘planning for tomorrow’ by a statistically significanct 5%.
  • Saving and budgeting: No significant results were found for measures capturing whether money is being managed well on a day-to-day basis including budgeting and keeping track.
  • Levels of over indebtedness, bills and commitments (Ability): No significant results were found for those measures designed to capture levels of over indebtedness, or for indicators measuring keeping up with bills and commitments, or for measures capturing product holding and credit use.

Mobile app usage:

  • Over the course of the RCT the apps were most frequently used at the outset and when two reinforcement exercises were undertaken. Engagement and user retention problems were regularly identified.
  • The apps were more frequently used by those with higher levels of education. Among frequent users, 69% were university educated.
  • Perceptions around the quality of the apps were encouraging as 58% identified their ease of use as being the ‘most liked’ aspect and 51% stating that they would recommend the apps to a friend. Only 4% indicated a dislike of the apps.
  • Infrequent users stated that they would use the mobile apps more often if the information provided by the apps was of greater relevance and if they had greater confidence in being able to understand the information provided by the apps.
  • A quarter (25%) of those provided with the apps believed that as a result of using the apps they now thought more about how money advice and guidance could help them; understand the importance of timing in repayments and interest charges and have a greater awareness of their future financial needs and the importance of setting financial goals beyond the day-to-day.

Points to consider

Methodological strengths and limitations:

  • Due to statistical testing and an acceptable sample size, the quantitative results are robust and can be used to build a case for similar interventions.
  • Due to data protection, the study was unable to make use of savings, loans, arrears and withdrawal information from credit union systems which had originally been part of the research plan.
  • ‘Intended-to-treat’ approach means all those in the treatment group were analysed regardless of whether they downloaded or used the apps available to them. 176 of the 240 treatment group downloaded the apps.
  • Incentives were provided to both treatment and control groups to participate in the study.
  • The project had a limited duration of one year, while many of the impacts on financial activities may emerge over longer time periods.

Relevance:

  • This study is relevant to all stakeholders and policymakers with an interest in using mobile-based apps to help people become more financially capable.

Generalisability:

  • Although the results are statistically tested, it must be remembered that they only pertain to members of one credit union in one area of the UK, and therefore the results may not be transferable to other settings.

Key info

Client group
Activities and setting
The programme delivered financial management apps to working adults in Northern Ireland.
Programme delivered by
Queen’s University Belfast in conjunction with Derry Credit Union
Year of publication
2018
Country/Countries
Northern Ireland