Evaluation Scotland Wales
The UK Strategy for Financial Wellbeing is taking forward the work of the Financial Capability Strategy Opens in a new window

review

What works to improve wellbeing across interventions

Evidence type: Review i

Context

The Office for National Statistics (ONS) has created a set of four measures of personal wellbeing known as the ONS4. These measures are increasingly used across government departments as well as the private sector and third sector organisations. They are being used across a wide range of intervention types to evaluate effectiveness, including housing improvement and neighbourhood design; community-centred approaches; and skills training. However relatively little is known about how they feature and perform in wellbeing impact evaluations in the UK. This study, the first of its kind, was designed to explore the use of the ONS4 measures in wellbeing impact evaluations.

The study

The What Works Centre for Wellbeing (WWCW) conducted this study, which is a rapid evidence evaluation (REA) to map the quantity and quality of studies that use the ONS4 measures. This is part of a longer-term project to summarise and synthesise findings on the effectiveness of wellbeing interventions. Specifically, the aim of the REA is to answer the following questions:

  1. What impact evaluation research has been carried out to assess the impact of wellbeing interventions that use at least one of the ONS4 measures?
  2. What is the strength and quality of the impact evaluation research?
  3. What are the key findings from the impact evaluation research?
    The study sourced over 7,000 papers, both peer-reviewed, and grey literature (not published in peer reviewed journals), and assessed the titles and abstracts against a set of eligibility criteria. This yielded over 900 papers for full text review, of which 28 were found to be appropriate for extraction and data synthesis.

Key findings

  1. What impact evaluation has been carried out?
    • Intervention types: Organisations are now using the ONS4 measures to evaluate the effectiveness of a wide range of intervention types. These include: community-centred approaches, volunteering, physical activity, social prescribing, psychological interventions, social care support, skills training, advice and support, arts and culture, and housing improvement and neighbourhood design.
    • Target groups: Many of the interventions are aimed at one or more specific demographic groups, and are delivered across UK regions. Populations included: older people, working age adults, women, Black, Asian and minority ethnic groups, people living in deprived areas and people with underlying health conditions.
    • Growth in the use of ONS4: More organisations are using the ONS4 to evaluate wellbeing interventions. The first eligible study found was published in 2014 and more than half the studies identified were published between 2018 and 2020.
    • Users of ONS4: The measures feature in evaluations produced by a range of universities and consultancies, and they are used in a variety of research designs. The most commonly mentioned funder of evaluations was the Big Lottery and the large majority of studies are classed as grey literature.
    • Measures used: The most commonly used ONS4 questions were LIFE SATISFACTION and HAPPINESS and almost half the studies used additional non-ONS4 wellbeing measures.
    • Impact evaluation methods: Although collecting quantitative data was an eligibility criterion for inclusion in the REA, the majority of evaluations also collected qualitative data on wellbeing through interviews or focus groups with participants.
  2. What is the quality of the evaluation research?
    • The use of the ONS4 was more common in uncontrolled studies with moderate to low quality scores. However this may be that there was not enough information available to assess the quality of the evaluation, rather than that they were methodologically poor.
  3. What are the key findings from the evaluation research?
    • There is promising evidence for some intervention types, in particular social prescribing, which may warrant its own, more targeted review.

Points to consider

  • Methodological strengths/weaknesses: The REA used the same rigour as systematic reviews to identify and select studies, albeit with narrower research questions and less extensive review methods. Data extraction and critical appraisal of the selected studies was conducted by one reviewer. The quality assurance lead independently checked 20% of the extracted study results and critical appraisal.
    • The report has an accompanying study database with in-depth information on the evaluation design, sample, findings and quality score of each impact evaluation.
    • Applying a common critical appraisal framework across all studies selected was beneficial in terms of providing a comparative overview of the different contexts in which the ONS4 measures have been used. However, the level of detail provided in the different reports included varied considerably so this couldn’t always be done consistently.
  • Relevance: This is the first study to make an assessment of the ONS4 as used in impact evaluations so it is an important and relevant study
  • Generalisability/ transferability: The study can’t be generalised to any other body of work. It is specific to impact evaluations in the UK
    • The study is applicable to anyone involved in wellbeing interventions and impact evaluation, including government, support agencies, policy makers and policy implementers.

Key info

Year of publication
2020
Country/Countries
United Kingdom
Contact information

Peto, L., Pittam, G. & Musella, M

What Works Centre for Wellbeing