Does robo-advice improve people’s loan repayment decisions?

• Description of the programme
• The study
• Key findings
• Points to consider

Description of the programme

When making loan repayment decisions, individuals tend to use counterproductive heuristics and biases, the negative impact of which shows through in wealth inequalities. The aim of this experiment, therefore, was to assess the adoption and effectiveness of robo-advice in helping to improve households’ loan repayment decisions and thereby reduce these inequalities.

The experiment involved the design of nine life-like debt-management games, each with three alternatives to reflect different levels of difficulty (easy, medium, hard). Across these 27 loan repayment problems in total, subjects were asked to fully allocate a specified sum of money across multiple loans in order to either minimise the monthly interest payment or minimise the sum of monthly interest payment and late fees (depending on the problem).

The pre- and post-intervention experimental design comprised random assignment to 5 conditions: 1 control condition (who received a repeat of the pre-test games); and 4 treatment conditions which provided free and paid-for robo-advice for loan repayment with and without additional educational features (e.g. tips and brief explanations).

In all treatment conditions, subjects were told the maximum interest and fees that they could save by using the robo-advisor (but received no feedback on optimal decisions), and those with paid-for robo-advice were asked about their willingness to pay for the robo-advice tool.

The study

The study sought to address four research questions:

• Whether (and by how much) exposure to robo-advice improved loan repayment
• Which forms of robo-advice were in the greatest demand
• How much subjects were willing to pay for robo-advice
• Which subjects ignored robo-advice recommendations

Data were collected in summer 2020 from 3,423 subjects who took part online; quota controls ensured a UK nationally representative sample of adults aged 18 and over on several demographic characteristics.

Each subject was randomly assigned to their treatment or control condition and, within this condition, completed nine problems in three phases of three problems each: Pre-intervention (no advice), Intervention (no advice, free robo-advice, free robo-advice with education, paid-for robo-advice, paid-for robo-advice with education), and Post-Intervention (no advice).
Subjects within the paid-for robo-advice conditions who sought advice were asked to state their willingness to pay for the advice tool using a slider that ranged from 0 pounds to the maximum possible savings available to them in interest and fees in their problem scenario. If the amount they were willing to pay was higher than a random cost allocated to their problem (36.6%), they obtained the advice.

Key findings

This study reports differences that are statistically significant where p<.10.

• Loan repayment decisions: The effect of robo-advice on subjects who sought it (‘Treatment on Treatment’) was large. The average losses (% savings forgone) of 21.9% at pre-intervention declined by 19.6 percentage points post-intervention (p<.01). The effect of robo-advice on subjects regardless of whether they used or disregarded it when offered (‘Intention to Treat’) was a decline of 14.6 percentage points (p<.01). When provided free, robo-advice was unlikely to be overridden, especially by the most vulnerable households.
• Willingness to pay for robo-advice: Subjects’ willingness to pay was higher than the monetary benefits they obtained from it on average. Other factors, such as the tool being easy and enticing, appeared to have played a role in this. Demand for robo-advice was greater among those with lower financial and numerical literacy, and also greater among those with low confidence in their own skills and high trust in robo-advice. Men and more trustful subjects were willing to pay more.
• Learning: Despite the behavioural improvements, there was no benefit to subjects in the treatment conditions in terms of their learning; they learned neither by imitation nor by the educational tips the robo-advice tool contained. In contrast, learning effects were observed among the control group, who had more problems to work through without robo-advice.

Points to consider

• Methodological strengths/weaknesses: The findings are based on hypothetical, experimental data, and may lack ecological validity, that is, it may not represent real-life situations well.
• Generalisability/ transferability: Although the loan repayment scenarios were life-like, the decisions subjects made were not real-life decisions which would affect them materially. Equally, the robo-advice tool was provided as the single tool subjects had access to and, within the ‘game’ environment, concerns around reliability and trust are likely to be attenuated. As such, we do not know how well the findings from the experimental setting would generalise into real life.
• Relevance: The experiment relates to the topic of robo-advice and therefore findings will be relevant to similar interventions or programmes.