Using Randomised Controlled Trials to Improve Public Service Outcomes

London Borough of Lambeth

LARIA Research Impact Award 2015: Best Use of Local Area Research

Lambeth has developed a radical new approach to evidenced-based decision-making, using Randomised Controlled Trials – the ‘gold standard’ for evaluation. Lambeth is increasing revenue collection and delivering improved outcomes by using RCTs to determine the effect of interventions, coupled with behavioural insights which demonstrate that small changes can make a big difference.

Summary: Randomised Controlled Trials (RCTs) are the ‘gold standard’ for evaluation. Unlike other methods, RCTs can help to accurately determine the attribution of an intervention through the use of a randomly assigned control group.

Resource scarcity makes it essential that how we work is as effective as possible and Lambeth has developed a programme of RCTs that improve decision-making and have directly contributed to improved performance and delivery of outcomes.

The council began with an RCT designed to test how council tax revenue could be increased by altering the bill sent to residents. The trial found that simplifying the bill led to a statistically significant increase in the number of people paying their council tax on time. Building on this initial success, a further seven trials are either underway or in development. These trials aim to improve the council’s performance across key policy areas and core challenges in order to strengthen an evidence-based approach to delivering corporate outcomes.

To embed the use of RCTs across the organisation we have instigated capacity-building training on the use of RCTs and behavioural insights, dissemination of trial findings and support to officers on trial design and delivery.

Synopsis: With millions of pounds having to be taken out of the system, ensuring that revenue is maximised is crucial. Increasing council tax collection by just one percentage point generates over £1m in additional revenue, whilst debt recovery adds significantly to administrative costs. Therefore increasing council tax collection rates can provide significant additional revenue, reduce debt-recovery costs and help residents avoid incurring additional costs (and potential financial hardship).

Lambeth’s first Randomised Controlled Trial (RCT) tested ways of increasing council tax payments by varying the standard bill sent to residents. The trial was conducted across 8,000 households in three wards. The simplified version of the bill drew attention to the key information and action required, and increased the proportion of people paying on time by nine percent compared with the control group. As a result of this finding the standard council tax bill has now been redesigned to reflect the simplification treatment – a clear example of local evidence-based decision-making. A further trial is now being run, testing the use of a social norm – stating that over 95% of Lambeth residents pay their council tax.

The effect of personalisation of text messages is being tested through a trial reminding residents to pay their council tax. The RCT is still running, but interim results suggest that the use of individuals’ names helps to increase response rates. Following the completion of the trial, the council intends to roll out text message personalisation to other service areas, such as reminders of benefits entitlement.

RCTs have also helped avoid making costly but ineffective changes, as well being used to make effective changes. One trial tested Dan Ariely’s theory that moving the placement of a signature to the top of a form would reduce the incidence of fraud and error. Our trial tested the effect of signature placement on Single Person Discount renewal claims. However, unlike Ariely, we found that the variation in signature placement made no difference to response rates. As a result, the approach was not adopted, avoiding the unnecessary expense of adopting an ineffective redesign.

Our use of RCTs is not confined to revenue collection: trials are being developed across the council to strengthen evidence-based decision-making, to reduce waste, to increase volunteering, to improve internal communications and to support council programmes.

Our RCTs have been developed through collaboration with Professor Peter John, a leading academic in the use of trials in public policy. Professor John (who is a technical advisor to the Behavioural Insights Team) has worked closely with Council staff to design, deliver and review trials. Our aim has been to build internal capacity and capability in the use of RCTs and to support their widespread adoption as a core methodology within the council. Whilst we still have a way to go to achieve our objective, the progress so far has been hugely encouraging.

What should LARIA members learn from this award entry? 

RCTs are not, despite common perceptions, prohibitively expensive or complicated to use and local authorities can adopt them as a routine methodology for evaluating interventions. Lambeth has demonstrated that RCTs can be successfully employed to determine the effectiveness of particular interventions, and this in turn can be used to guide and inform decision-making. The approach can be applied to a wide range of policy areas and services in order to ensure that assumptions are properly tested and decision-making is based on robust evidence of what works.

Public bodies are facing huge financial challenges and there is widespread acknowledgement of the need to work differently. Change and innovation is the ‘new normal’ in the public sector, and yet too often these new ways of working are not evaluated in ways that determine how much of a change can be attributed to the intervention. The use of a randomly assigned control group provides the reassurance that any observed change is the result of doing things differently – and not a result of other external factors.

Piloting new approaches is commonplace but doesn’t tell us if the approach will have the same effect if it is rolled out. Using RCTs significantly increases the quality of evidence and the confidence we can have in decision-making.

Photo credit: ‘The London Eye @ Night’ by Mostaque Chowdhury