Justice Data Lab
The Justice Data Lab (JDL) is a service that statistically evaluates the impact of rehabilitation interventions, by providing secure access to sensitive data and technical analysis to the NGO sector and beyond. The JDL publishes such robust evidence to help inform the bigger picture in what works to reduce reoffending.
The demand for access to government data to inform all sectors is growing all the time and statistical offices are rising to the challenge of being as transparent as possible. Among the 200 criminal justice charities surveyed about their experiences accessing offending data from government, around half tried to access this data, and only a fifth were successful on any given attempt. Over 80% of charities surveyed found the process of accessing data hard some or all of the time. There was a clearly demonstrated need to form a solution to improve data access.
As part of this transforming landscape, the Justice Data Lab (JDL) became a leader in frontline service evaluation in 2013 to feed into the bigger picture of ‘What Works’ in reducing reoffending behaviour. The JDL demonstrates how data security obstacles can be overcome and how statistical techniques have enabled innovation in using administrative datasets above and beyond their initial purpose.
This service continues to evolve to remain progressive in an ever-changing environment and shares lessons learned to inform development of similar working models in other Government departments as well as in other countries across the world. The JDL has been well received by those working directly to change the lives of offenders, policy makers and academics to understand what the results are telling us about the impact on changing an offender’s inclination to reoffend. We work closely with existing and potential customers to make sure that developments continue to help answer key questions.
What Makes Your Project Innovative?
The JDL is the first service for frontline rehabilitation organisations that overcomes evaluation difficulties by providing easy access to sensitive datasets as well as vital evidence to those providing rehabilitation in a secure and transparent manner. The JDL provides easy access to sensitive datasets as well as vital evidence to those providing rehabilitation in a secure and transparent manner. It uses highly-sensitive datasets, in particular the Police National Computer (PNC) which details the criminal history of every offender in England and Wales, to produce statistically robust analyses at an aggregated level to organisations who provide information on those who have taken part in their recidivism-focused intervention.
Collaborations & Partnerships
Working with the think tank New Philanthropic Capital helped understand the key problems faced by programme leaders and charities when it came to evaluation, understanding their impact on reducing reoffending in a robust manner and the ability to access the relevant data. They continue to be strong advocates of the JDL and advertise the service to introduce potential customers to the service.
Users, Stakeholders & Beneficiaries
To get the service off the ground, we needed senior colleagues to buy into the JDL concept and to drive the change required to ring-fence sufficient resource, achieving ministerial sign-off both at the pilot stage and to convert the service into a ‘business as usual’ offer. Through engagement with the JDL steering group, we make sure we retain links across relevant charities, academics and other sector leaders to help give appropriate perspective to development work and to share ideas to enhance the service.
Results, Outcomes & Impacts
Results to the end of June 2017 show that over 160 JDL comparison group analyses have been completed. Of these analyses:
- 42 analyses indicated statistically significant reductions in reoffending on the one-year proven reoffending rate • 112 analyses indicated insufficient evidence to draw a conclusion about the effect on the one year proven reoffending rate. Of these 112, 14 analyses detail statistically significant reductions in the frequency of reoffending
- 11 analyses indicated a statistically significant increase in reoffending on the one-year proven reoffending rate This shows that the majority of results have not crossed the significance threshold, most likely due to the size of treatment groups analysed. As the awareness of the JDL service grows, we expect that requests will continue to grow, both from new, small organisations trialling innovative approaches to address recidivism as well as from large organisations that should provide a greater number of conclusive findings.
Challenges and Failures
A key challenge when the JDL was initially set up, it was not able to incorporate Offender Assessment system (OASys) information that details criminogenic needs and issues of offenders. As such, it was deemed unsuitable to analyse any interventions that focused particular on drug and alcohol-based rehabilitation. Following the successful pilot, we worked with experts who owned the OASys information to determine the most appropriate way of incorporating the information to be able to control for aspects such as reliance on alcohol and substance misuse.
Conditions for Success
Even if the appropriate resource is available in terms of people, technical skills and appropriate IT packages, such a service is not possible unless there is buy-in from senior management (and in our case, ministers). Particularly as it is likely that setting up such a service would be in tandem with the existing business as usual role, so it is important that sufficient space is ring-fenced to make sure that the focus is not distracted.
The model of the JDL service can be implemented in a number of different areas with relative ease. Some may not require the same levels of security for accessing key datasets but the provision of technical analytical support would continue to be vital, particularly to the Voluntary and Community Sector. The JDL has shared their knowledge and expertise to a number of government departments in the UK and it was encouraging to see this process being tested by the Department of Justice in Northern Ireland (DoJNI) successfully in 2015 and repeated in 2016, showing that the approach can be applied elsewhere and adjusted to fit the purpose of the Department itself.
Similarly, the JDL fed into a funding proposal in New South Wales, Australia to get a Data Lab set up there by an independent organisation, proving that geography is not a hindrance.
The JDL team is enthusiastic about sharing their knowledge of their way in evaluating impact and how this can be translated in other settings, both nationally and internationally. We have shared lessons learnt with several other Government departments who are working on setting up similar services.
Key lessons learned include finding multiple ways to engage with the sector you are focusing on – both in terms on ensuring a steady supply of requests as well as making sure that any development work is helping to answer the right questions. Also to be clear that such evaluation is part of the answer, and can be supplemented with qualitative evidence to determine the impact of rehabilitation interventions, particularly if reducing reoffending is a secondary aim (for example, if secure accommodation is the primary goal).