Integrating Design and Behavioural Insights to increase the take-up of an education savings program for Canadians living in low income
The Canada Learning Bond, a government education savings program providing a financial incentive to low income Canadians, turned to behavioural insights (BI) to tackle persistent low take-up. Following unsuccessful BI trials based on academic literature and expertise, we turned to Design. In a Government of Canada first, combining lessons learned from human centred design processes including stories from citizens and BI we developed outreach that significantly improved the take-up of the Bond.
The Canada Learning Bond (CLB) is a financial incentive administered by Employment and Social Development Canada (ESDC) to encourage parents living on a low income to save for their children’s post-secondary education in Registered Education Savings Plans (RESPs). As of 2015, only 1 in 3 eligible children had received the CLB. Today, approximately 2 million children are eligible to receive the CLB. ESDC is running a program of research including iterative randomized controlled trials (RCTs) and qualitative design thinking approaches to understand the issue at greater depth and increase CLB take-up.
In 2016 ESDC conducted its first Behavioural Insights (BI) based RCT trial looking to increase the take up of the CLB through a promotional mailing to eligible families. The design of the letters was based on the latest and most relevant BI literature and was also peer reviewed by academics. Results from the trail were underwhelming. Though we were able to successfully conduct a RCT internal to government (a rare feat), many of the test letters statistically reduced take-up of the CLB when compared the standard letter used by the program in past outreach. The results suggested that the existing literature was insufficient to develop solutions to increase savings behaviours among Canadians living in low income.
In order to better understand the educational and financial decision-making of families living in low income we launched a Design project. Combining qualitative findings from a Design project to a quantitative BI project was a first for the Government of Canada. We engaged with 146 individuals through Interviews, workshops and ethnographic observations of service interactions. We engaged with parents and children (including Indigenous and rural populations), teachers, RESP providers, interdisciplinary stakeholders across government and community-based organizations. Iteratively co-creating research tools with our clients and other stakeholders ensured thorough qualitative data collection.
We were then able to design letters based on findings and insights from qualitative field research. Through this research numerous insights were gathered, including the ones outlined below. These insights/observations were translated into letter interventions which were tested in the trial to evaluate their effectiveness. For example:
Observation from qualitative research → lead to → Intervention/Hypothesis tested in the letters
Uncertain information in the letter (e.g., minimum or maximum CLB amounts that the child may receive) were highlighted by citizens as the most confusing parts of the letter. Moreover, older children are eligible for higher CLB amounts which they can claim retroactively (although most parents did not know about this). → lead to → Customizing the letter by presenting the exact amount of money that the child is eligible for (to date) could reduce confusion and increase CLB take-up.
An emphasis on ‘savings’ does not resonate well with all families living on a low income, as some may feel that they cannot afford to save. → lead to → Reducing the emphasis on savings in the letter could increase CLB take-up.
In order to test these insights, we developed a RCT targeting over 140,000 eligible children aged 12-13. Our previous trial found that this cohort, the oldest of eligible children, were also the least likely to sign up for the bond.
The findings from the trial were very positive. The letter which combined a the two insights described above improved the take-up of the CLB by 55% compared the standard letter (8.8% vs 5.7% take-up respectively). In practical terms, if we sent 10,000 standard letters we would expect 573 beneficiaries to sign up for the CLB. Combined, these families would receive $570,000 in education savings from the government. If instead we sent the letter informed by BI as well as the Design project, 878 beneficiaries would sign up. Combined, these families would receive $978,000 in education savings from the government.
The findings from this innovation have been implemented into the CLB communications and are now being received by eligible children.
The design project also provided a number of other insights that will be tested with other populations of CLB eligible children through RCTs.
Given the success of this mixed methods approach, we are considering it for a number of new challenges faced by the department.
What Makes Your Project Innovative?
This project demonstrates the successful use of mixed-methods that combined Design and BI, a first in the Government of Canada. Specifically, we gained a deeper understanding of the needs and experiences of Canadians living in low income using ethnography, and then integrated those insights into behaviourally informed communications. These were then tested using rigorous experimental methods (Randomized Control Trial). Given the success of the trial in that the letters based the insights derived from the ethnography outperformed the standard letter used in regular communications with eligible families, this project shows the complementarity of both methods and benefits of being able to test lessons learned from the design project (qualitative) at a large scale (quantitative) in a BI trial. The findings from this trial have been adopted by the CLB program for broader use.
What is the current status of your innovation?
With the successful completion of the first BI trial directly informed by a design project, the Government of Canada is now implementing principles from the highest performing letter into regular communications of the CLB program. Provincial and municipal governments, and community organizations are also looking to integrate these findings into their communications. We are now involved in a number of new trials to test and validate other insights generated from the design project, thus building knowledge rigorously and incrementally. Other insights generated by the design project remain to be integrated to BI interventions beyond letters and we plan to test and validate them in new trials. Additionally, we seek to extend the use of mixed-methods to other sticky problems faced by the department, given the success of this innovation. We are also disseminating these findings across the Government of Canada and hopefully globally in the near future.
Collaborations & Partnerships
The initial BI project involved academics contributing expertise in the field of behavioural science. The design project collaborated with provincial and municipal governments to better understand the inner workings and challenges of the program. We interviewed 126 Canadian citizens to gather their stories, concerns and aspirations regarding post-secondary education. Community organizations and teachers informed us about the best practices regarding working with Canadians living in low income.
Users, Stakeholders & Beneficiaries
The main beneficiary of the innovation is Canadian families living in low income. Because of this innovation, more of them are now benefiting from the CLB. Community organizations as well as other levels of government are also benefiting from our lessons learned as we continue to share our findings so that they may better promote the CLB to eligible families. Having such a project in government has also stirred interest in using mixed methods across other areas.
Results, Outcomes & Impacts
Outcomes included improved program take-up (i.e., increased education savings for Canadians living in low income), a greater understanding of the users (e.g., their perceptions of education savings, post-secondary education), and improved ability of the department to serve this population. A randomized controlled trial was used to conduct mailing trials comparing the effectiveness of different letters. Receiving the standard (no nudge, original) letter increased CLB take-up (5.7%) compared to a group that did not receive a letter (2.2%). Based on design project insights, presenting the exact amount of money they would receive when opening up an account increased take-up over and above the standard letter (8.0% take-up), and reduced emphasis on “saving” (living with low income, many participants felt this term did not resonate with them) further improved take-up (8.8%). Combining BI with design insights resulted in a 55% increase in take-up above sending the standard letter.
Challenges and Failures
This project may not have taken place were it not for the failure of the first BI mailing trial conducted in 2016. This trial showed that the best nudges developed performed only as well as the program’s standard letter, and the other nudges significantly reduced CLB take-up. These failures revealed our lack of understanding of our clients and spurred the development of the design project. Getting the design project up and running required substantial investments in time, including the lengthy process of obtaining clearance from the privacy division to be able to speak with Canadians. Though we have had successes integrating BI into the letters and increasing take-up, it is clear that letter communications will not fulfill the program’s enrolment goals. Other lessons learned from the design project that are more transformational in nature for the program have been suggested. However, these larger scale recommendations have been met with a degree of resistance by the program.
Conditions for Success
The Innovation Lab brought together design and BI, two complementary approaches, which was critical to the project’s realization. Another crucial component to this projects success is our access to eligibility and enrolment data for the CLB. This allowed us to conduct a randomized controlled trial and analyze its results all in house. The Innovation Lab has a high level of engagement and support from senior management. Without their support of experimentation and innovation in government, this project would not have happened. We were able to hire appropriate skills sets, including design, ethnography, behavioural insights, and data analytics. Funding covered travel costs to go meet Canadians in their homes and community organizations. Regarding values, this project could not have happened without the openness of the CLB program owners to be sometimes uncomfortable with innovation, and without the Canadians’ interviewed trust in government and belief in the value of education.
A project of this scale is a first in the government of Canada. Given its success, here at ESDC we are currently considering using this mixed-method approach to other program areas, such as employment insurance and student loans. There are also other similar efforts, though smaller in scale and complexity to conduct mixed-method projects in other federal departments. This approach does have the potential to be replicable in government and other types of organizations (e.g., private and non-profit). However, the Innovation Lab provides a unique environment conducive to innovation and experimentation and skills set that are unfortunately uncommon across other government organizations. Nonetheless, it does emit a strong signal of the value in building internal capacity for innovation and experimentation in government.
Much was gained by combining Design and BI to better understand Canadians living in low income. We achieved a better understanding of how living in low income interplays with programs that incent savings. For example, framing such programs as benefits instead of savings incentives, and removing uncertainties such as “could benefit up to”, can have positive impacts on program uptake. Finally, Canadians have expectations of government letter communications. We learned that “Bland is the Brand” (flashy, modern letters with images sometimes reduced take-up) and we should respect this for our communications to appear legitimate (not mistaken for spam).
Experimental methods can shed light onto the effectiveness of our programs and services. When management realizes the effectiveness of experimentation they are willing to act to learn more and innovate.
The success of this project depended on skills atypical and rare to government including ethnography, design, and behavioral science. From a broader perspective, the successes and failures experienced in our trials revealed the limits of using a one-size-fits-all approach with BI. Although BI principles are supported by decades of research, the reality is that interventions informed BI principles do not readily generalize across contexts, behaviours, and populations. Our research suggests that this seems to be especially true with vulnerable populations (low income families) who face unique challenges and barriers. As these populations have seldom been represented in the research supporting BI interventions, there is a need to adopt a tailored approach to their specific needs and experiences. The mixed-method approach proved to be a powerful tool to help us craft more tailored messages. Specifically, the qualitative research helped us better understand the program’s users, and in turn generated insights instrumental for designing more effective messages.
Here are a few quotes from citizens living in low income illustrating their complex situations which inspired some of our BI interventions.
“It’s hard to even picture something in the future now, because it’s hard to know what it will bring.”
“It seems really big – it’s adult – to talk about Registered Education Savings Programs (RESP) – It’s such an adult word. It scares me a little bit and it’s a bit intimidating.”
“I have heard about RESPs. The issue with that, it’s hard enough to save for the future for retirement – I don’t even have any retirement savings, let alone savings for education."
“It’s difficult as a low income person to take advantage of the programs that they do have.”
“Our kids, they need to have education, the same as the other kids. It doesn’t matter if you have low income, or high class.”
“Education is important for me for my kids, because I don’t want them to have to walk in my shoes.”
“I can see why people wouldn’t do it, because it is a process.”