These first 10 years, the LEO team has primarily been focused on creating evidence through rigorous research and has become a leader in that regard. But the use of evidence is becoming a greater priority–especially in terms of scaling and replicating proven programs.
What is Replication?
When it comes to our work, replication really means two different things. For a program it means taking what’s been done and proven effective in one environment and packaging it and replicating it in a new context. This is related to, but different from scale, which typically means taking what’s worked well and expanding it within the original context to serve more people. For research, replication means taking a proven program into a new context (new location and/or new type of organization) and replicating it with fidelity to achieve similar results as measured by implementing a replication RCT. One way to think about this is program replication can happen without research, but research replication needs program replication.
In the early years of LEO, we had the opportunity to research a couple of programs (“Stay the Course” in Texas and “Reading for Life” in Indiana) that both saw early evidence of promise. With each program, LEO researchers showed that the interventions were leading to the intended outcomes. Based on that initial evidence of promise, both providers decided to expand by way of replication. Here’s a closer look at each project.
Stay the Course® Learnings
Stay the Course is a program which pairs nontraditional students with a “Navigator” to help them overcome barriers that may otherwise prevent them from graduating. Upon completion of LEO’s initial research, Stay the Course was taken from one campus at Tarrant County Community College to all five of the college’s campuses. The provider, Catholic Charities Fort Worth (CCFW) set the goal of packaging this program and moving it outside of the original site via replication. With this goal came the need for increased fundraising, manuals, training and technical assistance for new sites, and for a replication study to determine if the new sites saw the same initial success as the original. In executing this plan, LEO and CCFW realized a few challenges:
- It was difficult to select new sites with partners equally committed to both the mission of Stay the Course and to continuing to build rigorous research.
- It was a challenge to achieve fidelity of program implementation and required more ongoing technical assistance from the provider.
- Replication sites wanted more control over the program and specific logistics or decisions throughout the implementation process.
- The funding model of using an RFP requiring a local match involved many local and national private foundations and donors, thus time and capacity.
Facing these challenges, and others, CCFW decided not to continue with the replication support and all but one site stopped running the Stay the Course program.
Reading for Life Learnings
Reading for Life (RFL) was a juvenile diversion program that used volunteers to guide first time offenders through reading groups. LEO’s research showed juveniles in RFL were significantly less likely to commit another crime and particularly less likely to commit a felony. The Reading for Life, Inc organization aimed to then scale the program and replicate it in three new communities with LEO doing a replication RCT to determine impact. In pursuing this rapid expansion, the Reading for Life, Inc organization faced many challenges, some of them being:
- The original designer of the program was also the person responsible for running the nonprofit, which proved to be a logistical challenge.
- New leadership at the nonprofit was committed to expansion and replication, but struggled to shift control of the program from the original designer to multiple new staffers.
- Criminal justice systems in different jurisdictions present challenges in creating buy-in due to having to receive sign-off from judges, probation officers, and other community stakeholders.
- The original intervention was delivered by volunteers, which proved cost-effective, but proved difficult with replication efforts due to challenges with recruitment and training.
Due to these challenges, Reading for Life, Inc. replication efforts ended.
A Model of Successful Replication
One of the most successful scale and replication efforts in social programs is the Nurse-Family Partnership® (NFP). This program partners specially trained nurses with first-time moms, starting early in the pregnancy and continuing until the child’s second birthday. This partnership, which centers around regular in-home visits, has proven to be successful via one RCT in the 1970s and two additional replication RCTs in the 1980s. Replication programs demonstrated the same strong results for mothers and children as the original and thus, an effort to scale nationally was begun. A national organization, the Nurse Family Partnership expanded and now has served over 366,000, since replication efforts began in 1996 and is a staple in 41 of the 50 states. A part of this success was policymakers supporting evidence—George W. Bush’s Administration piloted home visiting programs in 15 states and in 2010 the Obama Administration created The Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program. MIECHV now receives around $400 Million per year of federal dollars that are sent to states and local community agencies to run evidence-based nurse home visiting programs, like NFP, across the country.
NFP figured out crucial elements of replicating and scaling successfully. They built a full training and technical assistance program to ensure support and program fidelity at all sites. They have identified the appropriate and best beneficiaries of their program—first time mothers and their children. Further, they have identified the right kind of agency partners to run these programs in their own local communities.
Growing LEO’s Replication Efforts
LEO has big goals for building out and supporting replication efforts–both for our partners and for the vulnerable populations evidence-based programs can help. We have a strong partnership with the Goodwill Excel Center. Excel Centers provide an accelerated adult learning program to help adult students earn their full Core-40 High School degrees within 2½ years, with the goal of improving labor-market outcomes for graduates of the program. LEO conducted a quasi-experimental study which showed that Excel Center students have higher earnings and more stable employment. Because Goodwill Excel centers are already being replicated across many states, LEO is working with them to learn more about how Excel programs can benefit specific populations like justice involved persons and in different geographic locations.
While a key goal for LEO is to find the next Nurse-Family Partnership program and duplicate its amazing scaling success, LEO is spending time focused on meeting our provider partners where they are with sharing the impact of their programs—whether that means supporting the dissemination of results to their stakeholders or packaging it to be replicated in new communities. We’re excited for this work, as we see replication is a key to allowing our top notch research to have the broadest impact and serve the most people in poverty.
Stay in the know with LEO by subscribing to our monthly email newsletter!
Originally published by at leo.nd.edu on September 21, 2022.