A Revolution Of Outcomes: How Pay-For-Success Contracts Are Changing Public Services

Across the country, and around the world, governments have been experimenting with a radical new way of delivering social services. They’ve begun “paying for success.”

Pay-for-success (PFS) refers to when governments collaborate with outside investors who put up money for particular interventions that save money in the long run, such as therapies to help homeless mothers get back on their feet or programs to stop parolees from landing back in jail. The contracts, agreed between officials and “impact” funders and delivered by third-party nonprofits, pay returns if the interventions meet pre-agreed targets–for example, a reduction in the rate of recidivism by a certain amount. In theory, PFS represents a win-win for everyone: governments get to spend less money, while investors make a profit and have the satisfaction of facilitating new types of programs.

There are now seven active pay-for-success contracts in the U.S., with dozens more on the drawing board. For example, New York State and Massachusetts have initiatives to reduce recidivism. Chicago has a program to raise education standards among disadvantaged pre-school children. And Cuyahoga County, Ohio, has a contract to reduce the number of days children of homeless mothers spend in foster care. At the same time, there are at least 30 open pay-for-success contracts in the U.K.–which introduced the first contract in 2010–and a few more in Australia, India and elsewhere.

Also called “social impact bonds” (though the contracts aren’t really bonds), pay-for-success (PFS) is a hot idea among people who want to see more accountability and experimentation in government. PFS offers new sources of funding from investors who want to see innovative ideas tried, they say. It encourages risk-taking, as governments don’t have to come up with upfront money, and they only pay investors if the intervention succeeds. And, PFS brings with it new standards for recording and analyzing data, and for assessing the actual effectiveness of interventions.

“Paying when they achieve an outcome, rather than paying to deliver a service, has the potential to free up the entrepreneurship in the social sector, where we see people feeling the ability to innovate and be creative,” says Ben McAdams, mayor of Salt Lake County, Utah, which has one PFS program to educate pre-schoolers and is exploring three more covering infant and maternal health, homelessness, and adult recidivism.

“Governments in the past may have launched a program based on what they think might be a good thing to do–it was a well-intentioned investment. With the vehicle of PFS, we can spend time thinking about target populations where we think we can have maximum benefit, and where there’s greatest potential for taxpayer savings,” says McAdams, a 40-year-old Democrat.

The truth about government

Here’s a shocking truth about government: Many of the services it provides don’t meet a rigorous evidence test. Even programs that we know people use and find valuable haven’t been tested to standards that are expected in other fields, including medicine and business. Most of the time, governments don’t know or can’t prove whether their programs are more effective than doing nothing, or whether the intervention is more effective than other possible interventions.

“With most large federal social programs like Head Start, Job Corps, or Title 1 funding for high schools, the government basically plays the role of a faucet, meaning Congress has directed the agency to allocate large streams of money to state and local organizations,” says Jon Baron, a long-time advocate for a more “evidence-based” approach to government. “Evidence about whether the programs are effective or not effective plays very little role in what gets funded.”

By “evidence,” Baron means mostly the results from a randomized control trial, the gold standard for social science research. These kinds of trials split up a target population into two groups, giving the intervention to one group and not the other “control group” and allowing researchers to compare the difference, discounting factors that aren’t related to the intervention itself.

Since 1990, Baron says there have been 11 instances where large federal programs, like the Job Training Partnership Act and Department of Education’s Upward Bound, have been tested under randomized control trial conditions. In 10 of those, researchers “found small or no positive effects,” he says. “In other words, people not enrolled in the programs did about as well as those that were enrolled.”

But there are interventions that have been proven to have positive results. Baron’s Coalition for Evidence-Based Policy, which he set up in Washington D.C. in 2001, lists many of them at its website, and several of the ideas have been picked up by the PFS movement. For example, Cuyahoga County, which covers the Cleveland area, is adapting a program that elsewhere successfully reintroduced mentally-ill patients back to the community. Its program involves giving homeless mothers intensive therapy known as Critical Time Intervention and aims to address “the core problem” that led their kids being taken into care. The aim is to reunite children with their mothers because, all being well, that’s better for the kids’ development. And it’s better for the county, too. Foster care is one of the most expensive items in its human services budget.

Cuyahoga has identified about 100 families that will take part in its $4.2 million experiment. Half will get the treatment, the rest will act as a control. After watching the families for 12 to 15 months, the county will tally up how the two groups make use of expensive foster care services. If there’s a reduction in the group getting the intervention, it will repay the investors, including the Reinvestment Fund, the George Gund Foundation, the Cleveland Foundation, the Sisters of Charity Foundation of Cleveland, and the Nonprofit Finance Fund. Their potential return amounts to $75 for each day the families don’t use the normal service. Case Western University will have final say over whether the project has met its targets.

“The core of our exploration is a belief our services can be effective if they are appropriately funded, but we don’t always have the mechanisms to prove that effectiveness because we don’t scale them to a level they could be provided,” says David Merriman, who helped set the up the PFS for the county. “It has given us the ability to scale the program through full funding from our investors, but also to prove its effectiveness.”

Merriman hopes the project will break down the barriers that normally exist between services for the homeless and services for children in care. By targeting families for special treatment, he thinks the intervention should deal with the families’ problems more holistically across normally disconnected agencies.

Smart procurement

When PFS first started appearing five years ago, it was seen primarily as a financial tool–a way to bring private capital to bear on important social problems. But over time, it’s produced a wider change in how governments think about their services, according to George Overholser, CEO of Third Sector Capital Partners, which is advising on many of the U.S.-based projects.

“The essential source of change is the insertion of rigorous evidence into the way government selects and pays for social programs,” he says. “We’re using careful measurements of whether communities and individuals are better off, as opposed to how much they get paid [for those services].” Overholser sees PFS as a catalyst for governments to start buying services based on data and measurable outcomes, and perhaps not just as part of PFS contracts. One day, they might simply have a line of credit for relatively safe investments, the same way companies might have one.

Governments around the country are so keen to hire Third Sector, the firm has now stopped taking on new business. Overholser, who years ago was part of Capital One’s founding management team, says he’s currently working on 37 projects across 10 issue areas.

“Our expectation is that a subset of the projects we’re looking at will be particularly compelling in their performance, and those will be rapidly replicated around the country. Each time we replicate it, there will be track record to go on, which in turn will allow us to accelerate the pace of growth,” he says.

As yet, none of the open PFS projects have reported full results. The best we have are interim results from the first PFS initiative, a program to reduce reoffending at Peterborough prison, in the east of England. It managed to cut re-convictions to 142 per 100 prisoners compared to 155 re-convictions per 100 prisoners in a national control group. It has reduced reoffending by 8.4% so far, a little under the 10% mark that will trigger payments to the impact investors. New York State’s Rikers Island PFS is due to report its results later this summer.

Overholser argues that we shouldn’t judge PFS simply by whether the programs hit their targets, though. Even if they fail, they will still be worthwhile. “If it doesn’t meet its outcome target, that’s a form of success as well, in the sense that the feedback loop has taught us something,” he says. “It’s like venture capital in that way. No venture capitalist ever made an investment that they thought was unlikely to work. But they know the world is a complex place and, therefore, they don’t always work.”

The promise of PFS, in other words, is not so much that it will solve every problem. But rather that its process will begin to tell us what might work and not. In that way, it’s more like the experimentation that scientists do in the lab, or that business strategists carry out within companies. It’s a method of innovation where failure comes with the territory.

Potential problems

Not everyone thinks PFS is such a wonderful idea. Critics say private funders will skew public services towards narrow outcomes at the expense of wider government responsibilities. They complain about “soft privatization.” People say PFS collaborators will inevitably “cream” off certain participants to ensure success. They say transactions involving Goldman Sachs, which has funded several PFS initiatives, should immediately raise suspicions. And they say PFS contracts are expensive to set up, and that the contracts are bound to produce legal squabbles as participants fight over the final results.

Again, it’s too early to say what results the PFS schemes will produce, for good or bad. It’s definitely true that the contracts have been time-consuming and complex to set up so far. Merriman started working on Cuyahoga’s PFS in 2012 and the process took the best part of two years to complete. “There was a lot of work of involved. But part of that was that there were no county [PFS] projects in America at the time. We can now [help] other communities, so I don’t think they will be forced to spend as much time as we were forced to spend,” he says.

Overholser argues that PFS isn’t privatization because public officials are “always at the table,” and that PFS is unlikely to be used simply to cut public services (as opposed to optimizing them) because anyone wanting to do that will have to bring the rest of the partnership with them (which is difficult without consensus). At the same time, he admits you can’t apply a randomized control trial to every social question. For example, it could be unethical to test a measure on one group and not another if you knew the group not getting something could be harmed because of it. For this reason, we’re probably not going to see PfS for, say, anti-gun crime measures.

One of the big outstanding questions for PFS is how participants think about outcomes that don’t come with dollar signs attached, says Nima Krodel, a director at the Nonprofit Finance Fund, which advises on PFS contracts. “How the government is valuing those outcomes, so it’s not just about the cost savings piece, is one of the big conversations that is happening across the country,” she says.

Ultimately, it’s probably better to think of pay-for-success less as a thing that always produces good or bad, but rather as a tool and a process that could produce many types of outcomes, some good, some less desirable. “Ultimately [PFS] is only as good as the interventions it sponsors,” Baron says. “If they don’t work, then the funding model makes little difference to the outcome probably.”