The state of Utah is taking the problem of homelessness seriously and plans to use data as a major tool for helping to improve the quality of life for people who live unsheltered or in a state of housing instability. In its last session, the Utah Legislature passed HB294, which makes provisions for data-driven accountability in homelessness services.
Things should be getting better for people who experience poverty in Utah, and we should be accountable for results. Practices and interventions that create worse outcomes or do not improve the lives of vulnerable Utahns should not be supported by tax dollars. On these points there should be no debate. Utah has long been recognized as a leader in data-driven government, so we are in an excellent position to leverage data to create positive change. HB294 is a great start, but the state’s approach to implementation will make all the difference in the bill’s effectiveness.
There are two divergent approaches to data-driven results processes: 1) Results-based punishment, which can create a culture of fear, result in perverse incentives, erode capacity and trust, reduce collaboration between providers and is likely to cause additional harm and inevitable failure. 2) Results-based learning, which most often creates a culture of innovation, enhances collaboration and shared knowledge and is likely to find new solutions, bring them to scale and grow community capacity.
Utah should embrace results-based learning and avoid results-based punishment. Both approaches involve gathering data, incentivizing success and connecting funding to results. It all hinges on how you tie data to funding.
Results-based learning acknowledges that incremental, small-scale failures, when used as the basis for adjusting course, are necessary for achieving long-term, large-scale success. The key is to reject failure as an end result and to continually adjust in order to sustainably reduce failure over time. In this approach, data is used formatively to provide information about what is working and what isn’t, and funding is used to incentivize continuous improvement and innovation. Organizations are accountable for learning, which in turn leads to positive outcomes.
In results-based learning, decisions about specific practices and interventions are made by practitioners on the basis of carefully stewarded information about the lived experience of their customers. The state is responsible for providing necessary resources and capacity to ensure that such data for learning is available, and that state-funded organizations demonstrate positive adjustments based on this information.
In contrast, results-based punishment is an approach in which organizations that fail to achieve predetermined outcome targets are defunded by the state. In such a system, innovation, learning and adjustment are de-incentivized because failure to present the “right” results becomes an existential threat that changes organizational thinking away from problem-solving and toward organizational survival. This fosters a fear-based culture in which the appearance of success becomes more important than actual success. Global research on results-based financing demonstrates that such approaches quickly create perverse incentives to manipulate data, cherry-pick problems and clients, and engage only in programs that can easily yield desirable metrics, regardless of whether those metrics reflect the lived reality of program customers.
Results-based punishment reduces learning incentives for highly successful organizations, not just low performers. Organizations that receive funding due to their ability to maintain high outcome levels are unlikely to put their metrics at risk on unproven but innovative changes or new solutions.
In the very worst cases, results-based punishment systems can also result in extreme harm to the people they are meant to serve. For example, results-based funding tied to counts of people living on the street can incentivize mass roundups and incarceration, shelter in inhumane and unsafe conditions, and a reduction in focus on health, safety and long-term housing solutions. These are not the “results” we are looking for.
The choice of a results-based funding approach affects the capacity of our whole system. The reason the issue of homelessness has not been solved in Utah isn’t that the practitioner community isn’t trying hard enough, doesn’t care or is lazy. The problem hasn’t been solved because it is a complex, multisystem, self-reinforcing set of problems that are difficult to solve.
Results-based punishment cuts existing capacity and expert practitioners out of our system, signaling that we do not trust the motivations or experience of practitioners in this space. Most practitioners have been working on the front lines to make gains on this system of problems even as the nature of the problem continues to morph in real time. The punishment approach exacerbates burnout and employee exit without providing for sustainable alternatives.
Organizations that use data for learning in an environment of collaboration and trust are more likely to focus on the most important outcome: the genuine long-term well-being of vulnerable and unsheltered people in our state. Funders should use results-based learning to arm practitioners with information so they can make better decisions. When funding is limited and system success levels are low, funders should prioritize organizations that demonstrate learning, innovation and customer focus.
Eva Witesman, Ph.D. is a professor of public management and a Stewart Grow Research Fellow at Brigham Young University. She served on the Utah State Evidence-Based Workgroup for seven years, has consulted on evidence-based practice for dozens of agencies through the Governor’s Office of Management and Budget, created methodology for a collective accountability index on homelessness for the Utah County Mayor’s office, and has helped hundreds of practitioners to gather and implement evaluation data through the BYU GoodMeasure program. Witesman currently serves as director of the Ballard Center for Social Impact at BYU Marriott.
The opinions expressed here are the sole opinions of the author and do not necessarily reflect the views of her current or former employers, partners or clients.