I once had the opportunity to listen-in as another agency told the story behind the baseline for a poorly performing program. Due to under-staffing, the program was unable to achieve one of the primary outcomes; an outcome that would result in participants being better off AND lead to reimbursement for some program costs. The story and the factors identified helped the funders understand that in order to improve performance, the program in question truly needed increased staff resources to deliver services in a way that would ensure participants were better off after participation, and lead to the program being able to generate increased revenue in the future.
When reporting performance, Results-Based Accountability (RBA) practitioners need to include the story behind the baseline as an integral part of any report. In RBA, a “baseline” is a display of data with two parts: a historical part which shows where we’ve been, and a forecast part that shows where we are headed if we stay on our current course. The data are important; they let us understand the quantity (how much did we do?), quality (how well did we do it?), and outcome (is anyone better off?) of any effort. The story behind the baseline, however, provides the context for the reader to fully understand the performance, and it empowers the practitioners to understand the necessary actions to turn the curve on performance.
I call the story behind the baseline an opportunity to talk about “the good, the bad and the ugly”. It is an opportunity to increase the practitioners’ and the readers’ understanding of performance by identifying the forces at work that impact the program or effort that is being reported. The forces can be positive or negative. Consider the following examples of forces that may be part of the stories behind the performance of a human services agency:
Forces can be internal to an agency, like internal communication. For example, co-workers may not understand how to make referrals for a service, resulting in low utilization. There may also be positive or negative staffing impacts – e.g. new staff members may have created great outreach efforts to engage potential clients.
Forces can also be external to an agency, and can include forces as large as the economy. For example, there may be low hiring rates for trainees competing for jobs with experienced workers who have been laid off. Even the weather, which is out of the practitioner’s control, may have an effect – e.g. poor attendance at parenting classes due to multiple snow storms.
External forces can also be as direct as relationships with referral agencies, the need for system training, program accessibility, and modifications to outside schedules or program content.
Telling the Story
A picture is worth a thousand words, and that is one of the reasons the RBA approach includes the use of data charts to show performance over time. This allows the reader to quickly and effectively understand performance history, including past trends, and current performance within the context of trends. This also allows the practitioner to determine whether we are turning the curve on performance and where the performance appears to be headed.
When developing the story behind the baseline, list briefly, and in order of priority, the most important factors that influence the slope of the trend line (the “curve”) for each measure. Include positive and negative influences (contributing and limiting factors), as well as current and anticipated impact. This is not a re-statement of the data, but a discussion about what factors are impacting the data.
It is important not to rehash the data in the story behind the baseline – the data chart has already provided that information. Rather, provide the important contextual information to make sense of the data. Questions that may help to develop the story may include:
- Is the program performing as anticipated?
- What happened?
- Who are the partners in the effort and how do they relate with each other?
- What is the climate in which you are operating?
- What positively impacted performance?
- What negatively impacted performance?
- How does this story inform your understanding of your performance, AND
- How can it inform your efforts to turn the curve?
The Next Chapter of The Story
Understanding the story behind the baseline, in conjunction with performance data, helps the practitioner identify where to focus attention and effort to improve performance. Specifically, based on what is learned through the story, practitioners can identify the actions needed to improve performance, modify the approach to the service or effort to include those actions, implement them, and measure their impact. This can be described in the Turn the Curve Thinking Process known as Actions to Turn the Curve.
When the story identifies the need for additional information or data which would allow for better understanding or improved performance management, it can be added to a Data Development Agenda for ongoing attention.
To see how the Connecticut Department of Children and Families increased the number of children in kinship care, including the story behind the baseline, you can follow this link: https://app.resultsscorecard.com/PerfMeasure/Embed/164969
For information on how to create a baseline for your performance measures, check out this article
About the author:
Anne McIntyre-Lahner is the Director of Performance Management for Connecticut’s Department of Children and Families (DCF), the state’s child welfare agency responsible for child protection, behavioral health, juvenile justice, and prevention services. Throughout her career, Anne has focused on systems change by developing and overseeing accountability practices for providers, programs, and service systems, and leading strategic planning and performance management work. Anne is also the author of a new book chronicling DCF’s journey to accountability: Stop Spinning Your Wheels: Using Results-Based Accountability to Steer Your Agency to Success. Anne’s share of any profits from the book will be donated to nonprofit organizations that serve children, and families. Anne is a speaker at the upcoming Measurable Impact 2017 conference in San Antonio, Texas November 8th-10th.
Leave A Comment