How To Create a Baseline (Trend Line) for an Indicator clearimpact
How To Create a Baseline
(Trend Line) for an Indicator
The Short Answer
1. Baselines have two parts: a historical part which shows where we’ve been, and a forecast part that shows where we are headed if we stay on our current course.
2. The historical part should have as much data as you can get. Aim for three to five years. If you have no historical data, then start the baseline where you are and build the history part over time.
3. Build one or more forecast scenarios using partners knowledgeable about causes and forces at work. Use statistical techniques as a tool not an answer. See the long form and short form techniques below for creating a forecast.
The Full Answer
What Are Baselines?
A baseline is a multi-year display of data with two parts: a historical part which shows where we’ve been, and a forecast part that shows where we are headed if we stay on our current course. Baselines allow us to define success as doing better than the baseline or “turning the curve.” Adding a comparison baseline from a different location or program can also be helpful in creating your baseline.
Baselines Serve Two Purposes
First they allow us to ask and answer the question: AIs this future OK? If the percentage of 3rd graders reading at grade level has been declining for several years, is it OK for it to continue to decline? Look where we’ll be in two years if this continues! Most processes of serious change start with the members of the community saying “This is NOT OK”. We can do better. Baselines with forecasts allow you to have this discussion.
Second, baselines allow us to assess progress in terms of doing better than the baseline. This allows us to count as progress when we have slowed the rate at which things are getting worse, before we fully turn around and go in the right direction. This stands in contrast to the usual definition of success: that things get better right away. This is often unrealistic when trends have been headed in the wrong direction for a long time. It takes time to turn the curve on such a trend line. If we do not use baselines to measure success, we set ourselves up for failure by creating unrealistic expectations of quick fixes.
The box below displays the history part of a sample baseline prepared for the training sessions for Prop 10 commissioners sponsored by the State Commission, the California Endowment and the Foundation Consortium.
Forecasting Is an Art, Not a Science
It is necessary to get the right people around the table who can talk knowledgeably about the forces affecting the baseline and what are likely scenarios for change in the future. It often helps to have a statistical expert as part of the team, but this person should not be allowed to dominate the process. The best forecasting is not about technical statistical analysis. It involves people who know what’s happening on the street and who can create two or three believable scenarios of the likely future. Statistical folks can help this process by analyzing trends and presenting data. But do not relinquish control of the forecasting work to the statisticians. Forecasts should reflect the consensus view of key partners about where we are heading.
Techniques For Creating Baseline Forecasts
Creating a forecast for a baseline. The hard part of baselines is, of course, the future. How good are economists or meteorologists at predicting the stock market or weather? Uncertainty is an unavoidable part of this part of the work.
Shortcut: Consider two simple ways to organize a discussion of forecast: Ask the following questions and at each step ask ‘why?’
Do we think the trend will continue in the same direction?
Will it go in that direction faster, slower or about the same?
Do we think the trend will flatten out? When will it flatten out and at what level?
Do we think the trend will change direction? When? What will happen after?
Systematic: Here is another more structured way to do much the same thing:
Create a table with the months across the top and rows as follows:
Result: Stable Community With Adequate Housing Indicator: Number of vacant houses/properties
A. Begin quarter (3 months)
B. Changes during quarter
1. Increment rate of growth/decline
a. Historical pattern of increase at 1 per quarter
2. Changes due to demographics
a. Older population moving out
at faster rate
3. Changes due to economic conditions
a. Economic conditions get worse over the next year
4. Other changes
C. Forecast Path without action plan
D. Changes due to action plan
1. Fix-up 5 houses per quarter
E. Forecast Path with action plan
The group decides the factors to be considered in section B and the estimated value of each. So for example, a chart on forecasting vacant housing might look like this:
Story: Robert Reichauer, former director of the OMB described 4 factors that they considered in forecasting the surplus or deficit. (NPR interview). Conceptually their technique is very close to the process described above. The four factors were:
Performance of the economy (assume average for the next 10 years)
Tax policy (assume status quo)
Entitlement policy (assume status quo)
Growth in discretionary spending (assume same rate as inflation)
Free E-Book: The Results-Based Accountability Guide
Download the Results-Based Accountability Guide to learn how the RBA framework can help you develop baselines and use them to make data-driven decisions.
Tell The Story Behind the Baseline
What are the causes, the forces at work? Why does this picture look the way it does? Why are only 67% of our children immunized at age two? Why are so few of our 3rd graders reading at grade level? Why is it getting worse or improving so slowly? This is the epidemiology part of the work. Each part of the story is a pointer to action in the next stage. If one of the reasons 3rd grade reading scores are going down is that parents don’t do a good job helping children build reading skills, then this is a pointer to parent education, support for library reading programs, campaigns to get reading material into the homes of families with young children etc.
Be prepared for different stories and serious debate. As different partners add their perspectives, a rich picture will emerge about what is happening in the community. Remember: When considering the story behind the baselines, make sure, again to involve as many partners from as many different and diverse communities as possible. Those we are trying to help, often know more about the causes and likely successful approaches than the professionals.
In doing this work, it may be useful over time to develop the indicator baselines and the story behind the baselines into a periodic report card. Making this report card part of a larger report card on your overall organization’s strategy.
Another type of story which needs to be told is the cost of bad results. What is the price we pay if we do nothing and the curve gets worse? Such costs show up indirectly in many places and can be difficult to quantify but are rarely minor.