Chapter 2.8 – Where do we get the data for Indicators? How do we get better data?

Let’s skip number 1.Convince/pressure the holders of data to do special runs for your area: Very often the problem is not that the data does not exist, but that it is part of a large computer system that has not been programmed to produce the data you need. This is particularly true of local data. Most state data systems include information which can locate client records down to the zip code, but it has never been a priority of the agency to produce local data.

In some cases, an enlightened state agency will act to produce this data because it is the right thing to do. There are good examples of this in Vermont and Missouri. In Vermont, the state Agency for Human Services and the State Department of Education worked together to produce comparable data at the school district level. This required significant effort on the part of AHS because its systems counted things only at the regional level. But AHS saw the power of supporting local outcome based planning and the use of data was critical to this work. The department has published since 1996 report cards on the well-being of children and families for all 59 school districts in the state. These report cards include both human services and education data.

But sometimes it takes some form of pursuasion or outright pressure to get data from these systems. The starting point is understanding what to ask for, how to argue for it, and how to use the political process if reason fails.

What to ask for: The wrong thing to do is to ask for everything you have broken out for my area. It is important that you have thought about what you need and learned as much as you can about what’s in the systems. You can figure out what you need in three ways.

First you can do the work of choosing primary and secondary indicators for your results, and a data development agenda. This tells you what you think are the most important measures of child and family well-being; and
Second you can look at what other states have produced for their report cards or to support local planning efforts. While computer systems vary from state to state, the content of these systems is often very much the same. What one state has done, another state can likely do; and
Third you can look at the list of data elements in the agency data systems to see what’s there and if the data you want is part of the data base. This is not as hard as it sounds. All computer systems have descriptions of the data elements. These should be available under the freedom of information act if the agency otherwise refuses. Better, it is likely that your local department which administers or delivers services for a state agency has a very good idea of what data exists in that system.

How to argue for it: There are three principle arguments you can make in trying to convince an agency holder of the data to do something special to meet your data needs.

• Two way street: We provide the data to you. You should make it available to us. The truth is that most data in state systems is provide by local public and private service providers. But that data is then used to meet state needs but not local needs. The flow of data and information should be a two way street. “If we provide it to you, then you should help us access it.”
• It’s in your self interest. Giving us the data will help you meet your goals: There is a tremendous power in state local partnerships to produce improved results for children and families. State agencies that are smart will want to tap into that power for self interest reasons alone. For example, a state local partnership for stable families, might use foster care entry rate data as an indicator. A local effort to reduce foster care entry, while keeping kids safe, has a direct financial benefit to the state child welfare agency.
• It doesn’t cost much. No matter what they say, it doesn’t cost much to do this. It is a matter of priorities, not money. Most agencies have computer programmers on staff or under contract, and the agency has choices about how their time is used. It is rare that any new cash outlay will be required to produce this data. In most cases it will take one programmer, half to full time for 6 months to a year to do what you ask

How to use the political process if reason fails: You probably already know how to do this. But the obvious levers are the elected officials for your jurisdiction and the media. Use them to push the agencies holding the data. One restraint is recommended. It is easy to vilify the recalcitrant agencies. But it is better if you can hold back from doing this. You will ultimately need these people as partners and you lose more than you gain if the process produces “enemies.” Try hard to make it a win win solution.

Do the work without data: This should be seen as a temporary solution. Ultimately you will want to know how you are doing in concrete terms. The history of this work is to count “trying hard” as success – effort not effect. This is not, and never has been, good enough. But it is possible to do a version of results-based decision making and budgeting without data. This sounds like heresy. But here’s how it could work: Instead of asking the question: “What works to turn the curve?” ask the questions “What works to produce the result?””What works to produce the experience version of the result?” “What would work to impact the indicators on our data development agenda?” It may well be that these questions should be asked as part of the “regular” results process. But they can serve to give the look and feel of an “ends to means” process. And that is the essence of this work. For a more complete answer, and two other techniques to use in this situation, see 2.9 What do we do if we don’t have any good data at all?

Create new data: This is also not always as hard as it sounds. And the reason is that the data you create does not have to meet traditional standards for academic and scientific accuracy. This is heresy. But the truth is that the kind of planning and budgeting processes that we are talking about must ultimately be pragmatic. There is already great pragmatism in the question “What works?”There can be that same pragmatism in choosing indicator data to represent a result. Here are some specific examples:

• Many communities have done surveys of how safe people feel as a counterpart to the crime statistics. It is often true that people feel unsafe even when crime statistics go down. These surveys can be implemented with volunteers. And if you can get the local college or university to help with the design so much the better. Other community surveys are possible if the data is important enough.
• Go out and count: Sometimes it’s a simple as a count of the number of vacant and abandoned houses in the community. So go count them every few months or so. Put these counts on a graph on the wall. Use this as a baseline to drive the results-based thinking process (What works to turn this curve?) Use it to engage partners and keep the process disciplined and on track.
• Be creative about how and what to count: If you are concerned about child safety, then ask at every meeting of the collaborative “How many people have seen a child riding a bike without a bike helmet since the last meeting?” Put this data on a chart on the wall. It can be that easy. A favorite story here:

Senator Bernard Fowler of the Maryland Legislature grew up along the Chesapeake Bay and would catch crabs in the summer by wading out into the bay. As a boy he could still see his feet when the water was up to his chin. As he grew older the bay water became murkier, and he became a strong advocate for water quality. He decided that each year he would wade into the bay where he grew up until he lost sight of his sneakers. He recorded the distance as the Bernie Fowler Sneaker Test of water quality. At first this was considered strange behavior. But after a few years it became a media event with the Secretary of the Environment at his side. Create data, created attention, created will.