August 15th, 2022
By: Adam Luecking

What is the Data Troll?

Any social impact journey worth taking is fraught with obstacles and challenges that can seriously hinder progress. While there are no literal monsters on what I call the “Social Sector Hero’s Journey,” there is a tempting figure that can lead organizations down the wrong path if they aren’t careful. This nemesis disguises itself as “good data,” success, or something to celebrate. Job well done; we can go home, everyone! But under the surface, this nemesis is hiding something that can completely derail an organization’s mission. I call this two-faced nemesis the “data troll.” What is its main weapon? Convincing organizations that superficial data analysis is enough.

Over the past 15 years working with funders and grantees, I’ve seen many leaders with good intentions make a simple mistake: taking their data at face value instead of looking deeper into the numbers (this is the data troll at work casting its superficial analysis spell). But, when leaders fail to consider the multivariate nature of larger data sets, it can slow down their social impact journey. Only looking at total data figures allows us to act on faulty — or even dangerous — assumptions.

Defeating the Data Troll With Data Disaggregation

Lucky for everyone, there is a shield that can help social and public sector warriors ward off the data troll and avoid derailing their progress: data disaggregation.

Disaggregating data means taking a single measure — like “Number of program participants” or “Infant survival rate” — and splitting it into multiple measures that reflect particular characteristics or subpopulations. Characteristics are things like race, ethnicity, gender, disability status, and income. Subpopulations are things like regions, counties, and neighborhoods. There are endless ways to disaggregate data. Ultimately, through disaggregation, we are digging deeper into data to unlock insights that accelerate progress and maximize funding dollars.

Without disaggregating data, we can mask serious disparities in our communities. This is the data troll disguising itself with an aggregated data total (e.g. “All people” as opposed to “White people, Black people, Native American people, Latinx people, Asian people, Pacific Islander people, etc.”). A data total (eg. an 85% high school graduation rate) may imply that things are generally good or getting better if you look no further — but in reality, there are groups of people that fall behind others nearly 100 percent of the time. This is true whether you’re looking at indicators of community wellbeing like high school graduation rates or program performance measures. It’s true whether you’re looking at health, education, or financial security. It’s true whether you’re looking at infrastructure, the environment, or access to transportation. Unless you disaggregate data, you can really lose an important element of your community’s story — certain people succeed at much higher rates than others.

Data Disaggregation = Better and More Targeted Strategies

If you disaggregate your data, you unveil what’s really going on. These insights allow you to target services and funding dollars toward those most in need, thus maximizing the impact of your funding.

Think about the high school graduation rate as an example. On the surface, a county’s average graduation rate may be relatively high. But if you started to dig deeper, you might find that each of that county’s 30 high schools has varying degrees of success. Just a few of these schools may be getting worse, even though the average is going up. If you took it a step further, you might find that students of a particular race are graduating at much lower rates than their peers. You would then be able to design more targeted strategies designed to help these students succeed. If you want to accelerate your progress and defeat plateaus in the data, you must keep asking why and digging deeper. You’ll eventually get to the root of the problem and ensure that your programs and strategies fit the bill.

There’s Always a Way to Dig a Little Deeper

If you’re reading this and thinking, we already knew all this and have been disaggregating our data for years! It’s true – many social and public sector organizations have gotten serious about equity and have permanently integrated data disaggregation into their strategies. Whether you’re a disaggregation warrior or just learning the tools of the trade, there’s always room for improvement. For those looking to dig a little deeper, I have two suggestions:

  1. First, consider what categories you’re disaggregating your data by. Are you disaggregating by broad racial categories? Is it possible for you to get more specific? In, Making the Case for Data Disaggregation to Advance a Culture of Health, PolicyLink argues that broad racial categories often do not accurately represent people’s lived experiences. In response, the accompanying report proposes looking at ethnicities and cultures in addition to broad racial categories.
  2. Second, have you considered intersectional comparisons? If you couple race with another categorization, people of color usually fare worse than their peers within the original disaggregated group. Oftentimes, you can’t just look at gender disparities; you also need to examine racial-gender disparities. You can’t just look at ability/disability status; sometimes you need to look at race and ability/disability status. Consider reinforcing your data analysis with an intersectionality lens to bring your strategy to the next level.

If you’re interested in learning more about the “data troll,” strategies for data disaggregation, intersectionality, and achieving equity, check out chapter five of my book Social Sector Hero. Chapter five also includes two detailed case studies (March of Dimes and Whatcom County, Washington) showcasing how real government agencies and foundations have triumphed over the data troll to make measurable, equitable progress on their social impact missions.

Thank you for reading!