October 20th, 2022
By: Adam Luecking

Today is a special day because it’s all about celebrating the power of metrics to improve decision-making (if you know or have worked with me, you know how much I love numbers). Specifically, I’m talking about World Statistics Day, which occurs every five years. The United Nations initiated World Statistics Day as a global collaborative endeavor to celebrate the importance of statistics in community development. More grandly, it reminds us of the incredible power that metrics hold over every aspect of our lives and societies. Metrics dictate how our governments function, how our cities are planned, how we run our businesses, and how our economies run. In short, I like to say #MeasuresMatter.

This year’s World Statistics Day theme is “Connecting the World With Data We Can Trust.” Not only does this theme speak to my #MeasuresMatter mantra, but it also reinforces the care we must take when deciding what and how to measure. This is because mathematics does not exist in a vacuum. It can be influenced by political, scientific, and economic interests. Statistics can improve our lives for the better…but they can also cause destruction and mayhem when misleading. Lazily constructed sample sizes and surveys with leading questions are just two ways to create misleading statistics that can influence and manipulate people in self-serving ways. I call this Irresponsible Measurement.

On the other hand, Responsible Measurement – asking effective questions, disaggregating data, and understanding context – helps us make better decisions that positively impact the people we serve. With reliable statistics, we can determine which interventions are working, which might work better with some changes, and which are actually draining resources with no measurable results. We are better equipped to appropriately reallocate resources and funding, allowing us to accelerate toward our goals faster. You’ll be ensuring you’re not just an organization of “magnificent intentions” but an organization of magnificent action and impact. Responsible #MeasuresMatter.

For the past 15 years working with public and social sector organizations to improve their performance management, I’ve increasingly understood that Responsible #MeasuresMatter. Reflecting on this year’s Statistics Day theme, the responsible creation of metrics (like community indicators of wellbeing and program performance metrics) is the key to gathering “data we can trust” and, in turn, developing “decisions that we can trust.” Furthermore, when we collaborate with our community residents, leaders, grantees, and partners, we can bring diversity to the data development process, surface assumptions, and improve accuracy.

So, how do we create metrics and statistics responsibly? Here are four effective methods I’ve seen our Clear Impact clients utilize to improve data accuracy and maximize the positive impact of their measurement systems:

1. Ask effective questions

The questions you ask will differ depending on what type of data you’re collecting (e.g., qualitative vs. quantitative), but you should utilize effective questions nonetheless. If you’re collecting qualitative data for grant applications or renewals, focus on open-ended questions that are positive and foster action. If you’re collecting quantitative data to evaluate programs and strategies, Clear Impact developed the following seven strategies for creating effective surveys.

2. Focus on the most important metrics.

Data contains a treasure trove of insights, but not all data is created equal. Some information is worth spending more time on. For social and public sector organizations, the measures that matter most will describe key indicators of community wellbeing or performance measures that answer whether anyone is “better off” due to programs and services. When it comes to measures, less is more. Check out this article I wrote that goes into depth about the “less is more” data mindset.

3. Never analyze data without context.

I’ve said it countless times before, and I’ll say it again. Data is meaningless without context. This goes for all types of statistics, community indicators, or performance measures. When presenting a measure, the provider should always include information about how it was calculated, who calculated it, why it matters, an analysis of the root causes, and any critical conclusions. I know this may seem like more of a fantasy than reality. When was the last time you read a statistic in an article and all that context was included? It may not be how things currently operate, but this level of context ensures our data is trustworthy, statistically significant, and actionable. At Clear Impact, we call the process of contextualizing the data developing the “Story Behind the Curve.”

4. Dig deeper and disaggregate.

I’ve seen many leaders with good intentions make a simple mistake: taking their data at face value instead of looking deeper into the numbers. When leaders fail to consider the multivariate nature of larger data sets, it can slow down their social impact journey. Only looking at total data figures allows us to act on faulty — or even dangerous — assumptions. To avoid these pitfalls, analysts should consider disaggregating every metric for deeper understanding. Disaggregating data means taking a single measure — like “infant survival rate” — and splitting it into multiple measures that reflect particular characteristics or subpopulations (e.g., different racial and ethnic categories). You can learn more about this process here.

5. Use the right systems to manage your data.

Not all data management systems are built the same. If you’re not careful, you could actually achieve the opposite of what you intended when implementing your system — overwhelming amounts of data, data overlap, and data inaccuracies. So what should you consider when looking for a system? Here’s a list to help you navigate the research phase. This list isn’t extensive and may not account for everyone’s unique needs, but it’s a starting point.

  1. Provide qualitative context (Story Behind the Curve) with every measure 
  2. Share your data/ statistics publicly with the click of a button, a link, or a code to put on your website
  3. View data trends over time periods that make sense for you (daily, weekly, monthly, quarterly, annually, etc.) 
  4. Analyze your data in graphs and quickly spot areas of improvement using color bands, trendlines, forecasts, or baseline percentage changes
  5. Disaggregate your data by chosen subpopulations to promote equity
  6. Quickly set it up with minimal training
  7. You can read the full list of recommendations in a free download of my book, Social Sector Hero (Chapter 9).

***

Responsible #MeasuresMatter because they can mean the difference between making the world a better place and further entrenching social problems and inequities. To create a positive impact, we must exercise extreme caution when developing, analyzing, and reporting our metrics to the larger community. With careful planning and discipline, we can work to make sure every single metric that matters is trustworthy.

So, what’s next? It’s time to audit, audit, audit! Are you willing to audit your metrics and data management processes using the five recommendations above? Let us know in the comments below. You can also learn more about these strategies, including real-world examples, in a free download of my new book Social Sector Hero.

Thank you for reading, and Happy #WorldStatisticsDay!