The importance of stakeholder engagement in Turning the Curve

Recently, we sat down with Dr. Monica Shah, Director of the Trial Innovation Network at the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health. We got the scoop on how the center is using Results-Based Accountability to strategically improve performance.

According to NCATS, “Translation is the process of turning observations in the laboratory, clinic and community into interventions that improve the health of individuals and the public.” Part of this complex process relies on the use of clinical trials to test the effectiveness and safety of new health interventions (technologies, devices, pharmaceuticals, etc.). And without an efficient way to enroll trial participants…the entire process is stalled.

With about half of US adults experiencing at least one chronic health condition, and the cost of clinical trials ranging from 300 to 600 million dollars…much is at stake in the successful execution of a clinical trial. And successful execution involves decreasing the time it takes to conduct a trial and translate findings into viable health interventions – all components relying on enrolling an adequate number of participants. When it comes to enrollment, Dr. Shah and the Trial Innovation Network show us that serious gains have – and can – be made.

Through the use of the Results-Based Accountability framework, says Dr. Shah, “we were able to change the Network, increase the scientific productivity, increase enrollment, improve data quality, and improve time to trial startup. So, we had a lot of successes instituting RBA.”

So what exactly contributed to this change? Dr. Shah says that “we really instituted a number of interventions, including performance metrics, working as a team, getting stakeholder engagement, using other disruptive approaches, using the tools that we had or the influence of our leaders, to begin to Turn the Curve on enrollment. By having all these measures in place, we were able to increase enrollment, more than quadrupling enrollment in two years.”

According to the NCATS, an important part of improving enrollment is “patient engagement.” Limiting factors to patient engagement include general unawareness of the availability of clinical trials, geographic distance to trial centers, significant travel and time costs, and too narrow eligibility criteria. Part of the improvement strategy has involved streamlining procedures “in areas such as clinical training, institutional review board operations and patient recruitment methods for multisite studies, and evaluation and accountability of investigators.”

When asked about recommendations that might help new Results-Based Accountability practitioners achieve success, Dr., Shah responded, “I think you need to get stakeholder engagement. You really need to get buy-in from your partners…Talking about the measures you want to improve, agreeing upon those things. Collaboratively developing or identifying the positive and the negative factors, and collaboratively developing the strategies is really important. When people feel like they own a project, when they’re a stakeholder in a project, they are much more likely to take on, or to really do the work that’s necessary to get to the results.”

So ask yourself, who have you engaged in your results-based discussions? Sitting down and listing the people, organizations, business, agencies, and other entities involved in the issue you are trying to improve may be a worthy use of an hour or two of your time.

For more insight and advice, watch the video at the top of this post, or read the full interview below:

Who are you?

My name is Dr. Monica Shah. I am the Director of the Trial Innovation Network at the National Center for Advancing Translational Sciences, “NCATS,” at the National Institutes of Health.

How Many years of RBA experience do you have?

So I’ve been using RBA since 2013, so about 3 years.

What successes has your organization had implementing RBA?

So in 2013, I became the program officer for the Heart Failure Network. We were not enrolling robustly, we had high costs, we didn’t have a strategic plan, we weren’t working as a team. And so the challenges seemed formidable. But I worked with RBA, and sort of broke down the challenges, systematically, and through the use of RBA we were able to change the Network, increase the scientific productivity, increase enrollment, improve data quality, and improve time to trial startup. So, we had a lot of successes instituting RBA.

Can you give an example of a “Curve” you’ve successfully “Turned” using RBA?

I work a lot in clinical trials and some of the basic metrics, the basic things we measure enrollment, data quality, the time it takes to start a trial from the minute that the protocol is developed to the time that the first patient is enrolled into a study. And so those are the measurements we used. We essentially developed performance metrics around those key measurements of success for a clinical trial. And as I said earlier, in Heart Failure Network, we had a number of trials that were ongoing where enrollment had really plummeted. In fact and over the entire network, enrollment rates had dropped to record low levels. So that was one of the performance measures that we used. And as I said, we really instituted a number of interventions, including performance metrics, working as a team, getting stakeholder engagement, using other disruptive approaches, using the tools that we had or the influence of our leaders, to begin to Turn the Curve on enrollment. By having all these measures in place, we were able to increase enrollment, more than quadrupling enrollment in two years.

What were the biggest challenges working with RBA?

One of the challenges is sort of to get folks thinking outside the box a little bit, especially in medicine or the sciences. We’re very oriented towards our particular discipline. And I think sometimes, bringing in a new way of thinking is a little disruptive to people. Change is hard. So what I’ve found – a strategy that I use: sometimes I didn’t even mention RBA, I just did it. I just showed them curves and I showed them metrics. I said what are some of the things that are going well here, what are some of the things that are not going well, and how can we develop strategies around that. And without people even realizing it, they were using RBA to make a difference and to Turn the Curve.

Describe how scorecards help achieve measurable results.

Interestingly, when we started out, when I started out in 2013, the Scorecards were blinded, so people didn’t know how each other were doing. And so, you know, essentially performance was – you couldn’t compare your performance to your colleagues. And when we suggested that we un-blind these scorecards, the initial feeling was “absolutely not, that might embarrass somebody, that might not promote collaboration or collegiality.” But actually when we started instituting RBA, started using metrics, started tracking things, people became really comfortable, and it was fine to have Scorecards un-blinded, people saw how other people were doing – it was a powerful motivating factor.

What advice would you give to someone new to RBA?

A piece of advice that I would give is start slow to go fast later on. I think you need to get stakeholder engagement. You really need to get buy-in from your partners. And so I think doing collaborative discussion. Talking about the measures you want to improve, agreeing upon those things. Collaboratively developing or identifying the positive and the negative factors, and collaboratively developing the strategies is really important. When people feel like they own a project, when they’re a stakeholder in a project, they are much more likely to take on, or to really do the work that’s necessary to get to the results. And so I think that’s really important. And of course, engaging people, getting stakeholder involvement, takes a little bit of time. But at the end, you can go so much faster if you get that buy-in upfront.

Why is focusing on results important when building sustainable communities?

So I work for the NIH, which is obviously a federal agency, and we have a limited budget. And with a fixed budget, we need to have the most scientific impact possible. I mean, there’s lives a stake, we’re trying to produce science that makes a difference in public health. So, the more results-oriented we are, the greater outcomes we’ll have, and the greater impact that we’ll have on public health. It’s as simple as that.