To tame wicked public health problems, we need better evaluations

When it comes to America’s most pressing public health concerns, we’re inclined to look for a solution—a prescription, a program, a policy—before anything else. How can we fix or treat the people or communities who are hurting? What can we do?

Public health interventions that lead to more effective and sustained change require not just quality implementation, but also high-quality rigorous evaluation. In short, we shouldn’t just be asking what action we should take; we should also be thinking just as carefully about how we’ll evaluate the impact of that action.

Evaluation training and technical assistance (TTA)—a collaborative and coordinated approach to building capacity—is critical to clients implementing cross-cutting solutions designed to improve chronic disease health outcomes. This blog highlights five key considerations when designing a TTA model for public health professionals.

Demystifying America’s Wicked Problems

Chronic diseases—the leading causes of poor health, death, disability, and rising health care expenditures across the U.S.—testify to the persistent, stubborn, deep-seated and intractable problems that exist in public health. These are often called "wicked problems"; not in the sense that they are evil, but rather because they are so resistant to resolution and so difficult to prevent or treat.

They stem from human behaviors and all of the factors that drive those behaviors are complex in understanding and developing sustainable solutions to address them. In 2014 alone, seven of the top 10 causes of death in the nation were chronic diseases. Just two of them, heart disease and cancer, accounted for almost half of all deaths.

Many of these diseases have multifaceted causes. Say, for example, a person suffering from obesity lives in a food desert where it’s difficult to obtain healthy food. Maybe that person is also being treated for a psychological condition with medication that promotes weight gain. Numerous other determinants—from genetics to socioeconomic status—could also play a role, making it impossible to attribute the disease to a single factor. If that person has more than one chronic condition (as is the case for one in four Americans), the issue becomes even cloudier.

Successfully tackling or managing these issues require complex solutions assessed through rigorous evaluation.

In 2014 alone, seven of the top 10 causes of death in the nation were chronic diseases. Just two of them, heart disease and cancer, accounted for almost half of all deaths.

For public health puzzles of this magnitude, it’s not enough to develop traditional solutions or programs and implement them in communities. These issues need to be regarded as systemic population health issues and demands attention from multiple levels. For example, State health departments and local communities working to address issues like chronic disease are often supported by Federal funders like the Centers for Disease Control and Prevention (CDC). Funders need to know whether or not recommended interventions are actually improving health to make decisions about continued support on a small scale and/or to advocate for policy change on a larger scale.

Connecting the Dots: 5 Considerations For Designing and Delivering TTA to Public Health Professional

Public health practitioners need to demonstrate their contributions to improved health outcomes through evaluation and performance measurement. This requirement translates to a meticulous process of tracking and reporting performance measurement data; conducting local evaluations; using evaluation findings to inform decision making; and disseminating results and success stories to inform the field. Not all grantees are equipped to support such an undertaking, so evaluation TTA is becoming an increasingly important feature of the funding that Federal agencies provide their grantees.

Through our team’s critical TTA expertise, we support clients implementing crosscutting public health solutions. Developing strong, efficient, and productive working relationships among grantees, TTA providers, and funders requires patience and hard work. In our partnerships with stakeholders across federal/non-federal, state, and local sectors, we consider five key factors to ensure grantees demonstrate program accomplishments and promote sustainable strategies for improved health outcomes.

Read on to learn how those considerations should inform your project and to understand the types of questions you should be asking along the way.

1. Context and an understanding of the program evaluation requirements, evaluation skills capacity of the recipients, budget, and performance period are critical. This will determine whether the grantee will function better with a proactive TTA approach (standing meetings, planned trainings) or a responsive approach (ad hoc or as-needed discussions, meetings, and trainings). Also, consider how staff working relationships can help you anticipate changing circumstances; for example can you establish defined, assigned workstreams/partnerships? Or will the team pitch in to triage those events as they arise?
2. Program goals guide the TA model. Achieving agreed-upon outcomes is the ultimate goal of the program. To support this goal, everyone, from TTA team leader providers to entry-level staffers should be on the same page about the definition of success from an evaluative and funding standpoint. What are the program goals? What are the outcomes of interest? What type of evaluation (formative/summative) is being conducted? Being clear about the short-term, intermediate and long-term outcomes and deciding data sources at each level are essential. Understanding program implementation with associated activities fosters alignment of program and evaluation dialogue.
3. Scope and intensity of TA can be basic, moderate, or intensive (Beal and Luster, 2009). There’s a big difference between the delivery of simple, one-time training and a long-term evaluation or data collection project. Does the TTA recipient need help with evaluation design? Logic models? Implementation? Instrument selection? Analysis? Dissemination? The answers to these questions will help determine the scope and intensity of TA activities and deliverables.
4. Communication can make or break a project. TTA can be provided via dissemination of tools, trainings, or ongoing coaching and support. Having the right capabilities to understand the needs of the grantee and meeting those needs are critical to the delivery TTA. Carefully consider how you want to share and receive the information and materials that will help move your project along. In what format should you provide materials, from meeting notes to full reports? Also consider how you want to work with others on the team. Some stages of the project may require more intensive, ongoing coaching, 1-1 communication, or in-person site visits; other stages may be suited to less intensive methods like remote webinars, virtual trainings, or virtual site visits.
5. Qualified and competent staff create and maintain a supportive environment when engaging with grantees. Grantees feel safe to openly share thoughts without fear of being embarrassed. Additionally, staff should be able to present information accurately and clearly, tactfully clarifying misunderstandings if they occur without discouragement. Make sure you have a clear, documented picture of all of the stakeholders involved, including the number of TA providers, recipients, and special consultants (like statisticians). You should also make an effort to understand their areas of expertise, do they have 30 years of experience in their field or three?, and their availability (are they full-time or on-call?).

Tell us about your experiences with different TTA models and approaches. What kinds of challenges did you face, and what kinds of insights did you pick up? Tell us more on Facebook, Twitter, or LinkedIn.