Zooming out: How funders can combine data across multiple program sites
Four steps for funders to maximize their ability to aggregate data from multi-site initiatives.
Many large-scale public health, social, and education initiatives are implemented at numerous program sites (e.g., schools, clinics, cities, states) across the nation. Funders often want to combine data from multiple sites to demonstrate the full impact of an initiative to their stakeholders. Even so, variations in the indicators, measures, or data collection methods used in different locations can make it difficult—if not impossible—to aggregate data across sites and report common outcomes. As a result, funders are left to interpret findings on a site-by-site basis with limited understanding of the wider impact of the initiative.
Over years of working with numerous multi-site initiatives, ICF has identified the ideal conditions and processes that allow data collected from different sites to be aggregated into a single analysis. We distilled these lessons into four key areas with specific recommendations that funders can use to maximize their ability to combine data from multiple sites.
Clear guidance is welcomed by grantees. Sometimes funders worry about being overly prescriptive or heavy-handed when defining evaluation requirements. In our experience, grantees prefer clear evaluation guidance. They want their data to contribute to a larger purpose. Also, when funders do require indicators to be measured in specific ways, this should never preclude the pursuit of other site-specific evaluation activities to address local stakeholders’ needs. Funders don’t have to choose between a set of common indicators versus local indicators—you can pursue both within the same initiative.
Planning ahead is the key. Beyond clearly articulating the aims and targets of an initiative, funders should plan in advance to measure common metrics of success. It’s possible to collect and combine data from different program sites and grantees into a single dataset that informs the full impact of the initiative. If you are in the early stages of planning a large-scale initiative with multiple sites, this is the time to consider the metrics you want to be measured across all locations. Take the time to define the indicators that align with the aims of the initiative, select common instruments and methods, and set up a data system that harmonizes data across sites. You will be on your way to having a combined dataset that allows you to demonstrate the wider impact of your initiative.
