Monitoring, evaluation, research and learning

Drawing from our multi-sectoral team of experts, ICF designs complex monitoring systems using best practice approaches such as theories of change that help map causal pathways to change. As an industry leader, ICF’s monitoring systems rely on rigorous methodologies for collecting and using data for program design and decision-making and policy influence.

ICF works with development and health agencies, governments, their partners and stakeholders to define objectives they want to achieve, methods to assess these objectives, and activities that will make a difference. Through our monitoring, evaluation, and learning services, we help clients including bi-lateral and multi-lateral donors, foundations, non-profits, and government agencies to determine the efficacy and outcomes of programs dealing with food security, global health, global health security threats, forced labor, education, and climate resilience.

Our project monitoring systems track implementation of project activities across specific outputs; monitor the extent to which the overall objectives of the project have been achieved; and guide collection, analysis, dissemination, and use of M&E data to inform project planning and learning.

If data already exist on a given subject, we leverage them to our clients' benefit. We assess quality, availability and frequency of reporting of existing data to determine how to best leverage data and to identify any gaps. If new information needs to be obtained, we know the questions to ask, the data sources to use, and the right mix of approaches to employ. Working in close collaboration with stakeholders, we identify appropriate metrics to determine if proposed goals will be achieved or whether mid-course corrections are necessary. ICF experts help identify critical assumptions to ensure that project metrics are relevant, accurate, and timely.

For USAID’s Demographic and Health Surveys (DHS) Program, ICF works with both the Roll Back Malaria Monitoring and Evaluation Reference Group, or MERG, and the UNAIDS MERG to review, update, and define global indicators for malaria and HIV. For USAID’s Resources to Advance Low Emission Development Strategies Implementation (RALI) project, ICF developed climate change indicators. For USAID’s Food for Peace baseline studies and post-project evaluations, ICF implemented baseline and endline surveys to monitor performance of outcome and impact indicators in the target areas. Collection, analysis, and reporting of key indicators help the programs and have implications beyond the project in reducing poverty and stunting around the world. For the Centers for Disease Control and Prevention (CDC), ICF supports rapid assessments of Test and Start service delivery and the development of country-specific enhanced monitoring to identify implementation problems early and implement corrective action. ICF has assessed performance of programs and the impact they have in improving the livelihoods of the populations they serve. ICF has conducted complex evaluations for the Global Fund to Fight AIDS, Tuberculosis and Malaria; the United Nations Children’s Fund (UNICEF); the Affordable Medicines Facility malaria (AMFm); and of national malaria control programs in the President’s Malaria Initiative (PMI)-priority countries.

Rigorous impact studies by an independent, experienced partner like ICF are necessary for the confident attribution of observed changes to clients’ interventions. In our extensive international development experience, we have tackled the challenging aspect of experimental design that considers a program’s unique context, changes in key indicators before and after an intervention, and assessment of the program’s implementation process. For CDC, ICF supports technical assistance for the PEPFAR Evaluation Standards of Practice, including technical review of CDC PEPFAR evaluation protocols for evaluations.

ICF works with clients on the critical foundational work required to acquire and clean relevant datasets, develop data models, and develop analytics environments. We provide data science services via statistical modeling, algorithm development, and machine learning approaches to transform data into actionable tools for decision making and storytelling. Using rigorous strategic planning, we help capture, analyze, and share the information that clients need to achieve their missions and demonstrate the value of their health initiatives.

For CDC ICF provides analytic support on use of site improvement through monitoring system (SIMS) data. We also support CDC in the Office of the Global AIDS Coordinator’s Interagency Collaborative for Program Improvement, where ICF staff provide expert knowledge of data streams, analytics, and serve as PEPFAR Global Data Hub specialists to develop relevant single- and multi-data stream analyses.

Our data analytics team provides a variety of data visualization platforms to meet client needs. For CDC Uganda, ICF created a Tableau-based dashboard to visualize and monitor the management and organizational capacity of district health management teams to manage HIV/AIDS care and treatment programs. For USAID’s Demographic and Health Survey Program (DHS) ICF has developed STAT Compiler which allows users to carry out customized analyses of historical DHS data and then visualize it in charts, graphs and thematic maps.

At ICF, we nurture a culture of continuous learning among our staff and within our projects. We use new knowledge, monitoring data and evidence to increase program effectiveness and through technical assistance, support learning and capacity development at local levels. ICF helps put processes in place that generate platforms for stakeholders to convene and collaborate and use knowledge, information, and data effectively. Our data visualization experts create data visualization and other dashboards that facilitate learning and sharing of information.

For the U.S. Agency for International Development (USAID), ICF is a partner on the Measuring Impact project, which aims to improve biodiversity conservation programming through the practice of adaptive management—including using theories of change and systematic learning—and by developing evidence to support decisions in conservation programming and the integration of conservation with other development sectors.

As the development community increasingly embraces technology solutions, ICF is well positioned to help organizations use these tools to build their knowledge, exchange insights, and grow stronger. In Mali, for example, ICF developed a system that lets health facilities report monthly aggregated malaria service statistics to community health districts via a mobile phone application.

Whether it’s in the design, implementation, or analysis phases of a survey, or strengthening a national or local HMIS system, ICF’s goal is to build the technical and organizational capacity of countries at the national, district and sub-district levels. Through participatory processes with local stakeholders leveraging best practices in organization capacity assessment, ICF develops and tailors context-specific tools and processes to measure and strengthen capacity. We have staff co-located in nine provinces in the DRC to strengthen capacity of the provincial level Ministry of Health to generate high-quality data for decision making. In Kenya, ICF developed a process and toolkit to conduct a standard baseline assessment of monitoring and evaluation capacity: the Monitoring and Evaluation Capacity Assessment (MECAT) Toolkit. This Toolkit is designed for use by the ministry of health and national health programs to assess their own capacity and performance, guide actions to strengthen their health information system, and ultimate improve the use of health system data to guide decision making. Through Monitoring and Evaluation and Sustainability Planning of the World Health Organization’s Rapid Access Expansion (RacE) Programme and MEASURE Evaluation, ICF provides training and technical assistance at sub-national levels to strengthen local capacity in monitoring and evaluation and sustainability planning. Capacity strengthening has included developing performance management frameworks, standardized tools and technical assistance to sub-national survey implementation, and data quality assessments to assess the effectiveness of data reporting systems and data quality and integrity.

ICF supports CDC in measuring and evaluating the costs of HIV programs in multiple PEPFAR-supported countries, including The Bahamas, Botswana, Ethiopia, Guyana, Jamaica, Kenya, Mozambique, Nigeria, Tanzania, Swaziland, Uganda, and Vietnam. These studies collect detailed data on the financial, human, and infrastructural resources required to provide HIV treatment and prevention services, including antiretroviral therapy (ART), HIV testing and counseling, linkage to care, and prevention of mother-to-child transmission (PMTCT), and to conduct cost and cost-effectiveness analyses, and modelling exercises to project programmatic costs. ICF assisted the United Nations Scaling Up Nutrition (SUN) countries in setting up systems to cost national plans, tracking resources utilized, and identifying funding gaps. ICF staff participated as expert panelists in global and regional workshops to help countries cost and track nutrition investments and identify unmet funding needs. For the World Bank, ICF developed a conceptual and mathematical model to analyze health and economic impact of laboratory investment which was tested with the data from Kenya and Uganda.