When the Foundation for Partnership Initiatives in the Niger Delta (PIND) was launched with Chevron’s backing in 2010, it also meant that a time would come to assess whether the team was achieving its goals—primarily, to open doors for broad-based economic opportunity in the conflict-affected Delta.

With its activities grounded in a relatively new approach called market systems development, PIND required a similarly new methodology to measure the complex systemic changes it was encouraging.

The Washington, D.C.-based Initiative for Global Development (IGD) designed an innovative methodology for assessing Nigeria-based PIND and its U.S.-based strategic partner, the Niger Delta Partnership Initiatives Foundation (NDPI).

The year-long evaluation, which was published in July 2016, titled “Pioneering New Operating Models and Measurement Techniques for Private Sector-Led Development,” took a comprehensive and nuanced approach.

“Due to the complexity of dynamic market systems, the variety and interrelatedness of NDPI and PIND’s activities and interventions, and the numerous networks and actors involved, we had to develop a suite of methods and tools,” said Adrienne Gifford, the IGD senior advisor who led the evaluation. “This proved to be the best way to get our arms around such a disparate array of activities that are collectively working toward a unified outcome for the Delta.”

IGD assessed changes in 13 market systems, each crucial to long-term economic and social progress. These systems included key agricultural value chains, business and institutional networks, infrastructure, and finance. Evaluators scored the “maturity” level of each system’s progress toward systemic change and desired end result. These maturity levels are depicted on the scale below. Additionally, IGD calculated “rates of adoption” to estimate the number of individuals who have already adopted or are likely to adopt the market systems innovations introduced by NDPI/PIND.

In April and May 2016, IGD hosted two expert roundtables to obtain a well-rounded set of perspectives on the methodology: the first in Washington, D.C., the second in Abuja, Nigeria. In addition to obtaining feedback about the report and approaches used in the assessment, the roundtables engaged thought leaders in dialogue on private sector-led development, approaches to market systems-based development, and the state of the art of measuring the impact of such initiatives. IGD guided participants through questions centered on: 1) re-thinking development impact, 2) re-orienting evaluations to capture evolving concepts of impact, and 3) re-tooling evaluation methodologies. IGD separately engaged outside peer reviewers.

As a result, IGD produced a summary of seven themes and recommendations that are included in the evaluation and will inform future assessments.

In measuring the NDPI/PIND partnerships with stakeholders throughout the Niger Delta, IGD focused not on the achievements of any single partner but rather on the collective impact created to systems by the whole. For example, evaluators applied the well-regarded Bass Diffusion Model to measure broadly how NDPI/PIND’s new ideas were being adopted, and borrowed from the Lean Six Sigma and other measurement methods popular in the private sector to define clear stages of progress against which to measure NDPI/PIND’s target market systems.

“IGD’s highly tailored and holistic approach necessarily eclipsed traditional metrics such as the number of jobs created or beneficiaries reached,” said Bill Grant, DAI’s global practice leader for market systems development. “This new methodology promises to be a valuable innovation in the field of monitoring and results measurement for tracking progress toward achieving systemic change.”