Developing an Impact Framework one question at a time:
1. Mission & Impact Goals: What is the organization seeking to achieve?
The first step is to examine the organization’s Mission statement and its Impact Goals.
2. Activities & Internal Processes: How is the investee organized to achieve its goals and what is it doing to achieve them?
Here we examine the organization’s activities and internal processes, management, governance, policies, business model, main activities, products and services, affordability for target population, distribution channels, supplier integration, stakeholders, human resource practices and incentives. And we check alignment of these internal processes with the mission and impact goals.
3. Output: What is the organization’s actual outreach?
Together with the investees, we select Organizational and Product/service output indicators to track on a quarterly basis. Some are tracked across the portfolio, others are company specific.
Examples are:
- Number of employees
- Value of wages paid
- Number of ATMs installed
- Number of houses built
- Number of solar lamps sold
- Number of credit loans extended
- Percentage of occupancy rate of hospital beds
- Number of schools opened and
- Number of children enrolled
- Number of savings accounts opened
- Number of student loans to low income individuals
4.Impact: How has the client’s life changed due to the intervention?
This is the most difficult question to respond to. Assessing Impact is a challenging endeavor. To offer an accurate, scientific response, we would need to know what would have happened had our investee company not delivered the healthcare treatment, or the loan, or electrified the village. Impact is change adjusted for what would have happened anyway.
We take a pragmatic approach to this problem and, together with our investees, define evidence-based assumptions that allow us to envision the impact of the intervention, yet we cannot respond to the counterfactual (what if) question. The assumptions are based on client surveys, research, interviews and observation. We include them in the impact framework and fine tune these assumptions to reflect new learning.
The assumptions help us answer the questions: How, when and under what conditions does the usage of the product or service contribute to income generation, cost savings and overall improvement in living conditions?
Example: Assessing the impact of solar lamps
- Increase household savings (lower cost from kerosene alternative);
- Increased hours of light and increased quality of light
- More hours of study with possible impact on children’s school achievement;
- More hours of income generating activity (longer hours of business, also for street vendors and cart-based businesses);
- More hours of cooking with light;
- More hours of leisure (less time spent procuring wood, kerosene etc.)
- Health improvement (reduction in indoor pollution);
- Increase safety (reduction in injuries due to kerosene lamps tipping over and burning or refueling burns and less incidence of snake bites);
- Increase security (reduction on petty crime),
- ncreased connectivity (some models have a mobile phone charger, which means connectivity for telecommunications, mobile payments etc.)
- Reduction of greenhouse gas emissions (solar lamps replace combustion of carbon-emitting alternatives)
Some of these impacts are more easily quantifiable than others. For instance, to calculate how the purchase of one solar lamp affects household savings we need to know:
- What energy solution are solar lamps replacing and at what rates? Solar lamps often replace kerosene lamps, but at times also burning wood. The rate depends on the quality of the solar lamp but often is 1:1 going to 1:4 for higher intensity products.
- What are the prices of kerosene lamps? What is the average household expenditure in kerosene per month? How many lamps are bought per household? What is the lifetime of a solar lamp?
In assessing household savings we may reach fairly accurate numbers. Yet, as seen above, this is just one of the impacts; we would need to assess all other benefits to provide a more complete picture. To do this we need to know or define assumptions on: how the extra hours of light are distributed between leisure, study, income generation; what value we attribute to safety, etc. We need more detailed information to construct plausible assumptions.
As we move from Mission to Impact along the logical framework, the more difficult it is to quantify results with accuracy and to be able to claim attribution (i.e. to claim that the result is a direct consequence of our intervention alone). Collaboration with our investee to improve our learning is critical. As an industry, we stand to win from collaborating further with peer investors and development organizations and academia to define evidence-based impact assumptions.