Why is social impact measurement important to CAF’s Global Alliance


Magazine article

Sixty-five per cent of Russians don’t trust NGOs – they don’t have confidence that their donation will reach the end beneficiary.1 Sadly, this lack of trust in NGOs is endemic in the emerging markets. Likewise, 12 per cent of potential donors in India say that they don’t give because of lack of trust in organisations, with nearly four in ten saying that their contribution won’t make a difference in people’s lives. Indian donors also cited concerns about lack of transparency and the need for NGOs to communicate their impact.2

Charities Aid Foundation (CAF) has been operating internationally for over 20 years. CAF offices in the emerging markets of Brazil, Russia, India and South Africa work with corporate donors to deliver their CSR programmes, with high net worth individuals (including their family foundations) and regular donors. With a strong presence in emerging markets, our strategic objective is to grow philanthropy in these markets, as the potential for donations from the growing middle classes, wealthy donors and companies increases. In order to realise this potential, we need to address this very lack of trust in NGOs, and work with donors to build their understanding of what works to grow philanthropy and civil society and create social value in a fast-developing environment.

In order to address these (and other) challenges, CAF developed an approach to impact measurement for our major donors that, using robust methodology, would demonstrate the impact of the projects they are funding and provide valuable learning to support further work in their chosen field. It was important that the approach chosen was practical to implement in an emerging market setting, and spoke to our donors in a language that they could relate to.3

In selecting an impact measurement approach, we first defined the principles that were important for our global offices. We put society, the environment and the ultimate beneficiaries at the core of our analysis and determined that the approach should be outcomes focused.4, 5 We considered seven important principles to develop and design a social impact measurement strategy. These principles are fundamental but can represent a big challenge, particularly in emerging markets. They focus on:

  1. Involving stakeholders
  2. Articulating how change is created and evaluating this through evidence gathered, recognising positive and negative changes as well as those that are intended and unintended
  3. Valuing the things that matter
  4. Determining what information and evidence must be included in order to give a true and fair picture, such that stakeholders can draw reasonable conclusions about impact
  5. Being very careful not to over-claim impact or attribution
  6. Being transparent
  7. Verifying the result.

Overall, there are a number of tools available and the principles allow us to scale our impact measurement practice so that it is fit for purpose in each context. In particular, we have been applying social return on investment (SROI), a widely used social impact measurement approach, to a number of client programmes.

SROI reflects the principles described above and provides a very robust methodology that produces a return on investment ratio that can be articulated in monetary terms as a return on the initial funds socially invested (i.e. donated or allocated to a particular programme or initiative). The crystallisation of impact into a financial value of x reals, rubles, rands or rupees for every one real, ruble, rand or rupee invested is extremely tangible and this has made SROI very attractive to a wide variety of investors.6

For our clients in emerging markets, the SROI ratio reports social impact in a language that resonates with both corporate donors and private philanthropists, many of whom are first – or second – generation entrepreneurs. Although the methodology provides a much deeper analysis of impact, the ability to communicate this simple, compelling message with decision makers and wider stakeholders who may be remote from the operation of the programme was a key factor in determining our decision to adopt SROI as a methodology for a number of our clients.

However, we also chose SROI because this tool provides more benefits than just demonstrating impact. While SROI is primarily used for this reason (and to articulate and document social value created), many organisations gain other benefits that they don’t expect. For instance, within the SROI process, stakeholder engagement helps to test and refine the investment logic to ensure that it provides an accurate representation of the social value that is created on the ground. This results in a much clearer message about what the organisation (or investor) does and how this creates the impact it intends7 i.e. what works, what doesn’t work and which outcomes are most important to the different stakeholders.

The SROI analysis clearly links activities to consequences, which gives management a better understanding of the cultural and structural approaches that help create the desired impact. This methodology also helps to understand the triggers for change and identify the levers to increase social value creation further. As a result, organisations that have implemented SROI have improved their operations. SROI is a tool that, in comparison to many others, allows the investor to address this issue of attribution, as well as to analyse counterfactual,8 deadweight,9 displacement10 and drop off11, among other issues that need to be considered by social impact reporting.

In order to pursue SROI as a service to our clients, CAF is developing a cohort of SROI - accredited practitioners working in the emerging markets where we operate. They have been successful at delivering social impact reporting using SROI analysis at an international standard of assurance. For example, we have completed an SROI analysis for the Lucia and Pelerson Penido Foundation (FLUPP), a Brazilian nonprofit working to transform the lives of children in Sao Paulo. This SROI analysis was aimed at understanding the impact of the Enhancing Early Childhood (VIM) project on children, their families and educators, to demonstrate the value generated in return for the investment made, and to help future planning and decision making.

The evaluation involved interviews with parents, teachers and other stakeholders to identify the key areas of impact and to develop financial proxies for that impact. Our SROI assessment indicated that for every R$1 of investment in VIM, R$4.0812 of social value was being created. This financial value was assigned by using market values of paid-for services, which achieve similar results to VIM’s activities. For example, teachers reported their increased knowledge and confidence as equivalent in value to attending more formal training programmes.

Some of the benefits obtained by FLUPP after this SROI analysis included:

  • The implementing team learnt about the benefits of developing theories of change for each stakeholder group during future phases of the investment and programme. This will further empower stakeholders by including their voice and choices during the programme design stage and it will improve desired outcomes.
  • Costs, workload and stakeholder engagement will be more effectively planned and managed in future. The VIM programme will now invest further resources in programmatic lines related to the children’s parents, in order to scale and maximise impact.
  • Powerful stories surfaced through interviews with beneficiaries. This has better informed programme design, taking the beneficiaries’ choices into account.
  • FLUPP now better understands how the programme works for each stakeholder and for each different geographical region.
  • FLUPP now has a system to obtain better information to improve their performance: data collection during the SROI analysis led to better informed programme administration and design.
  • FLUPP has strengthened its own data collection and reporting systems.
  • A renewed sense of ownership exists among stakeholders due to the engagement needed to implement the SROI analysis.
  • FLUPP now has clear, data-based evidence that VIM is achieving its aim in having a significant, positive impact on young children’s development.

Overall, CAF’s social impact measurement services have affirmed the relevance of FLUPP and given the foundation a strong case for continuing its valuable work into the future. More importantly, the SROI analysis brought evidence that is now informing strategic planning and current and future investment for the VIM programme.

As we grow our portfolio of SROI reports, we will develop a body of evidence of the real, positive, social change created by NGOs and their donors, building the case to challenge the suspicion of the motivations and impact of the sector in emerging markets and to establish the trust that is the essential foundation for giving of time and money.

1 Russia Giving, Charities Aid Foundation, 2014

2 India Giving, Charities Aid Foundation, 2012

3 Ideally, the results would be of value not only to the specific donor who had supported a project, but would also help to communicate to the wider community of potential donors the social value delivered by NGOs.

4 This is, focusing on social change (outcomes) and not just social performance (outputs). In other words, the focus includes significant and long-lasting changes in the lives of investees, for instance.

5 We also use tools such as Theory of Change, logic models and impact frameworks to identify the issues that matter not only to donors and investors but also, to investees.

6 SROI is the most popular tool amongst corporates, private family foundations and investors in general (even when this tool might not be applicable for all investment types).

7 This can also be motivating to staff, potential funders and/or supporters.

8 The counterfactual and attribution must be measured in order to grasp the ‘net’ change (i.e. the change that can be specifically attributed to the intervention being analysed). The counterfactual is the amount of change that might have occurred anyway, regardless of a particular intervention. It is also called ‘business-as usual’. Measuring attribution is useful for multi-actor interventions. (nef, Simplified guidelines for Social Cost-Benefit Analysis)

9 Deadweight is a measure of the amount of outcome that would have happened even if the activity had not taken place.

10 Displacement is another component of impact and is an assessment of how much of the outcome displaced other outcomes. This does not apply in every SROI analysis but it is important to be aware of the possibility.

11 SROI considers how long the outcomes lasted. In future years, the amount of outcome is likely to be less or, if the same, will be more likely to be influenced by other factors, so attribution to a particular investment is lower. Drop-off is used to account for this and is only calculated for outcomes that last more than one year.

12 R$= Reals. Brazilian currency.

Download this article as a PDF. This article first appeared in Philanthropy Impact Magazine issue 13.