“How To”: Develop a Theory of Change (TOC)

Transcription

“How To”: Develop a Theory of Change (TOC)
Evaluation Capacity Development (ECD) project – Uganda
“How To”: Develop a Theory of Change (TOC)
By Olga Nakato Mugerwa
Uganda Management Institute (UMI)
1. Introduction
This guidance note expounds on the Theory of Change (TOC) as used in development and social
programming for evaluation practitioners. It is a methodological approach to understanding the theory of Page | 1
any initiative, and is pivotal in informing the evaluation’s purpose and focus, arriving at the key questions
that it will address as well as in helping to derive the selection methods.
1.1. What is a “Theory of Change”?
A THEORY OF CHANGE (TOC) is a description of how an intervention is believed to deliver the desired
results. It is a representation of how and why a complex change process will succeed under specific
circumstances.
TOCs embody beliefs about what is needed by the target population and what strategies will enable
them to meet those needs. They establish a context for considering the connection (causation) between
a system’s mission, strategies, and actual outcomes, while creating links between who is being served,
the strategies or activities that are being implemented, the assumptions, and the desired outcomes. The
TOC provides the following:
i.
ii.
iii.
A big picture analysis of how change happens in relation to a specific thematic area;
An articulation of an organisation or program pathway in relation to how change will occur;
An impact assessment framework which is designed to test both the pathway and the
assumptions made about how change happens.
Because of these components, TOCs are usually presented in the form of a diagram and these can take
any format as long as they aid in the process of uncovering and developing the assumptions within the
TOC. Understanding the TOC of any program or project is essential for carrying out any evaluation,
given the cause and effect focus of an evaluation research.
1.2. Why develop a Theory of Change?
Similar to the definition of TOC, there is a variety of purposes for using a TOC. Nonetheless, it is a given
reality that in our everyday lives we hold beliefs (theories) that guide our analysis of problems and the
way in which we respond to them. Having these theories of change therefore helps to clarify problems
and provides a framework on how to address them. TOCs help us to plan development interventions, as
well as aiding us when it comes to evaluating these interventions. Specifically:
i.
ii.
iii.
iv.
TOCs help move stakeholders from being passive collectors and reporters of information to
active users of information for system planning and service delivery;
TOCs help system and program staff better understand the kind of evaluation information
they need in order to make day-to-day decisions;
TOCs help the evaluator develop research questions that focus measurement on changes
that can occur given the particular strategies that are operative at the system, program, and
client level;
Because they facilitate understanding the link between strategies and the achievement of
outcomes, TOCs facilitate the integration of data from broader evaluation and accreditation
requirements into local evaluation efforts.
TOCs are best used when one desires to:
i.
ii.
iii.
Design a complex initiative and have a rigorous plan for success;
Evaluate appropriate outcomes at the right time and in the right sequence: this may involve in
some instances reconstructing a TOC for an intervention to better understand/assess
outcomes and impact;
Explain why an initiative worked or did not work, and what exactly went wrong.
Evaluation Capacity Development (ECD) project – Uganda
2. Theory of Change and Logic Models
Logic models focus on aligning the component parts of a program into a hierarchy of clearly-specified
goals, outcomes/objectives, outputs/results, inputs/activities, (usually) together with a set of measurable
indicators to demonstrate progress.
For instance, a results chain is a logical linkage of a transformative process that describes how inputs
are used to carry out activities, these activities will produce outputs which in turn will produce outcomes Page | 2
and finally how these outcomes will produce impact.
Inputs
Activities
• Financial, Human
& other Resources
Mobilised to
support activities
Budgets,
Staffing +
other
available
Resources
Outputs
• Actions taken or
work performed to
convert inputs into
specific outputs
Outcomes
• Products
resulting from
converting inputs
into tangible
outputs
Series of
activities
undertaken to
produce goods
and services
Implementation (Supply Side)
Goods and services
produced and
delivered, under the
control of the
implementing
agency
Final Outcomes
• use of outputs by
target population
• The final
objective of the
program. Long
term goals
Not fully under
the control of the
implementing
agency
Changes in
Outcomes with
multiple drivers
Results (Demand + Supply)
A TOC, by contrast, is broader. It seeks to establish the “miracle” that happens between the components
of the results chain and logical framework. It lays out an ultimate goal (sometimes called a ‘long-term
outcome’, ‘vision’, or ‘mission’) which is broken down into a causal pathway with preconditions,
intermediate outcomes, and assumptions for each step along the way. It is usually presented as a flow
chart.
Secondly, Logic Models require that program components are identified so that it is possible to observe
at a glance if outcomes are out of sync with inputs and activities. It does not explain why activities are
expected to produce outcomes. The TOC on the other hand requires justification at each step, i.e. asking
why and how at each step. A hypothesis about why something will cause something else needs to be
articulated. This underlies why it is often termed as a causal model.
Thirdly, Logic Models do not always identify indicators, whereas TOCs require identification of indicators.
This is so because it is necessary to understand within the TOC causal pathway, how well a precondition
needs to be met in order to get to the next goal.
In summary:
Logic Models
 Representation
 List of Components
 Descriptive
Theories of Change
 Critical Thinking
 Pathway of Change
 Explanatory
Nonetheless, TOC approaches can and are often used together with Logic Models.
3. Steps in developing a Theory of Change
It is crucial to take into consideration that there is no set or standardized format for deriving or arriving at
a TOC. However there are common elements.
1. Identifying the Goal: at times referred to as the Long-term Outcome or Vision. This should
be clearly articulated in a specific statement on what goal the organisation/group is working
on and as to how success will be identified.
Evaluation Capacity Development (ECD) project – Uganda
2. Develop a Pathway of Change1: TOC uses backwards mapping requiring planners to think
in backwards steps from the long-term goal to the intermediate outcomes and then early-term
changes that would be required to cause the desired change. This creates a set of connected
outcomes known as a “pathway of change”. A “pathway of change” graphically represents
the change process as it is understood by the initiative planners and is the skeleton around
which the other elements of the theory are developed.
Note that during the backwards mapping process the intermediate outcomes that link the
early and long-term steps should be as strong and robust as possible, and ideally grounded
in a body of best practice.
3. Define interventions: Identify the activities and outputs that will lead to each identified
outcome, it is prudent to select which outcomes are plausible and within the means of the
project to deliver upon.
4. Articulate assumptions: In creating the pathway of change one is required to articulate as
many assumptions2 about the change process as plausible so that they can be examined.
There are typically three important types of assumptions to consider:
a) Assertions about the connections between long term, intermediate and early
outcomes on the causal map;
b) Substantiation for the claim that all of the important preconditions for success have
been identified, and;
c) Justifications supporting the links between program activities and the outcomes they
are expected to produce.
A fourth type of assumption which outlines the contextual or environmental factors that
will support or hinder progress toward the realization of outcomes in the pathway of
change is often an additional important factor in illustrating the complete TOC.
The principle for a TOC is to include an explanation of how the intervention’s activities contribute to the
results – not simply a list of activities followed by the results, with no explanation of how these are linked
apart from the arrow. The TOC should articulate the theories and assumptions which underpin the
anticipated change process. The TOC should be presented both in a diagram to give a visual summary
and also a narrative text which gives more detail on each of the results, causal links and assumptions.
3.1 Pitfalls to be cautious about while developing a TOC
1. Confusing accountability with hope. A TOC must clarify what results must be delivered to be
successful. Defining results in this way will force your organization to get real about the impact you
are signing up to create, not just what you hope will happen. While dreaming big and setting lofty
goals, such as ending world hunger, can inspire your stakeholders, these are better left for your
mission statement rather than your TOC.
2. Creating a mirror instead of a target. A good TOC does not simply reflect what an organization is
already doing; rather, it articulates what the organization wants to be held accountable for, and works
backward to identify necessary activities, strategies, resources, capabilities, culture, and so on.
3. Failing to take the external context into account. The best TOCs explicitly integrate the
anticipated actions of regulators, the work of peer organizations, expected changes in the economic
climate, and other factors. A deep understanding of the external context will help you create a more
realistic TOC.
1
In some literature referred to as the “Causal Pathway”: which is a step by step, backwards mapping process through which a group
determines all the preconditions necessary to reach an ultimate vision or goal.
2
Assumptions within the TOC formation process refer the necessary conditions for change, or the underlying conditions or resources
that need to exist for the planned change to occur. The identifying and questioning of assumptions is generally deemed central to the
process of articulating a TOC.
Page | 3
Evaluation Capacity Development (ECD) project – Uganda
4. Not confirming the plausibility of your theory. While internal dialogue is a common starting point
for TOC development, the process should not conclude without a concerted effort to verify whether
your “theory” is plausible.
5. Creating a theory that is not measurable. To be able to test, refine, and improve your theory of
change over time, you need to be able to measure its key elements. A common way to
operationalize your TOC is to get specific — articulate the input, output, and outcome indicators the Page | 4
TOC suggests you should track. If you cannot actually gather these indicators at the right
frequencies to learn what is working, you may not be specifying your TOC at a deep enough level.
6. Assuming you have figured it all out. To get the most out of your theory of change, you need to
recognize and explicitly account for the uncertainties that underlie your plan. Learning organizations
carefully specify their assumptions, regularly reflect on whether those assumptions are bearing out,
and consider what new assumptions they might test to further improve impact.
An example of a TOC diagram
Goal
•Assumption
Outcome
Outcome
Outcome
Intermediate
outcomes
Pre
conditions
Assumption
Strategies
Pre
conditions
Assumption
External contextual assumptions
4. Conclusion
TOCs are now being used more and more widely, especially for evaluation research, both as a substitute
and as a complement to more familiar logic models. This guidance note has provided a workable
definition of what a TOC is, provided a sketch on how to arrive at a TOC, expounded on when to use a
TOC as well as explained the difference between Logic Models and TOC.
References and Recommended Readings
− Center for Theory of Change. (n.d.) Toco software. Center for Theory of Change. Available from:
www.theoryofchange.org/toco-software/.
− Anderson, A.A. (2005). The community builder’s approach to Theory of Change. The Aspen
Institute. Available from: www.seachangecop.org/node/215.
− O’Flynn, M. (2012). Theory of change: What’s it all about? Ontrac: The newsletter of intrac.
International NGO Training and Research Centre (INTRAC). Available from:
www.seachangecop. org/node/566
− DFID Working Paper series 38- Broadening The Range Of Designs And Methods For Impact
Evaluations (2012): Elliot Stern, Nicoletta Stame, John Mayne, Kim Forss, Rick Davies, Barbara
Befani.
− Understanding ‘Theory Of Change’ In International Development (2012); A Justice and Security
Research Program (JSRP) Paper Series: By Danielle Stein And Craig Valters.