Evaluation

More rationality in political
decision-making through
evaluation?
Seminario de Estudios Empíricos sobre la
Utilización de la Evaluación, Mexico City,
23rd-24th May 2016
Prof. Dr. Reinhard Stockmann
Director of the Center for Evaluation,
Saarland University, Germany
Premise
Individuals, including politicians, behave
rationally when they follow the principle [...] of
utility maximization (e.g. cost-minimizing) due
to certain preferences.
May 2016
Prof. Dr. Reinhard Stockmann
2
Preferences
career
party
interests
issue
voters
May 2016
Prof. Dr. Reinhard Stockmann
3
Presentation structure
1. Where can increases in rationality occur because
of evaluations?
2. Are evaluations actually used for increasing
rationality in decision-making?
3. What can we do to support
a more rational policy?
May 2016
Prof. Dr. Reinhard Stockmann
4
Three evaluation purposes
Evaluation
Implementation
programs / organizations
Policy
Democratic government
leadership (good governance)
Society
Norms and values
 opens up learning
potentials
 offers recommendations
for action
 enables control
 offers decision-making
capacities
 improves level of
information
 generates transparency
 makes measurements
more efficient and
effective
 makes policy more
credible
 creates legitimacy
 allows assessment of
politics
 offers enlightenment
Program control
Political control
Societal control
Manager
Politicians
Citizens
May 2016
Prof. Dr. Reinhard Stockmann
5
Management function
May 2016
Prof. Dr. Reinhard Stockmann
6
Legitimacy or accountability function
Source: https://www.gov.uk/government/publications/lough-erne-accountability-report
May 2016
Prof. Dr. Reinhard Stockmann
7
Enlightening function
May 2016
Prof. Dr. Reinhard Stockmann
8
Conclusion
Dissemination of evaluation functions
widely spread
for program
control
Program control
May 2016
information is
increasingly
provided by
evaluations for
policy making
Political control
Prof. Dr. Reinhard Stockmann
societal use of
evaluations is
seldom
Societal control
9
Aníbal Velásquez,
Minister of Health,
Peru, March 2015
May 2016
Prof. Dr. Reinhard Stockmann
10
US Government Accountability Office
(GAO)
May 2016
Prof. Dr. Reinhard Stockmann
11
Use in the USA
“our past work has shown that it
does not always reach the
interested parties in Congress,
and when it does, the
information may not be timely
or presented in a manner that is
useful for congressional decision
making” (GAO 2012: 2)
May 2016
Prof. Dr. Reinhard Stockmann
12
Use in Switzerland
“evaluation up to now has a rather limited role in
strategic political decision-making”
(Bussmann 2008: 499)
May 2016
Prof. Dr. Reinhard Stockmann
13
Use in Germany
Conclusion economic policy:
“Neither the politicians responsible
nor the affected voters show real
interest in reliable results”
(Bookmann, Buch & Schnitzer 2014: 314)
Conclusion labor market:
“there is no evidence at all that federal government,
state and communities use evaluation results for
evidence-based policy” (Baethge-Kinsky, Bartelheimer &
Wagner 2010)
May 2016
Prof. Dr. Reinhard Stockmann
14
Use in Mexico
“Evaluation use has improved in the government
sector, particularly in the executive federal branch and
in professional civil society organizations. But
evaluation use needs to extend to congress and to the
citizens in order for the evaluation effort to realize its
full potential and purpose.”
(Pérez Yarahuán 2013: 178)
May 2016
Prof. Dr. Reinhard Stockmann
15
Conclusion
May 2016
Prof. Dr. Reinhard Stockmann
16
Evaluation
journal
period of
number theme evaluation use
time
articles
source
by politicians
by citizen body
Evaluation
1995-2003
189
5 (2,6%)
6 (3,2%)
Evaluation
2004-2015
311
31 (10,0%)
12 (3,9%)
own survey
ZfEv
2004-2015
114
8 (7,0%)
8 (7%)
own survey
May 2016
Prof. Dr. Reinhard Stockmann
Pollitt
17
Utilization-Focused Evaluation
Michael Patton
May 2016
Prof. Dr. Reinhard Stockmann
18
Desire and reality
Eleanore Chelimsky
few empirical evidence
(1997: 52):
“Evaluation is so
of evaluation use for
important to any
the political decisionmaking process
democratic society.
It benefits those who
make decisions about policy,
and it benefits the citizens who
have to live with those decisions
once made“
May 2016
Prof. Dr. Reinhard Stockmann
19
GAO recommends:
“performance information must meet Congress’s needs for
completeness, accuracy, validity, timeliness, and ease of
use to be useful for congressional decision making”
(GAO 2012: 5)
May 2016
Prof. Dr. Reinhard Stockmann
20
Proposals, to increase evaluation use
1. Timing of an evaluation
2. Relevance
3. Timeliness
4. Briefness
5. Target group
6. Credibility
7. Advocacy
8. Attention of frame conditions
9. Laws and regulations
10.Participation and generation
of ownership
May 2016
Prof. Dr. Reinhard Stockmann
21
Hope?
May 2016
Prof. Dr. Reinhard Stockmann
22
Thank you for your attention!
Prof. Dr. Reinhard Stockmann
May 2016
Prof. Dr. Reinhard Stockmann
23