PETS/QSDS Core Guidance

Public Expenditure Tracking Surveys (PETS) and
Quantitative Service Delivery Survey (QSDS)
Guidebook
Bernard Gauthier
and
Zafar Ahmed
January 2012
Table of Contents
Acknowledgements ................................................................................................ 2
Preface .................................................................................................................... 3
Chapter I: Motivation and usefulness of PETS/QSDS ........................................... 5
Chapter II: Main purpose and key findings ......................................................... 14
Chapter III: Process, ownership and policy reforms ........................................... 25
Chapter IV: Step-by-step guidance ...................................................................... 29
Notes
................................................................................................................ 93
Examples ............................................................................................................. 127
Appendix ............................................................................................................. 178
PETS and QSDS indicators .................................................................................. 178
Governance and Service delivery ...................................................................... 205
References .......................................................................................................... 215
1
Acknowledgements
This guidebook was developed as part of the World Bank Poverty Reduction and Economic
Management (PREM) and Human Development network’s Web-based data platform for PETS
(Public Expenditure Tracking Surveys) and QSDS, (Quantitative Service Delivery Survey). The
goal of these methodological guidelines is to facilitate and encourage the use of these
instruments.
This guidebook received strategic guidance from Mrs. Ritva Reinikka (Director), Mr. Kai Kaiser
(Senior Economist) and Mr. Hooman Dabidian (Private Sector Development Specialist). It
benefited from comments and conversations with many colleagues in particular Waly Wane,
Jose Lopez-Calix, Dena Ringold, Margaret Koziol and Gary Reed. It received financial support
from the William and Flora Hewlett Foundation.
2
PREFACE
Public Expenditure Tracking Surveys (PETS) and Quantitative Service Delivery Surveys (QSDS)
have become important tools for analyzing service delivery and public expenditures. Typically,
PETS seek to trace the flows and uses of resources through the various layers of government to
service facilities in order to identify differences between the official and actual allocations and
to determine the extent to which resources reach service providers. QSDS seek to assess
quality and performance in resource usage at the frontline facility level. They have both proved
to be important tools for diagnosing various efficiency, effectiveness and equity problems in
public expenditures, in particular governance and incentive problems, bottlenecks, rent capture
and leakage of public resources.
The objective of this PETS/QSDS Guidebook is to assist practitioners in using these instruments
and explaining to those who are considering undertaking PETS/QSDS why and how to use these
tools.
The PETS/QSDS Guidebook is intended to help improve the quality and comparability of survey
results by providing guidance and recommendations for designing and implementing these
micro-level surveys and providing links to related resources and reports/documents, including
standardized survey instruments and indicators.
Practitioners considering PETS/QSDS might include project or team leaders in development
agencies, donor agencies, research institutions, sector specialists, government officials,
researchers, consultants, civil society organizations, NGOs, and other stakeholders.
The guidebook is structured such that the different users can find relevant information in
various sections depending on their interest and role in designing and implementing these
studies.
Users familiar with the theory behind improving service delivery and who are especially
interested in “how to do” a PETS or QSDS, are invited to go directly to Chapter IV which
presents step-by-step methodological guidelines to prepare and implement at PETS/QSDS.
3
User’s Guide to the Guidebook
This Guidebook is organized as follows.
Guidebook
The Guidebook contains four chapters, notes, examples and appendices:
CHAPTER I:
CHAPTER II:
CHAPTER III:
CHAPTER IV:
NOTES
EXAMPLES
APPENDIX:
MOTIVATION AND USEFULNESS OF PETS/QSDS
MAIN PURPOSE AND KEY FINDINGS
PROCESS, OWNERSHIP AND POLICY REFORMS
STEP-BY-STEP GUIDANCE
PETS/QSDS INDICATORS
GOVERNANCE AND SERVICE DELIVERY
REFERENCES
Notes
In many places in the Guidebook, the symbol  Note # appears. These are short texts that
explain further what is meant in the Guidebook. Often the notes contain references to
methodological material or to other sources for further information.
Examples
The symbols  Example # in the Guidebook text refer to excerpts of texts, tables, or figures
meant to show at least one interesting case of application of the themes contained in the main
text. In addition to using the varied examples, practitioners may find it useful to look at a few
actual PETS/QSDS reports noted in the References.
Appendix
Potential PETS/QSDS indicators are presented in the appendix. These are illustrated for the case
of Chad using the 2004 health sector PETS/QSDS.
References
Short references are found in the various chapters of the Guidebook as well as in the individual
notes and examples. A full bibliography contains the various reports, papers, and articles
related to these topics.
Resources
Various resources are available to complement the guidebook, including generic education and
health survey instruments as well as various other documents such as survey manuals, TOR and
concept notes which could be found on the PETS/QSDS Web Portal.
4
CHAPTER I: MOTIVATION AND USEFULNESS OF PETS/QSDS
1. Introduction
Efficiency, effectiveness and equity in the use of public resources are universal prerequisites for
good governance. However, in developing countries, weaknesses in expenditure systems and
poor quality of service delivery are observed and are frequently associated with a lack of
reliable mechanisms for tracking and monitoring resource allocation and use, and service
delivery performance.
Indeed, in many countries, it is often unclear whether resources in public programs are used in
ways consistent with budget allocations, reach the intended beneficiaries, and allow
achievement of intended results as little information is available on resource flows and use and
quality of services. Improving service delivery and outcomes requires diagnostics and
information that will help identify bottlenecks in the institutional and governance realm that
affect service delivery, and improve the quantity, quality and efficiency in service provision.
In recent years, various new tools and techniques have been developed to strengthen the
relationships of accountability in service delivery by improving the quality of information of
public expenditure and provider performance. The tools include the Public Expenditure
Tracking Survey (PETS), and Quantitative Service Delivery Survey (QSDS) which have been
developed to collect information about public expenditures and service delivery with a view to
improving the quantity and quality of services.
 Note 1: Other micro level governance tools
2. What are PETS and QSDS?
PETS are conceived to trace the flows of resources (financial, salaries, in-kind items) through
the various levels of government bureaucracies down to service providers (e.g., schools, health
clinics) to identify effective allocation of resources and to assess if funds are used as intended.
They collect information at the central level and, on a sample basis, within the public
administration and at the frontline level, to determine how much of the original allocations
ultimately reach service delivery units (such as clinics and schools), a precondition for providing
services and improving outcomes. By identifying differences between official and effective
allocations at different administrative levels and in time, they are useful tools for understanding
malfunctions in service delivery systems such as delays, leakages and capture of funds by
bureaucratic and political actors, corruption, and inequity in the allocation of resources.
Even if resources are available at the service provider level, their capacity to improve outcomes
depends on how efficiently they are used, which is the focus of QSDS. QSDS are provider
surveys that assess quality and performance in resource usage at the frontline facility level,
such as schools, health clinics and hospitals. QSDS collect information on characteristics and
5
activities of service providers and on various agents in the system, on a sample basis, to
examine the quality, efficiency and equity of service delivery on the frontline. They may focus
on activities and services at the provider level, staff incentives and behavior, input use, pricing
and quality, relationships with the demand side, or dissipation of resources. Various types of
providers could be examined: public, private for-profit and private non-profit.
PETS are often combined with QSDS to obtain a more complete picture of the efficiency and
equity of a public allocation system, activities at the provider level, and the various agents
involved in the service delivery process.
These instruments have proved to be important tools for diagnosing various efficiency and
equity problems in public expenditures, in particular governance and incentive problems, rent
capture and leakage of public resources.
3. What are the rationales for undertaking PETS/QSDS?
Several rationales could motivate the undertaking of PETS/QSDS.
■ Service delivery is at the core of government activities and is fundamental for social welfare,
especially for the poor. However, in many developing countries worldwide, basic services -such as education, health and water and sanitation -- are often severely deficient.
Problems include low quality of services, problems of access to services to specific groups
especially the poor, low quality of infrastructure, lack of basic material, poor accountability of
providers, corruption and rent seeking in the supply chain, and population dissatisfaction with
the way services are delivered.
In the context of the Millennium Development Goals (MDGs), many developing country
governments and donors have increased budgetary allocations toward social sectors, especially
health and education, to meet these challenges. However, governments must not only allocate
resources but also spend them well and be accountable to the population for performance.
■ PETS and QSDS can expect to achieve a range of results. PETS can shed significant light on the
actual functioning of public expenditure systems, in particular, evaluating financial and
institutional constraints on improving services in sectors, identifying the rules and mechanisms
in practice for allocating resources within sectors, planning and management capacities of
ministries; delays in disbursements, leakage of resources; equity in allocation; and
accountability mechanisms. By focusing on the operational impact of budgets, PETS studies can
help reveal whether spending from higher levels of government meets its intended budget
allocations within the government administrative system and at the point of frontline service
delivery. Hence, one tangible impact of a PETS process may be to institute greater
transparency as to what frontline facilities are actually entitled to in the public budget and the
effective allocation, and in turn, provide an entry point for citizens and civil society to hold
government to account.
6
■ Citizens, policymakers and donors in developing countries often have limited information on
actual public spending in many programs that directly affect citizens (e.g., education, health,
early childhood development, water and sanitation). Budgetary information is often in a too
aggregated form to provide the relevant and necessary information at the sector and program
level in order to assess the allocation and use of resources. It is thus difficult to measure
results, assess the performance of programs or sectors, and to identify potential inefficiencies
or inequities across regions in service delivery systems. In such contexts, the detailed data
provided by tracking and facility surveys could help in assessing effective allocation of resources
and identifying the binding constraints that impede quality of service delivery.
■ Apart from efficiency related to funds reaching targets, efficiency could also be affected by
delayed delivery of resources and services. Even if the resources reach the schools but not in
time at the beginning of the school year, it diminishes the effectiveness of the resources.
Similarly with salaries: if medical staff or teachers are not paid in time and face important salary
delays, it affects their incentive for attendance and providing quality services.
■ Equity considerations could be a rationale for PETS/QSDS analysis. There could be important
differences within a sector or program in actual resources available at the facility level across
districts, provinces, or types of facilities (public versus private, etc). These differences could be
due to political and historical reasons but also because of geographic factors. It might also have
a political dimension where the favoring of a district or province could be driven by electoral or
ethnic considerations. PETS provide information on these biases and preferences in actual
allocation rules. PETS, by measuring the actual resources available through the supply chain
and at the facility level, could shed light on such inequity problems.
■ In the context of budget support operations especially, donors could be required to ensure
the reliability and adequacy of public administrative systems in managing and allocating donor
funding. PETS/QSDS in such contexts could help to reduce fiduciary risk of donor programs by
providing detailed information to monitor usage of funds and the attainment of programs
objectives.
■ PETS could also be useful for evaluating specific policy reforms designed to improve service
delivery performance. They can be designed to examine the impact of a specific government
program or policy. PETS can also provide baseline data for impact evaluations if properly
designed to be integrated as part of an intervention evaluation.
 Example 1: Some examples of PETS/QSDS motivations
4. What are the ultimate objectives of PETS/QSDS?
The ultimate objective of these diagnostic and analytic tools is to improve the efficiency of
public expenditures and quality of services, and their impact on the wellbeing of the
population. The means by which the studies could lead to change include:
7
(a) Proposing plan of actions to revise budget allocations to improve efficiency,
effectiveness and equity of public expenditure;
(b) Improving budget execution by obtaining information on various problems in budget
execution (capacity, reallocation, etc) at different stages;
(c) Enhancing government systems of recording, reporting and information systems at
various levels in the administrative system toward service providers (for financial and
non financial resources);
(d) Strengthening relationships of accountability between the government, providers and
citizens by improving information on actual resource flows and quality of service
delivery;
(e) Modifying the relationships between the public sector and private and NGO sectors;
and,
(f) Strengthening domestic capacity, inside and outside government, to facilitate
monitoring and evaluation activities
5. Complementarities and value-added
PETS and QSDS complement other governance evaluation techniques. They have been
implemented in various contexts, often in the framework of Poverty Reduction Strategy Papers
(PRSP) or Public Expenditure Reviews (PER) to help improve the transparency and
accountability of public resources by collecting micro level information on the use of public
expenditures. 1 PETS have also frequently been used to examine the quality and targeting of
public expenditures in the context of Poverty Reduction Strategy Paper (PRSP) exercises and
Heavily Indebted Poor Country (HIPC) initiatives (Gurkan et al, 2009).
By focusing on quantitative information based on record review, PETS/QSDS mark a departure
from other assessment tools which are often based on official central government data or on
surveys of perceptions. There are considerable synergies and complementarities to be gained
by undertaking PETS in conjunction with other Public Financial Management (PFM) activities, in
particular Public Expenditure Reviews (PER) and sector reviews.
 Note 2: Public Financial Management Economic and Sector Work products
See, for instance: Picazo and Zhao (2009) “Results of Public Expenditure Tracking Component of the PETS/QSDS
2005-06” in ”Zambia Health Sector Public Expenditure Review”; World Bank (2008b) “Niger Public Expenditure
Tracking Survey: Education and Health”; and World Bank (2008c) Tajikistan: Second Programmatic Public
Expenditure Review (In Four Volumes) Volume IV: Public Expenditure Tracking Survey (PETS) Health Sector.
1
8
Through their focus on problems with the flow of resources toward sub-administrative levels
down to service providers and on the use of resources and incentives at service delivery level,
PETS and QSDS could complement PERs and other central government focused tools.
 Note 3: PETS as one instrument within PFM toolkit
Figure 1:Complementarities of PER, PETS and QSDS in the Service Delivery Chain
Policy
Framework
Budget
Allocations
Budget
Execution
Supply chain
process
PER
PETS
Intermediate and
Final
Outputs
Sector
Outcomes
Service delivery
process
QSDS
Source: Based on DFiD, 2008.
Indeed, sectoral PERs, which analyze patterns of public expenditure, budget execution,
financing and the sustainability of government budgetary allocations in a sector, could be
enhanced by the collection of survey data of expenditure flows in the administrative system
and of facilities and clients to identify bottlenecks and inefficiencies in the service delivery
system.
As underlined by Amin and Chaudhury (2008), a PER can benefit by linking public expenditure
and service delivery, and can hardly be complete using government data alone. It integrates
micro-level data on the use and transfer of funds, and on frontline service providers, clients and
households to form a more complete picture of effective allocation of resources and service
delivery performance. By collecting data on flows and usage of resources through the public
administration structure down to the frontline level, PETS and QSDS can serve to verify the
effective use of the budget, the performance of the administrative systems and the quality of
services.
They allow users to move from official to effective resource allocation figures based on hard
evidence at the decentralized level and fill an information gap on the supply side of service
delivery. They also often allow the linking of information on the demand and supply sides of
education and health services to guide policy reform.
9
A World Bank Independent Evaluation Group (IEG) review notes that PETS have proven to be a
powerful addition to the Public Financial Management toolkit (World Bank, 2008a, p.50). They
have been helpful in identifying problems with expenditure and financial management,
including corruption. The development of this tool by the Bank has also helped to improve
transparency and accountability.
Furthermore, in contexts where the budget support approach has become the preferred form
of donor assistance, understanding and verifying the allocation of resources at the
decentralized level is important to reduce fiduciary risks. Also, the information provided by
PETS/QSDS is potentially even more useful given that budget support makes conditional
disbursements of aid tranches linked to certain indicators; the non-attainment of the triggered
indicators because of problems in budgetary processes or other causes could have negative
effects on budgets and human development.
PETS can be useful instruments in middle income or high income countries too. Even if a
country has very good auditing capacity, it may want to look at resource flows and identify
effective use of resources and measure results.
Interestingly, the implementation of a PETS has itself become part of a benchmark for achieving
basic performance scores within the Public Expenditure and Financial Accountability (PEFA)
indicator concerned with frontline service delivery financing and accountability systems.
 Note 4. PETS and PEFA
PETS and QSDS could also be seen as complementary to national government Management
Information Systems (MIS). MIS typically collect, on a periodical basis, a large quantity of
information on various components at the different administrative and service provision levels
within ministries. However, several limitations are associated with routine MIS related to
incomplete coverage, poor quality of information and potential misreporting problems.
Furthermore, the reliability of national administrative records is often in doubt given that most
MIS do not have dependable audit systems. 2
PETS in this context are valuable as they specifically recognize the incentives of agents to
misreport resource reception and usage. They tend to deal with this issue by collecting data as
close as possible from the source and by using records kept by the providers for their own use.
National government administrative data usually do not capture information on the entire universe of service
providers (e.g., schools, clinics) but generally only those that are administered by the state or which receive public
funding, and hence not those administered by the private sector. In addition to incomplete coverage, information
could be of poor quality due to missing data for many public facilities. More importantly, problems of accuracy
arise with internal MIS due to important incentive for misreporting by facilities. Indeed, reporting accurate
information could expose inefficiencies in the system and in the performance of the individuals who are
themselves responsible for providing the information (see Amin and Chaudhury, 2008 in Amin, Das and Goldstein,
eds, 2008).
2
10
Furthermore, information is triangulated using data collected from various administrative levels
or providers (Reinikka and Svensson, 2004).
Still, it is important that PETS/QSDS, as mainly ad-hoc tools, should not weaken governments’
own internal reporting and external verification systems. Instead, the goal of sample-based
surveys should be to complement internal reporting systems on a series of other dimensions
and ownerships (i.e., non-public providers). They should serve in particular to verify the MIS
self-reporting data, provide feedback, and seek to improve and strengthen the routine internal
system over time. They should also provide information and feedback to strengthen the
country’s external verification system (i.e., General Controller office).
However, PETS are not audits because they cannot reconcile the use of funds and do not try to
find missing resources or identify the persons responsible. Care must hence be taken that PETS
are not perceived by the government as a punitive or audit type instrument, or by one agency
(e.g., finance) or government level in relation to another, but as a contribution to broader
reform efforts.
Finally, as another precaution, capacity constraints on the government side should be
considered in undertaking multiple studies and surveys simultaneously, particularly if they
stretch thin human resources. Capacity constraints on the side of the donor and/or
implementing agency (e.g., World Bank) should also be considered.
6. What is the scope of PETS/QSDS
The scope of PETS and QSDS can vary with the study’s objectives and characteristics of the
sector under study.
The focus of the PETS analysis could be placed on broad issues, such as assessing the
performance of the overall sector budget, or cover a broad range of resource flows (e.g.,
recurrent expenditures in a sector, salaries, investments). Alternatively, it could focus on
monitoring only a few specific resource flows (e.g., capitation grants), in-kind items (e.g., books)
or specific programs. The focus could also be specifically on domestic public resources or could
include donor funding or other private resources such as user fees.
The focus could also be on specific levels, for instance the primary education or health sector or
on higher levels. In Chad, for instance, the PETS/QSDS in the health sector in 2004 tracked
resources down to all level of providers (health clinics, local hospitals and tertiary hospitals) and
examined public, private and non profit providers.
Similarly, the focus of the QSDS analysis could be placed on broad issues such as service quality
or performance, or focused instead on narrower issues such as staff incentives and behavior or
absenteeism, or availability of books, medication or staff. In addition, various types of
providers could be examined -- public, private for-profit and private non-profit -- to identify
difference in performance across ownership types, or the focus could be placed only on public
11
sector services, for example (See Gauthier (2006) for a discussion of PETS-QSDS focus in SubSaharan Africa).
 Example 2: What motivates religious not-for-profit health care providers in Uganda?
7. What short term and long term goals could be pursued with PETS/QSDS?
PETS and QSDS could allow the pursuit of short and long term public expenditure reform goals.
■ In a short run perspective, by providing an evaluation of inefficiencies occurring in the
procurement and distribution channels of various resources and provision of services, PETS
facilitates the identification of measures aimed at improving the efficiency and equity in public
expenditure allocation and public resource management.
 Note 5: Stand alone PETS?
■ In a short to medium term perspective, PETS could be used as part of a monitoring and
follow-up mechanism, and could be viewed as part of a repeated process. In particular, PETS
and QSDS could be part of a larger strategy that seeks to improve equity and efficiency of public
management.

For instance, an initial PETS could be used as a baseline to measure the allocation of
resources and performance in service delivery and for paving the way for more
comprehensive follow-up surveys.

Hence, once mechanisms leading to shortcomings are understood and improved upon
through reforms and interventions, subsequent PETS could focus on other elements of
service delivery performance, for instance on service quality and population outcomes,
as well as users’ satisfaction (World Bank, 2008).

Repeated PETS could allow benchmarking of progress in implementation of reforms
over time. Indeed, PETS/QSDS could serve as a baseline providing micro-data for impact
evaluations. They can be designed to examine the impact of a specific government
program or reform on service delivery in a specific sector.
 Example 3: Repeated PETS
■ In a long term perspective, PETS could be part of a process aimed at improvements in public
expenditure efficiency and equity by focusing on capacity building.
12
8. What is the PETS/QSDS approach?
While the PETS approach is relatively simple --consisting of identifying resource flows and
allocation mechanisms (funds, personnel and materials) and measuring resources through
various government agencies, administrative levels and frontline facilities -- in practice, a PETS
is relatively complex to implement (World Bank, 2008b).
The PETS methodology consists of measuring the amount of in-and-out- resource flows and
delays between hierarchical levels or at each of the consecutive nodes of the resource
distribution channels through sample based quantitative data collection instruments.
Resources are then compared to assess leakage of resources, delays and other inefficiencies
and equity issues in the resource allocation system.
Furthermore, PETS could be complex to implement due to the intricacies of financial
management systems, the existence of official but also effective allocation mechanisms, and
the large number of financial transactions and material flows. Also, the generally low quality
and variability in the availability of records within decentralized public administration levels and
the large number of agents and services and administrative levels that could be involved in the
allocation and usage of resources render tracking exercises complex.
In contrast, QSDS takes the facility (e.g., school, health center) as the unit of analysis. It
involves collecting quantitative and qualitative data at the level of the school or health
establishment (public or private, for profit and non profit). Information is collected on various
elements related to school and health center operations. Without underestimating the
importance of qualitative information, specific emphasis is placed on quantitative information.
These include information on sources of financing (public, private, NGO, donors), qualification
and remuneration of personnel, infrastructure and material, incentives and supervision
mechanisms. It could also be complemented by a user or household survey.
The range of instruments necessary to implement a PETS/QSDS consists of a series of
questionnaires addressed to the different actors on the supply side of service delivery
(including at the level of the service providers, central administration, regional and district
levels, etc.) and sometimes on the demand side (e.g., students, patients).
Various methodological choices have to be made when designing and implementing these
survey tools to increase the capacity to collect reliable information. These issues are discussed
in step-by-step guidelines (Chapter IV). Before discussing methodological considerations,
chapter II presents some of the main findings of these studies. Chapter III discusses potential
recommendations and reforms. Potential PETS/QSDS indicators are presented in the appendix.
13
CHAPTER II: MAIN PURPOSE AND KEY FINDINGS
1. How many PETS/QSDS have been undertaken? Where?
Since the mid-1990s, more than 50 PETS/QSDS have been conducted in about three dozen
countries, covering more than 70 sectors. A large majority of these surveys have been
conducted in Africa, which currently accounts for 66 percent of the total number of studies,
PETS/QSDS (Figure 2).
In terms of sectors, initially, most PETS/QSDS were conducted in the health and education
sectors, but over the last decade their use has broadened to other sectors, including water and
other infrastructure sectors, agriculture, justice, early childhood programs, some social
protection programs and cross-cutting sectors involving more than one ministry (Figure 3).
Figure 2: PETS/QSDS by Regions
(1996-2009)
Figure 3: PETS/QSDS by Sectors
(1996-2009)
Water
5%
Justice
3%
Others
7%
Health
42%
Education
43%
Source: World Bank PETS/QSDS Web Platform
Since the first PETS in 1996 and the first QSDS in 2000, both carried out in Uganda, there have
been constant evolution and innovations in these survey instruments by the World Bank and
other users.
Initially, the focus was on tracking resource flows but other themes were gradually added: exit
polls, testing of students, measuring health center quality (e.g., vignettes), costing, etc. Various
extensions of PETS and linkages with QSDS were developed to understand better frontline
service providers’ incentives, constraints and efficiency.
14
2. What were the motivations of the first PETS and QSDS
PETS and QSDS were developed in the context of the World Bank supported programs in the
last 15 years to collect micro-level information on service delivery systems and service provider
performance.
The first PETS was implemented in Uganda in the education and health sectors in 1996. The
country was facing stagnant and even declining education and health outcomes despite
important increases in expenditures in these sectors in the past decade. In education, despite
the tripling of public expenditures, official primary school enrolment figures were stagnant
(Ablo and Reinikka, 1998). A decentralization policy had recently been implemented; district
authorities and urban councils were gradually delegating responsibilities of channeling funds to
schools and health facilities. The central government had very little information about resource
use and reasons for poor outcomes, but it was suspected that local governments might be
diverting resources for other purposes.
The objective of the first PETS was, therefore, to track expenditure flows in the hierarchical
structure in order to identify factors explaining these poor results, and to measure potential
leakages in school and health facility funding.
The hypothesis for the poor results was that public resources did not reach the schools and
health facilities (Ablo and Reinikka, 1998). Survey instruments were developed to compare
official budget allocations with actual spending at various tiers of government, including
primary schools and dispensaries.
In the education sector, the tracking exercise focused on a specific fixed rule budget-- non-wage
capitation expenditures to schools--which were officially based on enrolment figures at the
school level.
While data quality problems had hampered data collection in the health sector, the tracking
survey in education was able to identify various problems in the sector, in particular large-scale
resources leakage. The survey prompted the government to implement policy reforms,
including an information campaign. Significant improvements in expenditure flows to schools
were observed in the following impact evaluation PETS implemented in 1999 and 2001.
It is also in Uganda that the first QSDS was implemented in the health sector in 2000. Health
services in the country were perceived as not meeting the needs of the population (Lindelow,
Reinikka and Svensson, 2003).
Information about health service delivery and quality was severely lacking, in particular
regarding the scope and nature of the problems in government facilities, along with the
performance variance across ownership categories.
As improvements in data quality at the facility level were reported (such problems had
previously undermined the 1996 PETS in the health sector), a facility level survey was put
forward.
15
The survey’s objectives were to: (a) assess the types and quality of services provided by
different categories of providers, (b) identify problems in facility performance, including the
extent of drug leakage, as well as staff performance and availability, (c) provide information on
user charges and application of user fee policies, (d) measure and explain the variations in cost
efficiency across health units with a focus on the flow and use of resources, and (e) examine the
patterns of staff compensation, oversight and monitoring and their effect on performance.
Furthermore, the survey was intended to provide baseline data for future evaluation of policy
reforms in the sector (Lindelow, Reinikka and Svensson, 2003).
The survey collected quantitative and expenditure data among primary health care facilities,
about half of which were private for-profit and non-profit facilities. An exit poll of patients was
also conducted to gather, on the demand side, information on individuals’ characteristics,
behavior and perceived quality of health services.
Following these groundbreaking works in Uganda, PETS and QSDS, as well as combined PETSQSDS, were launched in other developing countries.
See Note 6 for an overview of the purpose, main motivation and objectives of various PETS and
QSDS implemented in Sub-Saharan Africa.
 Note 6: Some PETS/QSDS motivations and objectives
One could note that the original motivation of the Uganda 1996 PETS survey – i.e., to explain
poor performance and identify leakage -- was also generally the main motivation of the
following surveys.
Most PETS and QSDS have been motivated by common goals of increasing information on social
sector spending to understand the weak link between public expenditures and outcomes.
These surveys have been implemented in a context of weak institutional settings, marked by
deficient accounting, monitoring and reporting procedures.
In some cases, a number of other goals were added, in particular impact evaluation of
programs, such as in Ghana, Rwanda and Uganda, as well as more in-depth analysis of provider
performance through QSDS.
3. What have been some of the main topics of findings? 3
PETS and QSDS have proved to be powerful instruments at identifying several bottlenecks,
inefficiencies and wastages in service delivery, in particular, problems of leakages, delays,
information, record keeping, ghost workers, absenteeism, equity, decentralization, user fees
and efficiency.
3
See Gauthier (2006) and Gauthier and Reinikka (2007) for more detailed discussions of PETS/QSDS main
findings and contributions.
16
A. Leakage
One of the key finding of tracking surveys has been evidence of public resource leakage which is
broadly defined as the share of resources earmarked to specific beneficiaries which fail to reach
them. This phenomenon is associated with inadequate incentives and improper monitoring
and enforcement within the service delivery system.
Box: Definition of leakage
Leakage, generally understood as the share of resources intended but not received by frontline
providers, has been defined in two ways in tracking surveys.
The original or “strict” definition of leakage was introduced by Ablo and Reinikka (1998) for
rule-based expenditures as the share not received with respect to the expected (fixed-rule)
entitlement:
For non-fixed allocation rule flows (discretionary funding), Lindelöw (2006) proposed a
“narrow” (or soft) leakage measure which simply consists in the share of resources sent at a
certain level and not received at the other, in particular the facility level:
Within multi-layers administrative systems, there could be multiple leaks along the expenditure
chain toward the ultimate resource users.
In presence of multiple leaks along an expenditure chain, overall leakage rate is measured as :
Where n is the number of levels (links) in the chain and i is a link in the chain receiving
resources from another link j in the expenditure chain.
Local officials and politicians, for instance, could take advantage of the gap in information
available to central governments and citizens. Specifically, they could reduce disbursement or
procure fewer supplies for health centers and schools under their jurisdiction, which they know
17
would attract little attention given the weak supervision in most institutional settings in social
sectors in some developing countries.
 Note 7: Measuring leakages; issues
 Note 8: Average leakage levels in various Sub-Saharan African countries
As shown in the table below, several PETS have identified leakages on a large scale. Such high
rates of leakage of non-wage expenditures could have dire consequences for the quality of
public service delivery (Reinikka and Svensson, 2006a).
When 50 percent (Ghana), 87 percent (Uganda), or 80 percent (Chad) of funds for supplies
(medical and non-medical supplies, books and other schooling materials, i.e., non-wage inputs)
do not reach health centers or schools, leakage prevention must become a major policy issue in
these sectors.
Certain patterns in resource leakage tend to emerge from the survey findings, in particular in
terms of: (i) rule-based versus discretionary expenditures; (ii) wage versus non-wage
expenditures; (iii) levels of government; and (iv) in-kind versus cash transfers.
Table 1: Leakage by Sector and Country (%)
Country
Education
Health
Chad, 2004
80
Ghana, 1998
49
80
Kenya, 2004
38
Tanzania, 1998
57
41
Uganda, 1991-95
87
Zambia , 2001
10 (rule-based)
76 (discretionary)
Source: Reinikka and Svensson (2006a); Gauthier and Wane (2005, 2008b)
Rule-based versus discretionary expenditures
As emphasized by Reinikka and Svensson (2001), Das et al. (2004a), and Lindelöw (2006), the
level of discretion exercised in resource allocation influences leakage levels. Greater
discretionary power granted to particular administrative units, combined with weak supervision
and poor incentives, could lead to large fund capture. Indeed, differences in leakage levels
have been observed between funds allocated through fixed-rule and those that are at the
discretion of public officials or politicians. In Zambia, for instance, rule-based funding (perschool grants) presented a level of leakage of only 10 percent, versus more than 76 percent for
discretionary funding. As Das et al. (2004a) demonstrate, because rule-based funding is clearly
defined by a simple allocation rule, capture of funds is more difficult compared with
18
discretionary funds, which are not bound by any specific allocation rule. Generally, we observe
that the greater the agent’s discretionary power, the higher the leakage.
Wage versus non-wage expenditure:
There are also differences in leakage levels between salary and non-salary funds. As shown in
Table 1, tracking surveys, particularly in Ghana, Tanzania, Uganda, and Zambia, reveal that nonwage expenditures (channeled through intergovernmental transfers) suffer from more
extensive leakage than do salary expenditures.
In the case of non-wage expenditures, local officials and politicians can use their information
advantage to reduce disbursement or provide fewer non-wage supplies to health centers or
schools, knowing it would attract little attention (Reinikka and Svensson, 2004a, p.38). In
contrast, failure to pay health workers or teachers would attract attention given that workers
know how much they are owed. Indeed, salaries and allowances are observable and individual
recipients have a greater incentive to ensure that the funds actually arrive.
 Note 9: Salaries
Furthermore, salaries and other forms of staff compensation are generally governed by clearer
fixed rules rather than non-wage expenditures, which could also contribute to reducing funds
dissipation.4 Hence salaries and allowances seem to suffer from leakage to a much lesser extent
than other categories of public expenditures.
B. Delays
PETS have also shed light on the problem of delays and bottlenecks in the allocation of
resources through public administrations (e.g., salaries, allowances, financing, material,
equipment, drugs and vaccines). These issues could have important effects on the quality of
services, staff morale and the capacity of providers to deliver services.
 Note 10: Estimates of delays: examples from Africa
At times, the measurement of delays proved easier to estimate than quantitative or financial
data on the same flows, which would have allowed measurement of leakage levels (Gauthier,
2006).
 Example 4: Delays in Zambia, Rwanda, Tanzania, and Nigeria
4
In Zambia, important divergence was observed in terms of rules governing workers’ allowances, with direct
impact on delays and potential leakage.
19
C. Ghost workers
A few studies also quantify the share of ghosts on the payroll, that is, teachers or health
workers who continue to receive a salary but who no longer are in the government service, or
who have been included in the payroll without ever being in the service. 5 In Papua New
Guinea, for example, a 2003 PETS showed that 15 percent of teachers on the payroll were
ghosts (World Bank, 2004a). In a PETS survey in Honduras, 5 percent of teachers on the payroll
were found to be ghosts, while in health care, the percentage was 8.3 for general practitioners
in 2000 (World Bank, 2001). In Africa, the comparable figures are even higher: 20 percent in
Uganda in 1993 (Table 2) 6. In Mozambique, Lindelöw et al (2004) noted important
discrepancies between provincial, district and facility level staff records but could not confirm
the presence of ghost workers. Data problems were noted, but a further problem arose from
some health personnel being hired locally through community funds or user fees, which
complicated the comparison between the central payroll list and facility personnel. A similar
situation was observed also in Uganda (Reinikka and Svensson, 2004a).
Table 2: Ghost Workers on Payroll by Sector and Country (%)
Country
Honduras, 2000
Papua New Guinea, 2002
Uganda, 1993
Ghosts workers
Education
5.0
15.0
20.0
Health
8.3
─
─
Source: World Bank (2001) for Honduras; World Bank (2004) for Papua New Guinea;
and Reinikka (2001) for Uganda.
– Not available.
D. Absenteeism
Another important question studied in QSDS (or combined PETS/QSDS) is the problem of
absenteeism among front-line workers (see Rodgers and Koziol, 2011 for a review). Results
show absenteeism rates of between 27 and 40 percent for health care providers and between
11 and 27 percent for teachers. One of the main conclusions of these studies is that because of
poor accountability relationships and weak incentives, service provider absenteeism is
prevalent in developing countries, which translates into low quality of services (Gauthier and
Reinikka, 2007). 7 Apart from focus groups, QSDS have been used extensively to study
absenteeism among front-line workers.
5
Most public administrations in Sub-Saharan Africa use a centralized staff payroll system. Generally, teachers and
health workers are paid directly by the central ministry.
6
Note that this evidence was not obtained from PETS, but rather from a payroll clean-up exercise.
7
Das et al (2005) in particular have explored the relationship between teacher absenteeism and students’ learning.
20
Table 3: Absence Rates by Country and Sector (%)
Country
Bangladesh
Ecuador
India
Indonesia
Papua New Guinea
Peru
Uganda
Zambia
Primary schools
16
14
25
19
15
11
27
17
Health centers
35
40
40
19
25
37
-
Source: Chaudhury et al. (2006); Das et al. (2005); World Bank (2004a)
- Not Available
Table 3 presents the findings on absence rates from a large multi-country study (Chaudhury et
al. 2006; Rogers et al. 2004; and Chaudhury and Hammer 2004). The study reports results from
QSDS-type surveys in which enumerators made unannounced visits to primary schools and
health clinics in Bangladesh, Ecuador, India, Indonesia, Peru, and Uganda and recorded whether
they found teachers and health workers in the facilities. Averaging across the countries, about
19 percent of teachers and 35 percent of health workers were absent. The survey focused on
whether providers were present in their facilities, but because many providers who were at
their facilities were not working, even these figures may present a too positive picture.
 Example 5: Teacher and Medical Workers Absenteeism in India
 Example 6: Teacher Absenteeism in Zambia
E. Equity issues
Another research question examined using service delivery surveys is equity in the allocation of
resources and services by location and between income groups. In several countries, variability
of health and school spending across geographical areas, regions and districts, as well as within
districts, was observed. The considerable difference in resource allocation raises serious issues
of equity among socio-economic and demographic groups.
In Mozambique, for instance, a nine-fold variation in per capita health spending was observed
at the district level ranging from 5,000 to 47,000 Meticais annually. These variations seemed to
be driven mainly by staffing and infrastructure patterns across districts. In Chad, non-wage per
capita public health spending showed important variations across districts (16 to 1 ratio): at the
health center level, the most funded district received 38 CFAF per capita on average versus 2.3
CFAF for the least funded district (Gauthier and Wane, 2008b). In the schooling system in
Zambia, the most funded district received eight times more average per-student public
resources than the least funded district. This variance was due predominantly (90 percent) to
21
differences within provinces, as opposed to difference across provinces (10 percent). At the
school level, differences in Zambia were even more pronounced; the most funded school
received 3,000 times more funding than the least funded one.
 Note 11: Measuring equity
F. Decentralization effects
A few public expenditure tracking surveys have also examined the impact of decentralization on
resource allocations in the social sectors. In the 1996 Uganda education PETS, this
consideration was incorporated in the sample selection process and was studied through the
flow of capitation (per student) grants to schools. Findings indicated that decentralization had
not, at least in the early years, produced positive results in terms of better resource allocation
to service providers. Following decentralization, which was implemented gradually starting in
1993, district authorities and district and urban councils gradually gained control of the funds
allocated by the central government to primary education. Using the capitation grant as a
proxy to explore the impact of decentralization on the flow of public funds to schools, Reinikka
(2001) finds that decentralization was associated with a slight deterioration in the flow of funds
to schools.
 Note 12: Measuring impact of decentralization
G. Impact of user fees
Another research question that service delivery surveys have explored concerns user fees.
There has been much debate about the equity and efficiency effects of user fees on the
demand and supply sides. For frontline providers, user fees are sometimes the only source of
revenues. In several countries, as previously mentioned, providers receive only in-kind items
from upper administrative levels. On the demand side, user charges create clear problems of
accessibility and equity for the poor. QSDS have shown that user fees often constitute an
important part of service costs for users and can have considerable negative effects on access.
 Example 7: User fees in Chad
 Note 13: Drug mark ups
 Example 8: User fees in education: Uganda, Zambia, Rwanda
22
H. Issues of efficiency
PETS can address the question of efficiency of the budget which could have important benefits.
In the Chadian health sector, Gauthier and Wane (2009) show, using a PETS/QSDS that
reduction of leakage could have led to an important improvement in utilization rates of health
facilities. Indeed, the negative relationship between official public resources and health output
(Figure 4A) is reversed when leakage is taken into account given that effective public resources
(i.e., those that reach the regions) have a strong positive impact on health output (Figure 4B). In
Chad if all public resources officially budgeted for regional delegations in 2003 had reached the
frontline providers, the number of patients seeking primary health care would have more than
doubled during the year. Public expenditures could therefore contribute to the improvement of
the population’s health, provided they reach the population.
FIGURE 4: RELATIONSHIP BETWEEN OFFICIAL AND EFFECTIVE ALLOCATION OF PUBLIC RESOURCES
TO HEALTH SERVICES IN CHAD
Mayo-Kebbi
Mayo-Kebbi
500
500
Initial/Effective Allocations and Performance
Moyen Chari
Moyen Chari
Logone Occidental
Logone Oriental
Kanem
BET
guera
biltine
Tandjile
Production par 1000 Hbts
300
400
Production par 1000 Hbts
300
400
Logone Occidental
Logone Oriental
Kanem
biltine
Tandjile
Salamat
Salamat
Chari-Baguirmi
Ouaddai
Chari-Baguirmi
Ouaddai
batha
batha
Lac
200
200
Lac
500
BET
guera
1000
1500
2000
Initial per capita Allocation in FCFA
100
200
300
400
500
600
Effective per capita Allocation in FCFA
Source: Gauthier and Wane (2009)
Also noteworthy are various productivity analysis performed using tracking survey data. For
instance, Lindelow et al (2004) have examined the question of productivity of health centers
using PETS/QSDS data in Mozambique. They analyze seven categories of service output and
devise a composite index of output to deal with the problem of multi-output production. They
observe significant urban-rural and regional differences in service output per capita. Lindelow
et al (2004) also note important variations in output per health worker across districts (in an 8
to 1 ratio).
23
 Example 9: Measuring efficiency
I. Information problems
Another fundamental problem that PETS and QSDS have been able to highlight is the problem
of information asymmetry through the service providers’ supply chain, associated with
budgeting, accounting, reporting systems, supervision and monitoring. In most countries
examined, there is a crucial lack of information at various levels in the public organizational
structure, in particular at the central level, regarding resource use and transfers through the
supply chain. This problem of information associated with the agency context of service
provision is evidently one of the main issues motivating most tracking surveys. Furthermore,
information problems are also acute at lower levels of the hierarchy, as decentralized
administrative units are generally not aware of the budgetary resources to which they are
entitled.8
In such a context of data limitation, the detailed data provided by tracking and facility surveys
could help assess effective allocation of resources and identify the binding constraints that
impede quality of service delivery.
In addition, PETS could specifically address the demand side of governance. PETS could give
voice to citizens to help influence governments and providers. This requires hard information
about resource use that PETS can provide. By identifying resources available at various levels of
the supply chain and examining behavior and incentives of various agents within institutional
arrangements, they have identified problems of governance, capture of funds, and corruption.
PETS have been instrumental in bringing about effective changes in resource allocation, notably
through an information campaign in the education sector in Uganda.
In the Ugandan education sector, for example, Reinikka and Svensson (2004, 2005, 2006) use
PETS to study leakage of funds and the impact of a public information campaign on leakage
rates, enrolment levels, and learning outcomes. They find a large reduction in resource leakage,
increased enrolments, and some improved test scores in response to the campaign
J.
Record keeping
Finally, poor record keeping is one of the major challenges in public financial management that
PETS/QSDS have been able to pinpoint. Indeed, poor record management and weak internal
control at the various levels of the administrative system are certainly some of the major
findings of tracking surveys.
8 In Tanzania, for instance, information asymmetry was associated with the use of a cash budgeting system at the
central government level (REPOA/ESRF, 2001).
24
CHAPTER III: PROCESS, OWNERSHIP AND POLICY REFORMS
1. What are the impetus and sources of support for PETS/QSDS?
The impetus for launching a PETS, a QSDS or a joint PETS/QSDS could come from various
sources.

It could be initiated by donors or NGOs (or various demand side groups) that seek to
ensure efficiency or equity of public expenditure systems as part of their overall support
and request greater information and increase efficiency/equity of the resource
allocation system.
The interest could arise especially in the framework of a PER or sector analysis. A
PETS/QSDS could complement a PER by allowing collection of detailed micro level data
on effective allocation of resources within the sector and assessing the concordance
between sector objectives and effective allocation of resources (see section 1).

The study could also arise from a joint interest of donor and domestic governments that
perceive the importance of detailed data and analysis of sector expenditure as part of a
social sector development strategy or a poverty reduction strategy.

It could be initiated by the government that wishes to improve governance or efficiency.
In some countries such as Sierra Leone and Uganda, home grown PETS have sought to
fill a vacuum in information by providing diagnostic information on key poverty sectors
and evaluate government programs to increase transparency.
 Example 10: Impetus in Uganda, Chad and Ghana
2. The importance of dialogue with partner countries
Regardless of where the impetus comes from, it is fundamental to develop collaboration with
key ministries and to identify key partners. Policy dialogue, country involvement and country
ownership are fundamental in designing and implementing these tools.
 Note 14: PETS as part of country dialogue
 Note 15: PETS Within the programmatic AAA approach
25
Not only will this ensure access to the necessary information to conduct the study but it will
also promote the use of the study results. One of the key for the efficiency of such studies is
that they are used and that policy recommendations are translated into policy reforms. Indeed,
once a PETS or QSDS has identified inefficiencies and accountability failures, only the
government and other stakeholders can make the reforms necessary and mobilize civil society
to put recommendations into practice. Only if a government is prepared to make the necessary
efforts to reform the system will a PETS/QSDS survey have any real impact.
 Note 16: Raising the awareness of the government and potential stakeholders
Various donors have strong commitments toward public services and good governance (e.g.,
USAID, DFID). For instance, DFID’s 2006 White Paper supports public services and good
governance, and shows interest in how public expenditure systems are working in partner
countries, in particular regarding state capability, responsiveness and accountability.
PETS/QSDS are important tools for investigating these issues. Various donors and civil societies
in partner countries share this interest. It would make sense for donors to conduct joint analysis
in these areas. Ultimately, the objective would be for developing countries to undertake such
analysis on their own or in collaboration with donors, and feeding key feed back into their own
policies.
As mentioned earlier, PETS are not audits because they cannot reconcile the use of funds. PETS
do not try to find missing resources or identify the persons responsible. Care must be taken
that PETS are not perceived as a punitive or audit type instrument of one agency or government
level in relation to another, but rather should e viewed as a contribution to broader reform
efforts.
3. How can study findings be translated into reforms?
Beyond methodological issues and new applications, the biggest challenge for tracking surveys
may be to translate their important findings and contributions into policy reforms and
institutional changes to improve service quality and population outcomes. Indeed, tracking
surveys are a means to an end. The information on incentives and deficiencies in organizational
structures and rules should ultimately be used to identify policy reforms and help implement a
reform agenda in client countries.
 Note 17: Typical policy reforms resulting from PETS/QSDS
26
Box 1: Information campaign in Uganda
To date, one of the main achievements in service delivery improvements that have come out of
PETS-QSDS has been observed in the education sector in Uganda. The policy reform introduced
targeted information flows designed to give clients potential power over service providers.
Through an information campaign in newspapers and on radio that targeted school
associations, parents and the community, clients were better informed and given the ability to
voice their demands.
The results have been quite spectacular, in that the leakage rate was reduced dramatically. As
noted by Reinikka and Svensson (2004, p.23) similar information problems exist in other
countries, making the information campaign approach adopted in Uganda potentially widely
applicable. Such an information campaign approach to reducing corruption has also been used
in Tanzania. Following the identification in two PETS of fund diversion at the local government
level, the government started publishing information in newspapers covering allocations for
ministries, regions and local authorities (councils) of budget allocations for the selected propoor spending programs. Positive impacts of the campaign have been reported. While a formal
impact evaluation has not been conducted, reports indicate that the information campaign has
reduced leakage (World Bank, 2003, p.14).
4. Potential areas of reforms?
In addition to information campaigns, several other potential reforms can be on the agenda
following PETS/QSDS. The areas identified for reforms in various survey reports include:
(a) Speeding up budget execution at various levels of the delivery chain;
(b) Improving communication and information pass-through (dissemination of information is a
general problem between the administrative levels, including with the population);
(c) Increasing inspection and monitoring at all level of the service delivery system;
(d) Improving internal controls, in particular accounting and reporting systems to enhance
transparency and accountability (basic accounting systems are often not in place and rules
not followed);
(e) Improving logistic systems for managing material, supplies and various in-kinds (e.g., drugs)
at the decentralized level to allow adequate reporting and controls;
(f) Providing additional training for decentralized government levels and service delivery units
(problems often result from a poorly trained personnel);
(g) Establishing mechanisms and incentives in the system to make the service delivery system
more client driven at all levels.
27
5. Obstacles to implementing reforms
While it cannot be denied that many tracking surveys have been very successful at identifying
weak links in the service supply chain, as well as areas where reforms should be put forward, in
practice, however, few countries have effectively followed up on the diagnosis made by
PETS/QSDS and implemented decisive reforms in service delivery.
In several countries, lack of political will to put reforms in practice has certainly been a factor
behind weak institutional change. Lack of policy dialogue, insufficient dissemination of results
and discussions to ensure the transfer of information about problems identified in the service
delivery system are also noteworthy. More emphasis on client outcome would certainly
contribute to promoting institutional reforms in recipient countries. This would probably entail
reform of incentives on the project supply side. Success in project evaluation should therefore
be contingent not only on the survey implementation itself, but more importantly on the
capacity to translate and bring about policy and institutional reforms in the client country.
PETS and QSDS seek to improve the efficiency and equity of public service provision. As such,
the success of the exercise should be measured by its capacity to bring about improvements in
the quality of services at the population level, that is, in its capacity to lead to policy dialogue,
policy reforms in areas of weaknesses identified and improvements in the outcome at the
client/population level. Ultimately, in addition to devising robust methodologies to collect
information and detect corruption in public service delivery, the real challenge and the real
yardstick on which PETS/QSDS should be measured are the capacity to induce policy reforms to
correct the various governance problems identified.
28
CHAPTER IV. STEP-BY-STEP GUIDANCE
This chapter presents the steps involved in designing and implementing PETS/QSDS and
highlights methodological issues to be encountered through the process.
The steps are grouped in five parts (see Figure 5). The first part addresses the preparation of a
PETS/QSDS; the second, the background and preliminary analysis including the institutional
mapping report; the third, the design of the survey; the fourth, the survey implementation; and
the fifth, the data analysis, report writing and the dissemination of results.
Figure 5: Roadmap for designing and implementing a PETS/QSDS
Preparation
(Part I)
Preliminary Analysis
and Institutional
Mapping Report
(Part II)
Survey Preparation
(Part III)
Survey
Implementation
(Part IV)
Data Analysis, Report
Writing and Results
Dissemination
(Part V)
•
•
•
Rationale for the study and consultation with stakeholders
Resources and management structure
Concept note and TOR
•
•
•
•
Review of documentation
Analysis of policy and institutional arrangements
Choice of specific objectives, scope and tracking flows
Rapid Data Assessment
•
•
•
Questionnaire development
Sampling
Preliminary pilot
•
•
•
•
Training
Full Pilot
Survey Implementation
Data Entry
•
•
•
Data Analysis and Report Writing
Preparing Recommendations
Results Dissemination, policy reforms and future research
29
Phase 1. Preparation
> Step 1: Rationale for the study and consultations with stakeholders
A. Exploration and potential motivations
A PETS/QSDS study is concerned with identifying inefficiencies in public service delivery
mechanisms and assessing the performance of services at the frontline to improve the
effectiveness and accountability in the use of public resources and quality of services.
Before considering a PETS/QSDS, it is important to undertake an explanatory mission to analyze
the selected sector or program (in particular the resource allocation systems and service quality
and accountability issues) and conduct exploratory discussions to determine if detailed
evidence on expenditure allocations and diagnosis of the service delivery system could be
useful.
■ Various rationales could justify PETS/QSDS studies, in particular:

Assess why increased public spending in the sector does not lead to improved social
indicators;

Improve accountability and fill the gap in information on public expenditure and
resource use at the decentralized level by tracing expenditure flows toward end users of
resources;

Understand the poor performance and quality of public services to improve the
effectiveness of public expenditure;

Assess inefficiencies in public expenditure systems and services delivered to citizens
(delays, leakages, etc.);

Assess equity problems in public expenditure and service delivery among regions or
areas, income groups, and rural and urban locations;

Assess the gap in access to and utilization of basic services by specific groups, especially
the poor;
30
•
Reduce the fiduciary risk of donors’ programs of budget support allocations and ensure
the reliability and adequacy of public administrative systems in managing and allocating
donor funding;
•
Monitor specific programs and expenditure allocations, such as pro-poor expenditures,
by collecting quantitative information;
•
Provide baselines against which to monitor, through subsequent surveys, the
effectiveness of policy changes in the sector on quality and quantity of service delivery.
B. Consultations with key stakeholders
During the initial planning phase of the study, broad-based consultations should be held with
the sector or program’s main stakeholders to identify the main constraints and challenges
facing the sector and perceived inefficiencies in the system.
■ Stakeholders include key government ministries (e.g., Ministry of Finance, line ministries,
general controller), donors, and civil society organizations.
■ The consultations should:

Identify the issues and bottlenecks within the sector or program;

Convey information about the potential usefulness of the study;

Develop support and cooperation for its implementation;

Lead to agreement on the purpose and general objectives of the study (i.e.,
important efficiency and equity issues that need to be addressed).
C. Identification of broad objectives
During the initial phase, the broad objectives of the study should be identified and agreed upon
with the main stakeholders.
■ The broad objectives that could be targeted with PETS include:

Identifying the constraints in the expenditure and resource allocation system that
impede the efficiency, quantity, and quality of service delivery (including budget
execution and allocation, compliance with procedures, account keeping, and usage) to
generate recommendations for solving them;

Verifying the adequacy of the public expenditure system at allocating and monitoring
resources toward service provision in a sector or specific program;
31

Tracking the flows of public resources across various administrative levels of
government to identify malfunctions in service delivery systems, such as delays, leakage
and capture of funds by bureaucratic and political actors, corruption, and inequity in the
allocation of resources;

Determining if resources effectively allocated to administrative levels and final service
providers (e.g., schools, clinics) correspond to the official budget allocation and the
intended usage;

Assessing potential inequalities in the effective allocation of resources among regions,
districts, or geographical areas such as urban and rural populations;

Providing a quantitative performance assessment and a baseline in a sector or program
for monitoring progress or for a potential impact evaluation of a policy intervention;

Analyzing specific issues, such as recent policy reforms or the incentives of staff, or
establishing links between input, output, and outcomes.
■ The following broad objectives could be targeted with QSDS:

Assessing service delivery performance (quality and quantity of services, activities, and
operations);

Analyzing incentives and behaviour of staff (e.g., informal payments, absenteeism,
quality of services);

Assessing variations in service delivery across regions, districts, and types of clients;

Evaluating differences in performance between types of providers (e.g., NGOs, public
and private providers);

Identifying client characteristics and experience with service delivery (e.g., use of
services across income groups, cost incurred, satisfaction);

Identifying problems in facility performance, including the extent of potential resource
leakage, staff performance, and availability.
 Note 18: Typical PETS/QSDS objectives
■ Note that the overall objectives of the study will need to be transcribed into more specific
objectives during the institutional analysis phase (see Step 4).
32
Box 2: The five main elements that PETS/QSDS try to assess
Combined PETS/QSDS studies generally attempt to assess five main elements:
1. Resources (inputs) mobilized in the sector: their sources and importance;
2. Ways (process) in which these resources are mobilized: programs and allocation mechanisms
used and various channels toward users;
3. Ways (process) in which these resources are used: expenditure centers and usage at central,
decentralized, and service provider levels;
4. Inefficiencies and inequities in resource allocation within the sector/program
5. Results obtained by these resources: intermediate and final outputs, and outcomes/impacts.
D. Choice of instruments and alternatives
According to the broad objectives and issues facing the sector, a choice of instrument should be
made among a PETS, a QSDS, a combined PETS/QSDS, or other public finance management
instruments.
■ The overall process of a PETS/QSDS is relatively time-consuming and resource-intensive,
given its survey-based approach. It should therefore be determined if, among the alternative
instruments available to analyze public expenditure and service delivery, PETS/QSDS is the most
appropriate given the objectives, resources, and time frame of the planned activities.
■ A PETS seeks to improve the allocation of public expenditures and their impact on the
welfare of the population by providing precise and detailed information on the effective
allocation of resources and on potential weaknesses in the mechanisms used to allocate
resources.
It collects, on a sample basis, detailed information at various levels within the supply chain,
from central government or donors down to frontline providers (such as schools, health
facilities, and hospitals) on the allocation of resources. PETS involve the examination of
administrative records and reports of resource flows at different levels of the government
system, usually from central ministries down to service providers. Most PETS surveys include
interviews with staff in government offices and in front-line facilities, and the examination of
physical stocks of resources at various levels.
33
■ QSDS are provider surveys that assess the quantity and quality of services and performance
in resource usage at the facility level, such as schools, health clinics, and hospitals. They
examine the incentives and behavior of agents at the frontline level and gather information on
the availability of materials, such as books and desks in schools, drugs and functioning
equipment in clinics, or the attendance of teachers or health personnel.
A QSDS generally includes interviews with facility level staff, a study of records of facility level
inputs (financial, staff, in-kind material, and equipment), outputs (services delivered) and
revenues (e.g., from government, donors, and user fees), and a survey of clients of the facility.
Some have included the measurement of final outcomes such as pupils’ performance and have
linked it with school characteristics (e.g., Zambia 2002).
■ These two survey tools are often applied jointly to present a more complete picture of the
performance of service delivery systems. In some countries (e.g., Zambia), linkages have been
established with household population, and household information has been collected to
assess the impact of public expenditure on population outcomes.
Box 3: Single vs multi-sector surveys
Multi-sector surveys have been performed in various countries (e.g., Cameroon, Uganda, Niger,
Senegal, Rwanda, Tanzania), generally encompassing health and education in a single project.
Economies of scale, however, do not always translate into tracking survey quality. Indeed,
several problems are associated with multi-sector surveys.
First, there have been frequent problems of sample design, as the same enumeration areas
(village, wards, etc.) were often chosen to reduce costs. However, such an approach necessarily
means that some of the facility samples are biased, because the sample reflects only one of the
facility populations. Second, multi-sector surveys are generally associated with much less indepth data collection in certain sectors. Typically, one or more sectors will be sacrificed due to
budget or other resource constraints if, for instance, the same teams of enumerators have to
visit both schools and health centers in a given area within a limited time frame. Finally, with
respect to reporting, some sectors have been prioritized and were better reported in multisector surveys, while the other sectors tend to be much less thoroughly analyzed. For instance,
reports for the Tanzanian water supply and rural roads are not publicly available.
While one cannot categorically claim that multi-sector surveys should not be carried out, if they
are put forward, extra precautions have to be taken to ensure project quality. Separate
sampling designs must be chosen to ensure representative samples in every sector, interviews
should probably be conducted by different teams (or if by the same team, at different times),
and adequate time and resources must be allotted (see Gauthier 2006 for further details).
34
E. Government cooperation and ownership
A necessary condition for the success of a PETS/QSDS is to ensure that the government and
administrative units cooperate and are committed to the study and potential reforms.
•
Country ownership and clear commitment to reforms are key ingredients in successful
sector assessments and service delivery reforms.
•
An essential element is to ensure collaboration through discussion and the identification
of leaders and champions among key stakeholders.
•
Adequate cooperation of public officials at all levels is required to gain access to
information, and organized data collection can be ensured only with government
participation and commitment to the study.
•
Government ownership makes it more likely that the sector diagnosis arising from the
study and policy recommendations will be translated into policy reforms. Indeed, only
the government and other sector stakeholders can make the necessary reforms and
mobilize civil society to put study recommendations into practice.
•
The likelihood that the government will make the efforts needed to reform the system
following identification of inefficiencies and accountability failures by PETS is much
greater if a government takes ownership of the study.
F. Information and sensitization activities
To help build ownership, in addition to consultation and discussions, it could be useful to
organize larger-scale sensitization activities in the initial phase of the study to inform
stakeholders about the usefulness of collecting information on decentralized level expenditure
and service delivery performance.
■ These information and sensitization activities could take the form of seminars or workshops
on the usefulness of PETS/QSDS, and other service delivery tools. During these activities,
information could be presented on their methodologies, approaches, and potential findings.
■ These activities could help to:

Develop country ownership and build support for conducting the study;

Elicit views and identify problems in service delivery, and refine and identify specific
study objectives.
 Example 11: PETS/QSDS workshop and seminar in Morocco and Afghanistan
35
> Step 2: Composition of the team, resources and management structure
A. Composition of the team
A group of core researchers with relevant qualifications and experience should be formed to
conduct the study. Necessary elements for the core survey team include prior experience in
similar types of surveys and substantial country and sector-specific knowledge. The size of the
team will depend on the scope of the study and resources available.
■ The main skills required for a typical team to conduct PETS/QSDS studies include:

Skills in public finance, public sector management, and audits;

Sector-specific expertise (e.g., education, health, economics);

Technical skills in sampling strategies and questionnaire design;

Skills in data management and analysis, including statistical techniques, which are
especially valuable in the analysis and reporting stages;

Assistance in collecting relevant secondary data, documents, and information.
■ A project manager or task team leader should be appointed to undertake the day-to-day
coordination and management of the study. A clear line of responsibilities should be
allocated within the team to favor accountability.
B. Management structure and contractual arrangements
Donor agencies or policy makers can commission a PETS/QSDS using various management
structures and contracting arrangements.
■ While data collection is generally delegated to a local or international survey firm, the design
and day-to-day management of a PETS/QSDS study could entail various arrangements.

First, the initiating agency (donor or government) could decide to manage it directly.
This involves setting up a core team of researchers responsible for the overall study
design and then sequentially contracting out some subcomponents (such as data
collection) to external consultants. In this type of arrangement, the study manager
remains in the initiating agency commissioning the PETS/QSDS. For instance in such
setting, only survey implementation responsibilities could be delegated to a local or
36
international survey firm, and consultants (national or international) could be
contracted out to complete the in-house PETS/QSDS core team.

Second, the initiating agency may not have the capacity to design and supervise the
study and could decide to contract out the entire study. It would hence prepare terms
of reference specifying the study objectives, policy questions, overall methodology,
budget ceilings and minimum team composition (See Step 3). The TOR would be used to
launch a call for technical and financial proposals. The selected agency/consulting firm
would manage the study while the initiating agency team (donor or government)
provides oversight.

Third, a variant to this arrangement is for the initiating agency to split the
responsibilities of the study between two external groups (consulting firms or agencies)
while maintaining an oversight role. In the two-firm approach, the design of the study as
well as potentially data analysis and reporting, is contracted out to a core team of
experts/consultants, while data collection is delegated to another local or international
survey firm specialized in data collection. The two-firm approach has the advantage of
potentially allowing the recruitment of the most qualified team to design the study and
analyze results, as local survey firms in some countries often lack the competence for
designing and analyzing data for these types of studies. The core team of experts would
appoint a manager to coordinate and actively manage the study and to work effectively
with the data collectors and policy makers.
■ Choices of management structure vary according to initiating agency capacity, resources and
country conditions. The availability of specific survey competencies varies with the competitive
environment of the country. While data collection competencies are generally less difficult to
identify, design and specific competencies for survey design is generally less easily available at
the country level.
■ The management structure of PETS/QSDS studies requires a balance between technical
expertise and independence brought by a team of external experts, and the policy knowledge,
relevance and field coordination brought by government agencies and policy makers. For a
PETS/QSDS to be successful a degree of institutional separation is generally fundamental in
order to ensure objectivity independence and credibility (see box 4). However, the study
process requires close cooperation and coordination with policy makers and officials for
ensuring adequate design of the study, implementation and policy relevance. Still, PETS/QSDS
could have several goals including building internal monitoring capacity within government
agencies, and in that regard, closer sensitization and involvement of program operators in the
project could be chosen.
37
Box 4: Independence from government agencies
In some countries, an alternative to hiring private sector experts has been to engage
government officials to carry out the design work and sometimes data collection (for instance
statistical agencies or line-ministry staff). However, this raises questions about the capacity and
independence of the teams and the objectivity of the data collected. For instance, in the case of
the PETS in Indonesia, line-ministry staff were part of the enumerator team. While useful for
local capacity building, the enumerators were not necessarily well-trained to elicit correct
financial information (Gurkan et al., 2009, p.11). To ensure independent results, it is generally
recommended that government agencies not be directly involved in the design of the study or
data collection.
C. Steering committee
It is important for policy makers to be involved in guiding the study and ensuring its relevance.
In this regard, a small steering committee made up of representatives of the various
stakeholders to work with the PETS/QSDS team could offer some benefits.
■ A steering committee could help ensure cooperation and ownership, and promote capacity
building within government and civil society. It could provide guidance to the PETS/QSDS team
and foster cooperation among government agencies.
■ The inter-government steering committee could include a representative from each of the
line ministries involved, as well as from the Ministries of Finance and other government
agencies such as national statistics. It could also be composed of representatives of the Auditor
General’s office, development partners, and other stakeholders such as NGOs.
D. Budget
The budget necessary for conducting a PETS/QSDS depends on a number of factors, including
its scope, sample size, complexity of the survey instruments, sector, geography, and labor and
survey costs in the country. It also depends on the management structure chosen, in particular
the extent of work done internally or contracted out to local or international consultants or
survey firms.
■ Overall, the costs of a PETS generally range between US$ 75,000 and US$ 250,000 for a full
scale study (for cost estimations of small scale PETS, see Koziol and Tolmie (2010).
38
■ The budget needs to account for the costs involved in the design, implementation, data
analysis, reporting, and dissemination phases.

Data collection costs for a survey encompassing a standard sample size of about 150200 units (schools or health centers) in one sector range from US$ 50,000 to US$
120,000.

PETS core team costs need to be added, and depend on the composition (local or
international consultants) and responsibilities (e.g., survey design, questionnaire, survey
supervision, data analysis and reporting) of the team.

Travel costs need to be accounted for, especially the costs of field missions at the
various stages (e.g., consultations, institutional analysis and Rapid Data Assessment
(RDA), training and supervision, and results presentation and dissemination).
■ Note that an overall survey supervisor independent from the local or international survey
firm in charge of data collection should be present in the field for the duration of the survey
and data cleaning, and the supervisor’s costs should be accounted for in the overall budget.
■ If budgetary resources are insufficient, it could be worth using resources through
collaborations with other development partners or through trust funds, foundations, etc.
E. Time frame
Sufficient time and resources are needed to plan, design, and implement a survey, as well as for
data analysis, reporting, and dissemination.
■ It normally takes about 10-12 months to complete a PETS/QSDS, and sometimes more.

Sufficient time has to be allotted for conducting an in-depth institutional analysis,
realize a rapid data assessment, design the survey questionnaires, and pre-testing of
instruments.

Adequate time should also be allotted for data cleaning, analysis, and reporting, as well
as findings dissemination and policy reform discussions with the government.

Sufficient time should also be allotted to draft the TOR and hire consultants. Delays
could be expected when public tendering procedures are used for selecting the local
survey firm to conduct data collection.
o For instance, in Chad, the public tendering process to recruit a local survey firm
added one year to the survey duration, which overall lasted two years.
39
F. Project timetable and survey timing
■ An initial timeline should be prepared, accounting for the various steps and timing of the
different phases. It is useful to draw a list of tasks and a project timeline chart. The list of tasks
and chart should realistically align the successive activities and timeframe involved to promote
the efficient progress of the study.
■ An important element to consider in determining when to field the survey is the fiscal year
period in use in the country. If the target is to obtain quantitative data on flows of resources
over a one-year period, these clearly have to correspond to the fiscal year. The fielding of a
tracking survey should hence ideally be carried out two or three months after the end of the
fiscal year in order for accounting books to be closed.
 Example 12: List of Tasks and Project Timeline Chart
> Step 3: Concept Note and TOR
A. The concept note
■ During the preparation phase of the study, the project manager and core team will normally
prepare a concept note detailing the context and rationale of the study, main objectives and
research questions, scope, methodology, expected sampling and data collection strategy for
the study.

The document should present the background and context of the sector or specific
programs under study, the main challenges facing service delivery, the overall objectives
of the study, the main elements of the proposed activities, scope, timetable, budget,
dissemination activities, etc.
■ The concept note would serve as the basis for obtaining the support of government agencies
and other stakeholders and potential funders.
■ See the PETS/QSDS Web Portal for examples of Concept Notes.
B. Drafting TOR and launching the process for identifying survey firm/consultants
During the course of the project, various Terms of Reference (TOR) will normally be prepared
for instance for the local and/or international consultants responsible for designing and/or
40
implementing the survey. Composition of the TORs will be function of the governance
structure selected for the project (see Step 2B).
■ The selection of local/international consultants/survey firms could be a time-consuming and
challenging process depending on the rules in place. Initiating agency or government
procurement rules could entail relatively long procedures involving, for instance, public
tendering rules which could takes several weeks or months. The survey calendar needs to
account for these potential contingencies (see Step 2F).
■ It is crucial that the selected local consultant team implementing the survey be independent,
free of conflict of interest, and of high quality. This has tremendous impact on data quality, as
difficulties of implementing the survey are generally very important, and potential conflict of
interest could be present (see Box 4).
■ Local survey firms are likely to be more cost-effective to conduct the PETS-QSDS data
collection phase and contribute to local capacity building.
■ See the PETS/QSDS Web Portal for examples of Terms of Reference for survey firms and
international consultants
41
Phase 2. Preliminary Analysis
As a background phase, given that PETS/QSDS are designed to gather information on the flow
and use of public resources and the performance of service providers, a review of
documentation and analysis of the institutional arrangements and policy framework should be
conducted. This should lead to the elaboration of an institutional mapping report and a rapid
data assessment which will guide the development of the methodological strategy, in particular
the identification of the specific focus of the study and, in the case of a PETS, of the tracking
strategy.
> Step 4: Review of documentation
An initial step is to collect and review all the relevant documentation related to the sector and
programs under study. The review of documents seeks to examine the main aspects of the
sector and programs, the core policies and strategies, and the variety and levels of funding
flows, and to identify the main challenges facing the sector.
■ The key documents to be collected from various sources, including the government, donors,
NGOs, and research organizations, include:

Country and sector reports and analyses (PER, sector PER, CAS, etc.) from various
international agencies (e.g., World Bank, IMF, UNESCO, UNICEF, WHO);

Government sector documents and program reports;

Government financial and budgetary documents from the Ministry of Finance and line
ministries (central government consolidated accounts, line ministry—e.g., Ministry of
Education, state or provincial budgets if separate from consolidated government
accounts, medium term expenditure framework documents);

Sector administrative data and reports, such as routine information system data (e.g.,
Education or Health Management Information Systems) and annual reports;

Research and publications on public expenditure and sectoral issues
> Step 5: Analysis of policy and institutional arrangements
Following the analysis of documents, a thorough analysis of institutional arrangements and
policy frameworks should be conducted to understand the policy environment and
administrative arrangements and rules governing the allocation of resources within the sector.
42
■ Through field visits and interviews with key informants within central and sub-national
services down to frontline providers, the institutional analysis seeks to:

Identify the policies and programs in the sector and the sources of funding and channels
of allocation of sector inputs within the administrative structure toward frontline
providers;

Determine how the public hierarchy is structured, and the roles and responsibilities of
various administrative levels and units in the budget execution processes toward
frontline providers;

Identify the main issues and challenges facing the sector, which affect service quality;

Identify the allocation rules or mechanisms used to allocate the resources within the
programs at the various administrative levels;

Assess the accountability framework and identify the information system and reporting
mechanisms at each level for the different programs and delivery paths for the different
sector inputs;
 Ensure the distinction between official and effective rules and procedures (e.g., for
resource allocation, accounting, recording, or monitoring) that are observed in practice
for the different flows and levels. Rules and procedures effectively used to allocate
resources (or recording funding flows) could differ from official rules and may vary from
one location to another (i.e., province), and should be understood and analyzed.
■ Findings should be presented in an institutional mapping report.
■ See the PETS/QSDS Web Portal for examples of institutional mapping reports.
A. Identifying and assembling available data
PETS/QSDS studies use two types of data: (i) secondary data arising from administrative
systems or collected during previous data collection exercises, and (ii) data to be collected by
the study through survey techniques.
During the institutional analysis phase, it is important to identify and assemble important
existing secondary data that will be useful in the diagnosis of the sector, and start identifying
primary data to be collected by the study to track and monitor resources and results within the
sector.
43
i) Secondary data to be assembled
An early task in the preparation of the PETS is the identification and evaluation of the sources
of existing/secondary data and documents on sector financing, spending, activities, and output.
Copies of relevant documents and electronic records should be collected. These data are
dispersed among various services and government agencies.
Budget data
Central government public expenditures for health or education can be obtained from central
government budgetary records from the Ministry of Finance and line ministries such as
education or health, and potentially from other ministries (e.g., Planning, Human Resources).
Often, budgetary data available directly from finance ministries, finance commissions, and
other auditing bodies are more detailed than those published in public documents.
Central government budgetary figures could provide an overall view of the size and growth of
the sector or program under study. They also define a context for thinking about detailed
categories of spending and identifying the channels of budgetary allocation toward
decentralized structures (provinces/regions, district, local governments) and ultimately
frontline providers.
One needs to be aware of the ways expenditures in the sector are defined in the country. In
some countries, central government data include central/federal, state/provincial/regional, and
local/municipal health expenditures. Are there several ministries engaged in health care or
education spending, and is this spending included in the reported expenditures? Are special
expenditures, such as poverty programs, included? Are external resources included?
There are also differences in the level of detail available in budgetary records and the way data
are aggregated. The greater the level of disaggregation by program purpose, the better.
However, even within countries there could be important differences in the data availability,
quality, and capacity of public administration at producing and analyzing such data across
provinces or decentralized administrations.
 Example 13: South Africa: Western Cape vs. Limpopo
Executed vs. other budget data
For PETS purposes, one needs to make use of the executed budget, which presents, among the
various budget laws of a country, the best information available on official resources transfers,
and which reflects the variations in the levels of revenues and expenditures. 9
9
Indeed, there could be expenditure leakage between the Ministry of Finance and the line ministry. This
information also allows one to determine how adjustments are made when there are positive or negative shocks
(i.e. which ministry gets hit).
44
While in theory audited accounts of actual government expenditures are the most reliable and
are preferable to data on unaudited spending, there is typically a 1–2 year delay between the
end of the fiscal year and the availability of audited data on actual expenditures. Generally, the
most practical solution is hence to use provisional (unaudited) figures on executed budgets.
 Note 19: Forecasts vs. effective expenditures data
Routine data
In addition to budgetary data, most public administrations also collect routine information on
their activities. In particular, Management Information Systems (MIS) typically collect, on a
periodical basis, a large quantity of information on various components at the different
administrative and service provision levels within the education or health ministry. 10
If available, these routine data, sometimes compiled within education or health statistic reports
(e.g., “Annuaire statistique sur l’éducation”) or available at the disaggregated level, could be of
great use, especially to triangulate the survey data that will be collected by the study.
International donor assistance
In countries where international assistance is significant, collecting information on financial
transfers could be important. In addition to official development assistance, such as bilateral
aid and loan programs, one should include the activities of international non-governmental
organizations (NGOs).
Government records on external assistance should be collected, if available. However, while
some countries require all external financing assistance to be reported to a central government
agency, there is often a lack of standardized reporting systems across organizations and at the
country level. The complexity of financing arrangements also complicates the development of
usable data. Appropriate survey instruments could be developed in such cases.
ii) Primary data to be collected
During the institutional analysis phase, start identifying the primary data to be collected by the
study to characterize and monitor expenditures.
Indeed, given the limited existing secondary data available in the sector, the collection of such
detailed quantitative information of sub-national units and frontline providers collected
through survey techniques is at the core of the contribution of PETS/QSDS.
■ The choice of primary data to be assembled and collected depends on the specific focus of
the study; specific constraints facing service delivery, data availability and quality; and other
factors such as time and resources to be allocated for the study (see Step…. for a detailed
discussion). These primary data:
In Chad, for instance, primary health facilities are required to report their activities on a monthly basis to the
regional administration, which itself reports annually to the central health ministry.
10
45

are generally collected from a sample of frontline providers and lower level
governments and administrative services by which resources transit;

typically concern various themes, some generic to most PETS, such as assessment of
leakage of resources, and some more specific to some studies, such as staff
absenteeism, delays in budgetary execution, procedures for allocating public markets,
the use of the resources provided to frontline providers, etc.;

are frequently collected from staff working in these structures and users of services
(e.g., health patients, students, or households).
B. Understanding administrative structure and procedures
■ The complexity of resource flows in social sectors makes any tracking survey challenging.

Indeed, the resources required for public service delivery in education and health
(financial resources, human resources, and in-kind transfers) originate from several
sources (central government ministries, decentralized administrative levels, bilateral
and multilateral donors, NGOs) and take various paths in the organizational system.

In addition, these flows are generally governed by different allocation rules,
administrative processes, recording and accounting procedures, etc.

It is fundamental to identify the sources of funds, i.e., on-budget and off–budget, from
national and foreign sources.

A very thorough institutional analysis is hence required to detect these idiosyncratic
elements and to be able to choose an adequate and realistic focus for the study.

It is also essential to understand who is in charge of what and how the resources flow. A
thorough knowledge of paths of flows, by types of expenditures (investment, salary,
current) and types of programs and material, is necessary.
Generic funding flow structure
To illustrate funding flows, Figure 6 represents a generic administrative structure and resource
flow system in the social sector. A three-tier hierarchy is represented (central, provincial/
regional, and district/ local government) through which resources flow to local service
providers.
In some countries (e.g., Nigeria, Uganda), the administrative structure governing providers’
resources flows comprises only two layers (e.g., central and district/local government) above
frontline providers.
46
In most cases, central government ministries allocate resources (human, financial and/or inkind) to the various layers of the administrative structure. These resources pass through
various channels. Financial and in-kind resources might be the responsibility of a line ministry
while human resources could be the responsibility of a civil service ministry.
In addition, in most developing countries, international and national donors are pivotal to the
social sector allocation process, and their role and funding are important to grasp in the service
provider supply chain.
Some donors provide program or sector financial support to the central government, which is
then responsible for administering and allocating resources to the various sectors. Donors also
provide financial or in-kind support to provincial or district administrations, and sometimes
even directly to service providers (e.g., school items, health care materials, drugs or
equipment). Donors are also involved in capital investments such as construction of schools,
clinics and hospitals.
Figure 6
Funding Flows in Social Sectors
Generic Case
Central
Government
Donors
Provincial
Administration
District
Administration
Local Service
Providers
Funds
In-kind Transfers
Staff Remuneration
47
The phase involves analyzing donor expenditure levels in the sector, allocation procedures,
coordination mechanisms and monitoring systems in place. Interviews and review of budgetary
and program documents should help understand the role and approaches chosen by donors in
the sector and potentially include it in the PETS/QSDS analysis. Donors’ specific perspectives of
the issues and shortcomings affecting the sector should also be understood and specific
programs in place to try to address certain of these issues identified.
■ In the case of a combined PETS/QSDS, the focus of the inquiry, in addition to public
providers, could also include private for-profit and non-profit providers. The relationship
between public providers and private for-profit and not-for-profit providers could be important
to grasp, especially with respect to the environment of the health or school systems, the kind of
clinics or schools operating in the country, the nature of competition, the mixture of public,
private, religious, and community facilities, etc.
Box 5: Field visits and Rapid Data Assessment
Field visits of sufficient duration should be organized during the institutional assessment phase
to grasp the fundamental elements of the key programs in the sector, to understand the
administrative system, roles, and procedures, and to understand the effective allocation rules.
Core team members must spend sufficient time in the field to understand the sector and
institutional arrangements under study.
Visits and interviews with key informants within the various administrative levels involved in
the service delivery supply chain and frontline service providers should be conducted to collect
information on resource allocation systems, procedures, and potential bottlenecks within the
sector.
In Zambia for instance, some members of the PETS/QSDS 2002 core team spent months in the
field to better understand the idiosyncrasies of education supply chain and budgeting system
(Das et al, 2004). This team commitment has proved to be a crucial reason for the success of
the data collection exercise and study.
 Example 14: Objectives of Field Visits and Contents of Rapid Data assessment
■ It is fundamental to properly understand the specificities and idiosyncrasies of the sector or
program analyzed to develop an adequate methodological approach to implement tracking
surveys.
 Example 15: Mozambique health funding flows
48
 Example 16: Chad health funding flows
 Example 17: Zambia education
 Example 18: Senegal and Cameroon
C. Classifying resource types and allocation mechanisms
To better understand the flow of resources within the sector or program at various levels
toward service providers, it is useful to classify resource flows based on the types of resources
(e.g., material, salaries).
■ For instance, flows could be divided into three categories:
i) Monetary (cash transfers);
ii) Materials (such as textbooks, drugs, or equipment);
iii) Payroll (remuneration of staff).
Further subcategories, such as investment expenditures, could be introduced along functional
classifications in the budget.
■ Flows could also be classified based on the level of discretion at each level to use and
allocate resources, for instance between:
i) Fixed (hard) allocation rule;
ii) Soft (discretionary) allocation rule.
■ The Zambia 2002 PETS/QSDS in education, for instance, used such a typology to identify five
types of funding and resources, which helped guide the tracking flow choices and the
measurement of leakage (Das et al., 2004).
 Example 19: Zambia classification of funding flows
E. Identifying information flows and processes
In addition to allocating resources (funding, material, personnel), a sector’s institutional
structure is characterized by various accountability relationships within the hierarchical
structure.
Mechanisms to exercise control and reduce opportunism through supervision, reporting, and
enforcement of rules, particularly accounting and recording procedures carrying information
49
flows, need to be identified. Some information about budget and resource allocation flows in a
top-down direction, while other information circulates in a bottom-up direction (e.g.,
information on reception of resources, use of inputs, activities, needs, outcomes).
These supply and information flows and procedures should be identified and analyzed to
understand accountability systems and potential risk areas (see item F).
 Example 20: Chad health PETS/QSDS
D. Drawing resource flow diagrams
It is useful to draw diagrams showing how expenditures flow through each administrative node
within the public administration. This mapping exercise will help identify flows and procedures
used for allocating various resources within the administrative system toward service providers.
As mentioned earlier, it is very likely that each expenditure type is handled through different
systems and with different rules and procedures at various levels. For instance, in-kind items
and investment expenditures would be handled very differently by different services and
agencies using different procedures. Without such flow diagrams, it is more likely that mistakes
in data collection and analysis could go undetected (Ye, 2010).
■ These flow charts seek to trace the various resources and funding flows through the system
from the line ministry central administration to ultimate users at the school or health provider
level.
■ In particular, it is important to identify key “decision points”; that is, places where resource
allocation, deployment, and procurement are made. Decision-making may be centralized at the
national ministry level or be spread across different departments at different administrative
levels.
Such decision-making points are important: they provide opportunities for reallocation of
funds, bottlenecks, leakage, etc.
■ Diagrams should reflect the current circuit and approval system for resources (i.e., effective
instead of official by-the-book rules, if different), especially those that are expected to be
tracked in the data collection phase.
 Example 21: Flow charts in Ghana education
50
F. Identifying bottlenecks and risk areas
In general, in each of the branches of the supply chain system toward frontline providers, there
are possibilities of bottlenecks and leakage: funding and supplies (e.g., drugs, equipment, or
materials) could be delayed or captured. Similarly, salary expenditures could be delayed or
leaked through the creation of fictitious (ghost) workers.
However, in the specific sector or program under study, given the specific procedures to
allocate resources and accountability mechanisms for monitoring their use, some expenditure
flows, delivery channels, or areas will be more at risk of bottlenecks, inefficiencies, delays, and
capture than others.
■ It is especially important to identify higher risk decision points and weak nodes in the supply
chain that could affect the quality and availability of services within the sector to focus the
analysis and data collection strategy on these risk areas and resources.



Indeed, the identification of specific risk areas in a system could help determine the
specific focus of the PETS/QSDS study and condition the design of the survey instrument
and specific data to be collected.
Decision-making areas include personnel, supplies, financing, material (textbooks,
drugs), and supplies.
The analysis should focus on weak nodes and links in the system, and data collection
instruments should be adapted to the particular types of inefficiencies and corruption in
place to thoroughly understand potential bottlenecks and inefficiencies in the system.
 Example 22: Bottlenecks and risk areas
> Step 6. Choice of specific objectives, scope and tracking flows
A. Identifying specific objectives and research questions
■ Once institutional arrangements are understood and main risk areas identified, the study’s
broad objectives should be translated into specific objectives and research questions that will
drive the data collection strategy.
■ Also, hypotheses should be formulated to explain the problems facing the service delivery
system or program based on evidence and understanding of the system. These tentative
answers to each research question will help determine the specific data required to test these
hypotheses.
 Example 23: Examples of hypothesis
51
B. Defining the scope of the study
Another important decision relates to the determination of the scope of the study; that is, the
domain or coverage of the study. Given the complexity of administrative and supply chain
systems and multiplicity of transactions, it is essential to weigh the scope of the study against
the depth of analysis.
Answers to these questions should be guided by the ultimate objectives of the study, the main
findings of the institutional analysis, and the time and resource constraints of the study.
(i) Levels of services


A choice needs to be made about which levels of providers will be part of the analysis.
Will the study be limited to the primary sector (for instance primary schools or primary
health), or should it include the secondary or tertiary sectors?
If only the primary sector is targeted, how will the sector’s boundaries be determined?
The boundaries are relatively simple to determine in the education sector, but less so in
the health sector. Indeed, which units should be included: health centers, dispensaries,
hospitals, university hospitals, etc.?

If, for instance, the focus is on primary health, which includes outpatient consultations,
family planning, and maternal and child health services, these services are generally
provided both in lower level units (health posts, dispensaries, community health
centers, etc.) and hospitals. A key question is whether all levels of providers should be
included, or whether it is sufficient to include levels for which provision of primary
health services is the main objective.

In the case of the 2010 health QSDS in Senegal and Tanzania , the approach has been to
include both lower and higher level facilities until one was confident to have covered a
significant majority of people’s encounters with the primary health services. Coverage
of 75-percent was used as a rule of thumb. Hospitals were not covered in either country.
 Note 20: Choice of units in the primary sector in health and education in Senegal and
Tanzania 2010

For instance, in the Chad 2004 PETS/QSDS, all levels of hospitals (primary, secondary,
and tertiary) as well as health centers were included. (See Gauthier and Wane, 2005,
2008)
(ii) Ownership types

Which facility ownership types should be included? Will the study be limited to public
facilities, or will it also examine private for-profit and not-for-profit facilities?
52

While inclusion of all the types of service providers (levels, size, ownership, etc.)
presents several advantages, allowing a wider view of the supply side and in particular
identifying differences in performance across ownership types, it may not be compatible
with the budget or time constraints of the study or the specific questions it wants to
answer.
(iii) Regional coverage

Will the survey have a specific geographical coverage? Will the study be representative
of the whole country, or will it focus on specific regions, provinces, or urban/rural areas?
See section xxx for discussion on this issue.
C. Choosing tracking flows
■ A fundamental decision in a PETS, associated with the choice of scope of the study, is the
choice of specific expenditure flows to track.

For which funding or resource flows will financial and quantitative data (primary and
secondary) be collected, and at which levels?

Which types of expenditures or programs will be analyzed? Will the study be limited to
the analysis of specific programs or types of expenditures, such as non-wage recurrent
expenditure, or will the analysis also include salaries or investment budgets?
■ While there are possibilities of inefficiencies (e.g., delays, leakage) in each of the branches of
the funding flow system, not all flows are amenable to tracking.

Nonexistent records or accounts, data inconsistencies, and other problems will make
certain flows untraceable, or make the data too noisy to be informative.

It is thus important to restrict the domain of data collection and analysis.
■ Common trap of past PETS: too wide coverage
■ Several factors could influence the choice of resource flows to track.

The choice could be derived directly from the research question or survey objectives.
o For instance, if the objective of the survey is to identify the prevalence of ghost
workers, then the domain of financial flows to track could be restricted to salary
flows.
53
Box 6: Trade-off between wide coverage and survey feasibility
Given the complexity of administrative and supply chain systems, the trade-off between wide
scope and depth of the study needs to be resolved. In particular, a common trap of past surveys
has been overly wide coverage, which has reduced overall study quality.
Given the data limitations in most countries, gathering information on line ministries’ entire
sector flow or all recurrent expenditures can be very difficult.
Experience has shown that PETS/QSDS studies that have tried to track a wide spectrum of
expenditures have run the risk of not being able to collect consistent, high-quality data.
It is better to focus on a limited number of specific funding flows for which records or accounts
of good enough quality exist on at least two levels of government.
 Example 24: Some features and lessons of successful PETS
 Example 25: Features and lessons of less successful PETS
o Similarly, if the purpose of the survey is to evaluate the impact of a specific
program in the education or health sector (e.g., HIV/AIDS), then the focus of the
tracking exercise could naturally be limited to this specific program.

If the objectives of the study are more general and seek to identify leakage or other
inefficiencies in the resource allocation system, then factors related to data availability
and quality have to be taken into account.
D. Measurement of leakage and tracking choices
The measurement of leakage is conditioned by the rules governing resource allocation,
especially the presence of fixed and soft allocation rules. The selection of tracking flows should
hence be done with a clear understanding of the allocation rules in place to ensure that the
proper choice is made and that the relevant data required to measure leakage are available.
 Note 21: Hard and Soft rules governing resource flows
(i) Fixed (hard) allocation rule
If the tracking of resources is done on an expenditure flow for which a fixed allocation rule is in
place, then leakage could be readily measured as the ratio of entitled funds that did not reach
the facility during a specific period (See Boxes 7 and 8 below).
54
In such cases, the only data required to measure leakage are the variables of the formula to
calculate transfers and data about the funding reaching schools (assuming that reliable data
could be collected about the funds reaching facilities).
The existence of fixed rules tends to greatly simplify the measurement of leakage because it
places fewer requirements on the data to be collected.
Indeed, a factor that facilitates the measurement of leakage and that drove the success of the
Uganda education PETS, for instance, was the existence of a fixed allocation rule for the
program tracked (capitation grant).
As mentioned in example …., each school in Uganda was entitled to a specific amount of cash
(or in-kind) transfers based on the number of students enrolled. This fixed allocation rule
greatly simplified the leakage measure exercise because only a measure of arrival of resources
at the facility level was required to be collected, and leakage was measured in a straightforward
fashion, as the ratio of what a school received to what it was entitled to.
ii) Discretionary (soft) allocation rule
In the absence of a fixed allocation rule (i.e., where discretion is left to officials to determine
the allocation to facilities according to needs or other considerations), the measurement of
“narrow” leakage between specific levels should be targeted. This consists in measuring the
ratio of actual resources disbursed at a higher level and the resources received at a lower
administrative level or frontline facility.
■ Caution has to be exercised with respect to the interpretation of leakage levels (in particular,
but not exclusively, the “narrow” measure), because reasons other than corruption might
explain low arrival rates or observed differences in resources between levels (Reinikka and
Svensson, 2004a; Lindelow, 2006). These include incomplete records or problematic
accounting, data collection problems, and data entry error. These issues will be discussed in
detail in Section 5.
The normal use of resources at an administrative level also has to be accounted for in the
leakage calculation. However, it is often difficult to measure input use at a certain level. For
instance, in the Ghana report, an 8 percent administrative overhead cost was used to measure
(narrow) leakage.
■ In the countries surveyed, it was generally observed that the greater the agent’s
discretionary power, the higher the leakage observed.
 Note 22: Rule based versus discretionary expenditures
55
Box 7: Measuring leakage with hard allocation rule
The way leakage is measured, as well as the relevant data required and the sampling strategy, is
influenced by the rules governing resource allocation.
While leakage could be measured in the absence of fixed allocation rules, rule-based allocation
systems simplify the analysis; in that case only the effective transfer to the unit (e.g., school,
health center) needs to be collected and compared to the intended transfer.
In this case, “strict” leakage is measured as the ratio between how much the facility actually
received in financial transfer during the period under study and the entitled funds; that is, how
much it should have received during that period.
For instance, in the case of an allocation to schools based on the number of enrolled students,
one would need only to know the per capita transfer level, the number of students enrolled,
and the value of receptions.
 Note 23: Measuring leakage on resources allocated with fixed/hard allocation rule
Box 8: Measuring leakage with the soft allocation rule
In the absence of a fixed allocation rule, the leakage rate between different levels is measured
as the ratio between how much the facility actually received and how much the central level (or
other hierarchical level) has sent to the facility.
In the absence of fixed rule, in addition to data on resources received by a sample of frontline
providers (as in the case of a fixed rule), one needs to collect data on the value of resources
disbursed/sent by the upstream unit (e.g., central, regional, and district levels) toward a lower
level unit.
Note 24: Measurement of leakage in case of discretion/ soft allocation
Given that transfers are earmarked (e.g., by student, school, or health center) and hence there
is no benchmark against which to measure the expected transfer to the lower unit, narrow
leakage measures instead the transfer not received; that is, the share that is used or leaked at
the upstream levels. For instance, in Chad, where no fixed rules are in practice in the health
sector, the narrow leakage definition was used. It was observed that overall transfer to health
centers was estimated at less than 1 percent of the regional non-wage budget.
56
Recommendations





The choice of resources to track has to be guided by the specific country conditions,
targeting flows for which risk areas have been identified and for which quality data on at
least two levels of the service delivery chain could be collected;
Tracking flow selection should account for the types of rules governing resource flows
because they affect the way leakage is measured and the specific data required to
measure leakage;
Successful past surveys (e.g., Uganda education 1996, Zambia education 2002) have
restricted the tracking domain on flows for which good quality and consistent data were
available;
Bypassing some levels is sometimes recommended given the limited and potentially
inconsistent information available at these levels (e.g., Mali, Mozambique);
The sampling strategy needs to be adapted to the allocation rule governing the
resource flows selected for tracking (See step….).
 Example 26: Recommendations of potential tracking choice strategies
> Step 7: Rapid Data Assessment (RDA)
Before finalizing the tracking choices, a rapid data assessment (RDA) should be performed to
determine the survey’s feasibility and verify the choice of tracking flows and data availability
and quality.
■ The RDA is an assessment of data availability, quality, and consistency.
■ The RDA is fundamental to ensure that the data required to test the hypothesis are available
and of sufficient quality, and to avoid a costly and time-consuming exercise of gathering
inconsistent data.

If the specific data required are not available, the empirical strategy needs to be
adapted to the available data.
o It could lead to the redefinition of the survey objectives, hypothesis and choice
of tracking flows.
o It should lead to adequate design of survey instruments based on the available
data.
57

A simple questionnaire administered at various administrative levels is usually sufficient
for the purposes of the RDA. At each level of the public hierarchy, as well as facility
types and various locations:
o Verify the existence of records (receipts of resources sent by the government,
services rendered, etc.);
o Verify data availability, quality, and consistency given that there could be
differences across geographical locations (e.g., rural vs. urban) or among
provinces.
See  Example 14: Contents of Rapid Data assessment
■ See PETS/QSDS Web Portal for other RDA Instruments used in previous surveys.
■ If information is of poor quality at the local government level (region or district), for instance:
 This level could be bypassed and information could be collected only at the facility level
(to measure resources available for service) and at the central ministry level (to know
how much was officially sent).

Some specific programs or budget lines could be excluded from the tracking exercise if
inadequate or poor quality data are observed.
■ Some surveys have failed because this essential step was skipped, or because there was
inadequate RDA. Data problems are frequent at various levels within the administrative system.
 Note 25: Rapid Data Assessment
■ During the rapid data assessment phase, the length of the data collection period should be
determined and relatively standardized among types of data, with the objective of balancing
the goal of completeness with data accuracy (see step….).

Completeness and length of the data collection period (or targeted) generally detracts
from accuracy, as there will typically be more missing data for a longer time period.
Given that data at the facility level are generally quite disaggregated, the work of
enumerators becomes very tedious, if not unfeasible, when too much data and too long
a period are targeted.
58
Phase 3: Survey Preparation
> Step 8. Questionnaire development
The design of survey instruments depends on the study’s objectives, scope, and specific choice
of tracking flows.
■ It is fundamental to identify the relevant data required at each administrative level to verify
the hypothesis formulated to explain inefficiencies or iniquities in the system and to
systematically measure leakage levels.
■ A set of generic questionnaires and optional modules for primary education and health are
proposed:
■ See the PETS/QSDS Web Portal for Generic survey instruments in education and health
■ Questionnaires have to be customized to the country’s context and specific survey objectives,
particularly to the instruments of the administrative level in line with the specific programs and
decision rules in practice.

In particular, quantitative data questions (e.g., financial, input and output data) should
be adapted to the country’s administrative system; that is, the categories used in the
administrative reporting structure. This would maximize the probability of having data
reported and collected by enumerators.
o Note that staff questionnaires and exit polls (e.g., patient questionnaires)
generally need less customization, while input and output activities need more
customization to the country context due to variability in the accounting systems
and items used.
■ Generic questionnaires also include a list of basic service delivery indicators for PETS and
QSDS in each sector, allowing potential benchmarking across countries measuring the evolution
and monitoring of progress across time and countries.
■ Survey instruments should be circulated among stakeholders for comments and approvals.
A. Survey instruments and units of analysis
PETS and QSDS mainly assemble objective and quantitative information from surveys of
administrative levels involved in the supply chain and service delivery units using various
modules.
59
Given the complexity of the resource flows and number of actors and administrative units to
consider, one of the specificities of PETS/QSDS is their wide variety of survey instruments and
data collection methods.
Indeed, PETS and QSDS are composed of various instruments designed to collect information at
the different organizational levels and among stakeholders involved in service delivery, on both
the supply and demand sides.
Typically, combined PETS/QSDS studies have included specific instruments to perform data
collection on the following units of analysis: the central government’s Ministry of Finance and
line ministries (e.g., Health, Education), regional administration, district administration,
frontline provider (i.e., unit representative, staff), and clients (i.e., patients, students). Table 5
presents the levels and main units of analysis examined in PETS and QSDS.
Table 5: Units of analysis in PETS/QSDS
Type of Survey
PETS
Levels
Units of analysis
Central
government
Ministry of Finance
Line ministry (e.g. Health or Education)
Regional
Provincial (or regional) administration
District
District (or local) administration
Frontline
ESDS
Service provider (e.g. school or health centers)
Staff
Clients (e.g. patients or students)
QSDS
Population
Households
Note: Combined PETS-QSDS are sometimes called Expenditure and Service Delivery Surveys (ESDS) or
Expenditure Tracking and Service Delivery Surveys (ETSDS).
PETS
PETS focus on tracking resources through the supply chain down to the service providers and
on the use of these resources among levels and service providers. PETS are composed of
several modules.
60
Central government unit module
At the central level, it is essential to collect information about the roles played by each ministry
and administrative unit involved in the allocation of resources in the sector or program
analyzed. It is particularly imperative that budgetary and administrative data be collected at the
central level. These annual data, both for the entire sector under study and specific programs
tracked, should be disaggregated at the provincial, district, and potentially service provider
levels, if available. Such data would allow the measurement of resources officially allocated to
decentralized levels. It may also make it possible to “triangulate” information provided at lower
levels.
(i) Ministry of Finance: The first unit of analysis in a PETS is the Ministry of Finance (MoF) at the
central government level. Within the MoF, information on the budget allocated to the sector
under study is collected. It is generally necessary to collect the budget law from the Ministry of
Finance (MoF), which identifies the disbursements to the different line ministries and functions.
It is important to collect not only forecasts but also real allocations. (See step….)
(ii) Line Ministry: Data collection at the central government level also requires one to identify
how much resources were received and used by the line ministry (e.g., Ministry of Health,
Ministry of Education), as well as transferred to lower administrative levels (e.g., regions,
provinces, districts). Resource transfers could take the form of human resources and financial
transfers, but also in-kind items.
If tracking of in-kind resources is part of the study, information on invoices needs to be
collected. Alternatively, if in-kind tracking is done on a sampling basis, a list of the materials
allocated to lower levels needs to be collected (See step sampling….).
State/provincial/regional administration module
For countries with a health or education sector comprising a state, provincial, or regional level
of administration, a state/provincial/regional administration questionnaire is generally
administered to the Directorate of Health or Education.
For the state/provincial (regional) and district levels, as for the line ministry level, it is necessary
to quantify how much resources were received from higher levels and sources (domestic and
external), and how much was used and sent to other levels. Information on invoices and
reception also needs to be collected at this level.
At such levels, information should be collected about roles played in the frontline provider
supply chain and the monitoring process. Quantitative annual data should also be collected for
the sector under study and specific programs tracked.
The data collected at that level should cover areas such as: (i) allocation and execution of
budgets; (ii) distribution and management of drugs and other supplies; (iii) human resources;
(iv) infrastructure and equipment; (v) monitoring and governance; and (vi) service outputs.
61
District administration module
At the district or local government level, a District Administration Questionnaire is generally
administered to the district director (district head doctor, etc.)
Again at that level, information should be collected about roles played in the frontline provider
supply chain and monitoring process. Quantitative annual data should also be collected for the
sector under study and specific programs tracked.
The questionnaire should cover at least four areas: (a) health or education infrastructure; (b)
staff training; (c) support and supervision arrangements; and (d) sources of financing.
A District Data Sheet is generally used to collect detailed quantitative information for the last
completed fiscal year on: (1) staffing and salary structures; (2) basic and supplementary
supplies (material, medication, vaccines, etc.); (3) services and activities (students, outpatients,
inpatients, immunization, deliveries, etc.); and (4) financing (government budget, donor
support, user fees).
(iii) Service providers (e.g., school or health center): At the provider level, it is especially
important in a PETS to quantify how much resources were received during the time period
covered by the study focusing on the same resources tracked through the supply chain. Other
data on characteristics, performance of providers, and quality of services are also useful.
QSDS
QSDS specifically seek to assess the use of the resources at the service provider level and
quality of services offered to the population. A QSDS is composed of several instruments
addressed to various agents active at the frontline level and could also cover the demand side
by including instruments on student, patients, or households.
Service provider module
At the service provider level, the Facility Questionnaire seeks to collect a large spectrum of data
and information on the characteristics and activities of facilities. The questionnaire is generally
administered to the person in charge of the facility. In schools, this person is generally the head
teacher or school director. In health centers, the person tends to be the head doctor or head
nurse.
The general structure of the service provider questionnaire module should include data
collection in at least five areas: (1) facility characteristics; (2) financing; (3) inputs; (4) outputs;
and (5) institutional support:
1) Facility characteristics/infrastructures:
Location, type, level, ownership, catchment area, organization and
services offered, competition in provision of services, etc.
62
2) Financing
User charges, cost of services by categories, expenditures, financial and
in-kind support, etc.
3) Inputs:
Staff, material, and other inputs: books, supplies, drugs, vaccines, medical
and non-medical consumables, infrastructure (capital inputs), etc.
4) Outputs:
Sector-specific: number of students/teacher, enrolment rate, passage
rate, facility utilization and referrals, etc.
5) Institutional and organizational issues:
Supervision, reporting, performance assessment, auditing, rewards
systems, procurement, citizens’ participation, information about services
and pricing, etc.
The key information that needs to be collected includes measures of access to services. These
include the physical distance of the targeted population (e.g., walking distance to the health
clinic or school), the variety of facilities available in the area and services offered, and the
opening and closing times of facilities. Measures of affordability of services, which include
direct costs (such as user fees, transport costs, bribes and other informal payments) and
indirect costs (opportunity costs, such as waiting time, etc.), must also be captured. Measures
of quality of services, which could be proxied by physical and human capital, consultation
duration, availability of material, etc., are also essential. Measures of the accountability
relationship should also be targeted. These could include proxies such as supervision, reporting,
auditing, performance assessment, rewards systems, and feedback mechanisms.
In addition, the Facility Questionnaire is generally supplemented by a Facility Data Sheet, which
is used to collect detailed quantitative information from the school or health unit records or
accounts on various themes, in particular: (1) staffing and salary structure; (2) student
enrolment or patient records for the last completed fiscal year; (3) types of students or patients
using the facility; (4) services offered (class levels, health services such as vaccinations); (5)
supplies available and usage (books, chalk, medication, vaccines, etc.).
As previously mentioned, survey instruments, in particular facility data sheets, should be
adapted and customized to the standard accounting and reporting procedure in practice at the
facility level in the country to facilitate enumerators’ work and increase data quality.
63
Staff module
At the facility level, a Staff Questionnaire is generally administered to obtain information on the
characteristics of the personnel as well as their incentive structure. The questionnaire is most
often applied to a sample of teachers or health workers. It includes information related to the
following areas: (1) characteristics: status, formation; (2) incentive system: salaries, allowances,
supervision, performance assessment, performance-based promotions, sanction, or rewards;
(3) perception of work environment.
Staff questionnaire second visit
It could be useful to collect information on absenteeism patterns of the staff. For that purpose,
during an unannounced second visit, information is collected on staff presence in the facility
and reasons for absence, if any.
Client module
The perspective of users (students, patients, etc.) sheds valuable light on problems of quality or
efficiency in service delivery, given that users are the ultimate beneficiaries of services. Most
surveys to date have included a patient exit poll or student survey.
In general, at each of the school or health facilities surveyed, a number of users were
interviewed. In health facilities, outpatients were surveyed after having completed a visit. If the
patient was a child, the caregiver would be interviewed.
 Note 26: Size of user surveys
The Client Questionnaire generally covers the following main areas: (1) client’s characteristics;
(2) reason for visiting the facility; (3) access to the facility and other health care providers; (4)
services received and medication prescribed; (5) costs of services (payments made the day of
the visit and on previous occasions); (6) exemptions; (7) perception of quality: reception in
clinics, consultation duration; (8) informal payments/corruption; (9) alternative sources of
services (schools, health services).
B. Data categories
A specificity of PETS/QSDS studies is their focus on the institutional environment of service
provision, particularly the incentives of the actors. It seeks to capture these features by
collecting information on the production process, accountability mechanisms, financial flows,
and reward structures.
64
Table 6 classifies data collected in PETS and QSDS in seven categories: (i) environment and
characteristics, (ii) financing, (iii) inputs, (iv) institutional arrangements and production process,
(v) intermediate outputs, (vi) final outputs, (vii) outcomes. 11
Table 6: Data categories in PETS/QSDS
Data categories
Description
Examples of variables
Environment and
characteristics
Environment and characteristics of the
units and agents in the supply chain,
including frontline providers
Size of facilities, ownership
structure, types, location,
competition
Financing
Financing at the different levels and from
the various sources
Amounts of financing, types,
sources, reliability
Inputs
Inputs (transfers, uses, and reception) at
the different levels. Inputs could include
less tangible elements such as staff quality
or realized inputs such as the number of
facilities.
Material inputs such as staff
and salaries, textbooks,
equipment, medication.
Institutional arrangements
and production process
Production process at different units
through the supply chain, i.e.
management structure, oversight,
incentive structure
Management practices,
supervision, reporting, audits,
record-keeping procedures.
Intermediate outputs
Intermediate outputs of the production
process
Level of absenteeism, penalties
and rewards
Final outputs
Final outputs of the production process
Outcomes
Outcomes and quality. Overall measures
of final outcomes within the sector.
Number of patients treated,
enrollment, graduation rates,
class repetition rates
Mortality rates, student
performance, patient or
student satisfaction
Source: Amin and Chaudhury (2007), Gauthier (2006)
 Note 27: Rationale for collecting various categories of data
C. Lessons and recommendations
■ Data sources: records versus recalls
In order to minimize measurement errors, it is recommended to use records, accounts, or
invoices to collect quantitative or financial data.
11
Amin and Chaudhury (2007) classify data collected in micro-level surveys into six categories, grouping together
process and intermediate output data.
65

In exceptional cases, when no other sources of data are available and data based on
recalls are collected, clear indications in that respect should be reported.

It should be noted that for certain inputs, such as resources allocated directly by donors,
information based on records could be very difficult to gather. Indeed, resources
received from NGOs or directly from other donors at the facility or local government
levels are generally not recorded using standard procedures. Enumerators then have
little choice but to collect such information from the recall of the respondent in-charge.
 Note 28: Data sources records versus recalls
■ Length of quantitative data tracking
As mentioned above, the length of the data collection period should be assessed during the
rapid data assessment phase (and verified during the pilot phase) and relatively standardized
among types of data to balance completeness of objectives and data accuracy.

Ideally, data collection should involve annual data and cover a period of a maximum of
two financial years (one financial year is probably even more than adequate) to
maximize data collection quality.
 Note 29: Length of quantitative data tracking

If monthly data are collected, seasonality issues have to be considered and a proper
strategy devised and clearly explained to reduce potential biases.
 Note 30: Seasonality of data
■ Customization and parsimony of data

As mentioned above, survey instruments, particularly facility data sheets, should be
adapted and customized to the standard accounting and reporting procedures in
practice at the facility level in the country in order to facilitate enumerators’ work and
increase data quality. (See step….).
 Example 27: Chad: Health PETS: data collection

In the past, surveys have collected information that is not used.
66

Parsimony of data collected is recommended to reduce costs, but also to increase the
quality of data collected. The questionnaire should be focused and contain a reduced
number of questions.
■ Tracking in-kind items: In many countries, a large proportion of resources transferred to
frontline providers are in the form of material or other types of in-kind items (e.g., school
supplies, drugs, and medical and non-medical materials). In some countries, they are even the
only resources provided other than human resources (e.g., Chad health sector, Morocco
health). Tracking in-kind items is hence crucial.
Box 9: Tracking in-kind items
To track in-kind items, when the number of items is too large, a sample approach should be
used instead of a census approach.
Indeed, a limited number of items should be targeted; for instance, the 10 most frequently
used medications or school items. The list of items needs to be country-specific.
High-frequency items should be tracked instead of high value items, and the choice of materials
(or medications) should be based on their shipment frequency in the Ministry invoice list (if
available).
With such an approach based on frequency instead of total value, the risk inherent in choosing
a rare but high-value material (e.g., a car) is not finding that material in the visited service
provider simply because not all of them were able to receive it. In contrast, by choosing
frequently shipped materials of small value (e.g., Windex), it is likely that a maximum number
of facilities would report receiving them (Gauthier and Wane, 2005).
This would give an upwardly biased percentage of facilities receiving materials from the
authorities. Valuation of in-kind items should be done using a standardized price list (such as
the line ministry list, if available).
Resources received should be estimated relative to the share of the sample items in the global
budget allocation.
■ Translation of questionnaires
The translation of questionnaires into local languages could prove necessary, especially for the
instruments intended for service users (e.g. patients, students, households), if applicable. To
minimize the need for successive translations and divergence between the original and
translated versions, translation should be realized once questionnaire testing and revisions
have been completed.
67
> Step 9: Sampling Strategy
The sampling strategy is a crucial methodological component of PETS/QSDS. Presented below is
a brief overview of some of the main considerations to be taken into account when designing
PETS/QSDS survey strategies. Detailed guidelines of sampling procedures could be found in
sampling textbooks (see for instance Turner et al, 2001 and…..). Survey teams should involve
early a statistical/sampling specialist to design the sampling strategy.
A. General considerations
■ The objective of PETS/QSDS is to provide detailed information on the flows and arrival of
resources at the frontline and of the performance of service providers and quality of services.
■ PETS/QSDS studies use a sample survey methodology. The sample generally seeks to be
representative of the supply side of services within the sector or program.

Representativeness could be sought at the national level and to capture differences
among administrative or geographical areas (e.g., regions, provinces, urban-rural
locations).

However, for costs, logistics, or other criteria introduced by government or other
stakeholders, representativeness could be limited to specific areas (e.g., districts or
provinces). These considerations should be explicitly discussed and justified in order to
facilitate interpretation of results.
 Example 28: Niger education PETS 2008 sampling
■ For a sampling strategy to be valid, the sample ultimately needs to be representative of the
universe it aims to describe.

This objective is rendered more complex in the case of PETS because there are several
units to be analyzed (e.g., central government, provincial and district administrations,
local service providers, staff, service users, households, etc.).
B. Sampling units
■ Depending on the scope of the PETS/QSDS, one needs to distinguish between two or more
types of sampling units.

In a tracking survey, the first statistical unit is constituted by the administrative services
involved in the planning and allocation of the budget in the sector under study. These
services are, for instance, part of the Ministry of Education or the Ministry of Finance
68
(and others, such as Ministry of Planning, etc.) The sub-national levels, which could be
the regional and local authorities, are also concerned.

The second sampling unit (and generally the primary one in PETS/QSDS) is made up of
service providers (e.g., primary schools or primary health care centers) and their staff.

Depending on the scope of the study, one might also include the users of these services
(e.g., patients, students, or households) as a third sampling unit.
■ For each of these units, an adequate sample choice has to be made.
■ In addition, for policy or political reasons, some sub-categories could be important (e.g.,
administrative units such as states or ownership types), and the survey might be expected to
provide separate reliable results for them.

Defining the most adequate partitions for a sample (stratification) entails establishing
some priorities at the design stage, which often will be dictated by policy relevance and
political or cost and time considerations.

The survey design will then have to ensure a minimum sample size within each of these
subgroups (called analytical domains), and the sampling procedure needs to be applied
independently within each of those domains.
■ The sampling design is further complicated when PETS and QSDS are conducted jointly.

Indeed, to adequately measure leakage in a PETS, it is better to sample a relatively large
number of local governments (districts), which implies, with a budget constraint,
reducing the number of service providers sampled in each district.

However, in a QSDS, it could be preferable to interview a greater number of facilities in
a smaller number of districts in order to assess differences in behavior and performance
among types of facilities within districts (Reinikka and Smith, 2004).
C. Sampling strategy
While the sample strategy should be chosen to fit the objectives of the study, it should be
designed scientifically in order for the sample estimates to provide valid inferences about the
sector under study. The most commonly used sampling procedures are random sampling,
stratified random sampling, and multistage stratified random sampling, with or without
probability proportional to size sampling.
 Note 31: Problems with informal sampling
69
■ Random Sampling: With a simple random sampling, one would draw the facilities (e.g.,
schools or health centers) directly from within the population (each individual element, e.g.,
schools or health centers, in the population thus has the same probability of being selected).

In the case of PETS, one would also include the administrative levels that administer
these schools/health centers.
■ Stratified Random Sample (SRS): Stratification divides the survey population into subcategories, which are then sampled independently as if they were independent populations.

It is often introduced given that different administrative levels (e.g., provinces) or
different types of facilities (public-private, rural-urban, etc.) may be targeted.

It is also introduced for cost considerations given that the sample population is generally
large.

Stratification also improves the efficiency of the sample design because it reduces
sample variance and ensures a sufficient number of observations for separate analysis
of different sub-categories. 12

Homogeneity within strata should be sought rather than between strata to increase
precision.

In an SRS, one could randomly draw the schools or health centers equally among each
stratum or draw the sample weighted according to size of the population.
 Example 29: Senegal QSDS 2010 sampling strategy in education and health
■ Multi Stage Random Sample: Multistage sampling could be used to further reduce costs.

Multistage procedures have the advantages of reducing travel time and cost relative to
a single stage sample because the facilities are located in a relatively specific area rather
than spread out evenly over the entire country.

However, multistage samplings yield larger sampling errors (called the cluster effect)
because neighboring facilities tend to have relatively similar characteristics and will
therefore reflect population diversity less than a simple random sample (Grosh and
Munoz, 1996).

In two-stage sampling, for instance, districts might first be drawn randomly (with or
without probability proportional to size, size being defined as the total population or
12
The greater the homogeneity of the elements belonging to the same group increases the efficiency of the sampling
design. (Grosh and Munoz, 1996, p. 99)
70
number of students for school selection). Facilities are drawn from each selected district
(cluster), giving each facility in the area the same chance of being selected, and urban
and rural locations.

If, alternatively, sampling units in the second stage are selected with probability
weighted by their size (i.e., student population or number of nurses within a clinic) and
an equal number of individuals is drawn per sampling unit at the second stage, the end
result is a self-weighted sample.

With probability proportional to size sample, every individual in the universe described
by the sample frame has the same probability of being included in the sample. The
advantage of probability proportional to size is that it eliminates the need to weight the
data during analysis.
Box 10: Example of stratified (by region) two-stage cluster (by district) sampling
Start by stratifying service providers (schools or health facilities) by regions (1 to N);
In the first stage, select a simple random sample of one or more districts in the first stratum
(region);
Draw a simple random sample of facilities within each of the selected clusters/districts;
Repeat for the other strata (region).
■ The sampling strategy needs to be adapted to the allocation rule governing the resource
flows for which quantitative data tracking is sought. In particular, in the absence of a fixed
(hard) rule to allocate the tracked resources, a mixed sample/census approach needs to be
followed in order to measure resource leakage. (See Box 11).
D. Sample weights
While it is not necessary to use weights with a simple random method or with MSRS with
probability proportional to size, results must be weighted when stratified random sampling is
used. The weights of each sampling unit (e.g., providers, pupils) need to be constructed as the
inverse of the probability of selection. These design weights must then be corrected for unit
non-responses and frame problems (see…..)
E. Sample size
In general, PETS/QSDS samples are relatively small, ranging from 150 to 250 service providers,
but some have been larger (e.g., 600 in South Africa 2010). Overall, sample size depends on the
level of confidence and precision sought, population size, and the variability of the parameter
to estimate.
71
Box 11: Sampling in the absence of hard allocating rules: the mixed sample/census approach
In the presence of a hard allocation rule governing the resource for which tracking is realized,
the various sampling strategies proposed above could allow the measurement of (“strict”)
leakage (the ratio between how much the facilities receive and the entitled funds).
However, in the absence of a hard allocation rule, that is, when discretion is used at certain
levels in the supply chain of the resource tracked (i.e., soft allocation rule), one cannot use a
standard sample approach.
The problem with a (pure) sample approach is that one cannot estimate reliable leakage figures
in the presence of soft allocation rules, as one would need complete data on the flows tracked.
Indeed, when measuring “narrow” leakage (as the ratio between how much the facilities
receive and how much another hierarchical level has sent), all resources allocated to facilities in
an area have to be assessed. With only a few facilities visited per district in the pure sampling
strategy, it is not possible to say anything about resource use in that specific district in terms of
reception of materials, financing, drugs, user fees, etc., relative to other districts (or aggregated
at the provincial level).
A potentially adequate sampling strategy to use in the presence of soft allocation rule is a
mixed sample/census approach. The mixed sample-census approach is a sampling design
alternative to a “pure” stratified sample approach (where samples are drawn at each stratum).
In such a strategy, areas (for instance, districts) could be selected in a first stage. In a second
stage, a census approach could be used in which all facilities in the district are inventoried and
then surveyed on a census basis. This strategy is better suited to measure leakage than the pure
sample approach (in which a small number of facilities are visited in each district) in the case
where no fixed-allocation rules are in practice for the tracked resource in the sector and only
“narrow” leakage could be measured.
This approach allows the evaluation of the use of all resources within districts, and hence
measurement of leakage in the case of fixed allocation rules, and presents the further
advantage of producing representative results without the need for weights.
 Example 30: Chad: mixed sample-census approach
At least four issues have to be taken into account in the choice of a sample size (Reinikka and
Smith, 2004, pp.55-56).

First, the sample should be sufficiently large and diverse to represent the various types
of service providers.
72

Second, some sub-categories may need to be oversampled to infer statistically
significant results.

Third, the adequate sample size is a trade-off between minimizing sampling and nonsampling errors. 13 Non-sampling errors, which increase with sample size, are generally
more of a concern than sampling errors in tracking surveys as data are often in a highly
disaggregated form and hence difficult to collect. Enumerator training and field testing
are therefore critical in obtaining high-quality data (see Steps 11-12).

Finally, resource and time constraints must also be taken into account in determining
sample size.
F. Sample frame
Once these choices are made, the development of a representative sample of service providers
requires information on the population under study.
■ The population of the survey is made of all the service providers offering, for instance,
primary care or primary education.

Developing a good sample frame is crucial, as the accuracy of any sampling procedure
also rests on the accuracy of the frame.

The most common problems with sample frames are incomplete or inadequate
coverage, out-of-date data, or failing to include all elements of the target population.
■ While many developing countries do not have a reliable census of service providers, a list of
local service providers, though imperfect, is generally available through the statistical services
of the MoE or MoH or the country’s education or health Management Information Systems
(MIS) (i.e., list of schools or health centers).

If, in certain cases, the coverage of these MIS includes private non-profit and for-profit
facilities, not all types of facilities participate in the reporting procedure. Indeed, lists of
schools or health centers are often incomplete or outdated.

Nonetheless, most tracking surveys tend to use this information to constitute an initial
sample frame of the facility population, which needs to be verified and updated.
13
Sampling error, which decreases with the sample size, is the error inherent in making inference for a whole
population while collecting information on only some of its members. Non-sampling errors, which increase with
the sample size, are all other errors, mainly caused by poor survey implementation. Turner et al. (2001, p. 106)
define non-sampling errors as all survey errors other than sampling errors, including response error, non-response,
interviewer error, data entry and coding errors, errors of concept, and questionnaire design and wording errors.
73
G. Sampling in case of multi-sector survey
In past multi-sector surveys, there have been frequent problems of sample design, as the same
enumeration areas (e.g., districts) were often chosen for both sectors in order to reduce costs.
However, such an approach necessarily means that some of the facility samples are biased, as
the sample reflects only the facility populations of one of the sectors.
If more than one sector is covered by the tracking exercise or facility survey, separate sample
frames have to be constructed and separate sampling strategies have to be developed for each
sector in order for the samples to be representative of the facility population of each sector.
H. Some examples of sampling practices
■ These objectives and constraints have been resolved differently in past PETS/QSDS.
However, a number of common characteristics emerge.

As mentioned above, PETS/QSDS samples tend to be relatively small, generally from
150-300 service providers, allowing a balance between sampling and non-sampling
errors.

PETS/QSDS samples are generally drawn in two or three stages and stratified by subgroups, such as provinces and urban rural locations. In the first stage, districts are drawn
randomly in each province. In the second stage, a certain number of facilities are
selected in each selected district (cluster) and urban and rural locations.

The following examples present overviews of sampling procedures in various countries.
 Example 31: Various sampling methods used in Africa PETS/QSDS
 Example 32: Sample stratification in Mali education PETS 2004
 Example 33: Uganda education PETS 1996: sample strategy
 Example 34: Uganda 2000 health PETS: sample strategy
 Example 35: Mozambique PETS/QSDS: sample strategy
74
Box 12: Illustration of alternative stratified sampling strategies in Zambia
In the 2002 Zambia PETS/QSDS, four alternative stratified sampling strategies were considered
to assess leakage and school performance through the testing of grade V student outcomes.
Scenario A: To obtain information on school performance and student outcomes representative
at the national, provincial, district, and rural/urban levels, the following strategy could be
followed:
- Two-stage stratified sample (proportional to student population) by province, district, and
urban/rural location in which sampling units are the schools and the pupils.
- Schools are sampled and student outcomes are measured within all provinces (nine) and
districts within the country (72).
-In the first stage, schools are selected within the district (the lowest stratum) using
probabilities proportional to the estimated size of grade V enrolments per school.
-In the second stage, up to 20 pupils are randomly selected from each sample school.
Scenario B: For costs and logistics reasons, one could choose a sampling only representative at
the provincial and urban/rural location levels but not at the district level.
-Provinces and urban/rural locations are the stratification (and reporting) variables, while
districts become clusters in a multi-stage sampling procedure. (The sampling units are the
districts, the schools, and the pupils).
-In the first stage, in each province (nine), districts might be drawn using student size weighted
random selection.
-In the second stage, schools are randomly selected using the same approach and in a third
stage, pupils.
Scenario C. Alternatively, to further reduce costs, keep district as the stratification and
reporting level (as well as urban/rural locations), and use provinces as a cluster in a multi stage
sampling.
-The primary sampling units are the provinces, then the schools, and then the pupils.
-The advantage of such an approach is to cover all districts within the selected provinces,
allowing for within-province district comparisons; that is, one could assess how schools or
pupils are affected by differences in characteristics of district levels in terms of actions or
structures.
-However, in this approach, one loses somewhat in terms of sampling error for national results.
Scenario D. In the Zambia 2002 PETS/QSDS, in addition to districts and urban/rural location, a
further stratification variable was introduced: whether the province was part of a
decentralization process (of fund disbursement and decision making).
-Hence, in the first stage, instead of random samples of provinces, two provinces from each
category were selected.
-Schools were then drawn in all the districts proportional to size.
Source: Das et al. (2002).
75
I. Linking PETS/QSDS to household surveys
■ Linking PETS/QSDS with household surveys could bring important value additions. In a linked
survey, the facility survey provides information on the service supply environment to which the
population included in the household survey is exposed (Turner et al., 2001).

It allows investigation on how population behaviors and outcomes are affected by the
presence of service providers. 14

This linkage is also fundamental for the purpose of using PETS/QSDS for specific
program impact assessment.
■ Sampling methods differ when linkage is done with a household survey. In such cases, the
design of the sample requires the adoption of the same sample areas as used to generate the
household survey.
■ While the link with household surveys brings important added value, it also constrains the
facility sample in several ways and could affect its representativeness. Indeed, there is a tradeoff in the linked survey approach as a choice has to be made regarding whether the sample is to
be representative of the population of individuals or of the facility population.
■ A few tracking surveys in Africa have chosen the route of linkages.
 Example 36: Ghana PETS 2000 linkage method
 Example 37: Chad PETS/QSDS 2004 linkage with household survey
 Example 38: Zambia 2004 matching school and household data

When a household survey is not directly incorporated in the PETS/QSDS, teams should
try to establish contacts with LSMS (Living Standards Measurement Survey) teams or
other household surveys in the country.

If a household survey is almost completed, it could be useful to establish links with it.
Whatever the route taken (stand-alone or linked surveys), the sampling strategy must
again be scientifically chosen in order for the sample estimates to provide valid
inferences about the sector under study.
14
Non-linked surveys could be used to conduct multivariate analysis, but in this case, the household data need to
be aggregated at a given administrative or geographical level to match the outcome of interest with program data.
Consequently, the unit of analysis will no longer be the individual. (Turner et al., 2001, p.25)
76
J. Summary

PETS/QSDS samples are small, generally from 150-300 service providers, to balance
sampling and non-sampling errors.

PETS/QSDS samples are generally drawn in two or three stages (and are also generally
stratified by sub groups, such as provinces and urban/rural locations). In the first stage,
districts are drawn randomly in each province/region. In the second stage, a certain
number of facilities are selected in each selected district (cluster) and urban/rural
location.

When tracking resources governed by soft (discretionary) allocation rules, one should
consider using a two-stage mixed sample-census strategy, which combines a stratified
sample in the first stage (to choose districts, for instance) and a census in the second
stage (within districts). This allows the evaluation of the use of all resources within
districts, and hence the measurement of leakage in the case of soft allocation rules, and
presents the further advantage of producing representative results without the need for
weights.

The choice of linking facility and household surveys depends on the objective pursued.
Still, there is potentially huge added value from linking supply and demand results to be
able to examine the impact of service provision in terms of population outcomes.
> Step 10. Preliminary pilot
Once survey instruments and sampling strategy are identified, a preliminary pilot of the
instrument should be conducted to test all questionnaires among a specific number of units,
ownership types, or geographical regions.
■ The preliminary pilot consists in administering the questionnaires to a small number of units:
frontline facilities and various administrative levels covered in the survey (including
provincial/regional or district, facilities, staff, users, households, etc.) using all the instruments
developed in urban and rural areas.

The units visited should be selected outside the sampled providers and administrative
units chosen for the full-scale survey.
■ This test of the instruments should allow the reassessment of the choice of tracking flows, as
well as the quality and consistency of data.

In particular, quantitative data questions, financial data, inputs, and outputs, which have
been customized to the country’s administrative system, should be carefully examined.
77
■ Wording of questions, ambiguous responses, answer codes, etc., should be revised at this
stage.

This exercise should also be accompanied by the verification and revision of the
sampling procedure.
■ Following the preliminary pilot, instruments should be revised.
■ Revised survey questionnaires should be circulated anew among stakeholders for comments.
78
Phase 4. Implementation/Fieldwork
> Step 11. Training and survey protocols and manual preparation
Once the consulting firm responsible for survey implementation has been selected (see step 3B)
and survey instruments have been pre-piloted, preparation for survey implementation could be
put forward. A training workshop need to be held in the weeks before the planned survey
fielding to enable enumerators and supervisors to familiarize themselves with the survey
instruments and field work. The training workshop requires at least one week and ideally two of
training, and should be completed by a field pilot of the instruments (see step 12).
■ Prior to the training, a detailed enumerator and supervisor manual, including a survey
protocol describing the main procedures for survey implementation, should be prepared. The
manual should discuss the questions of each survey instrument one by one, explaining the
rationale of each question and potential interpretation problems. The manual should be used in
the training sessions and will also support the teams’ field work.
■ The training workshop consists in extensive classroom and on-site training (see step 12) of all
survey enumerators and supervisors to ensure that all questions and procedures are well
understood.

The classroom training should educate prospective interviewers on the role and context
of the questionnaire; this is of particular importance given the sensitive nature of the
subject and the need for confidentiality.

As part of classroom training, prospective enumerators should then interview each
other to ensure that all questions are fully understood and that questions can be raised
and addressed before field deployment. Enumerators should also conduct initial
interviews with respondents in “teams” or paired with a supervisor.
■ See the PETS/QSDS Web Portal for examples of survey manuals
> Step 12. Full pilot phase
During the training workshop, especially in the second week, a minimum of three to five days of
field pilot of the questionnaire should be carried out by the enumerators and supervisors to
test all questionnaires and to train enumerators and supervisors in the field.
■ Field testing of instruments is essential for increasing the likelihood of obtaining good quality
survey information.
79

A test on about 5 percent of the sample, including all types of respondents and levels,
should be adequate.

All enumerators and supervisors should be required to perform full-length field tests of
the instruments.

Debriefing of teams and further in-class training should be conducted following each
day of field testing to ensure that all questions and procedures are understood.

Enumerators and supervisors who do not perform satisfactorily should be replaced.
■ The units visited during the full pilot should be selected outside the sampled providers and
administrative units chosen for the survey.
■ Following the pilot, a final revision of instruments should be carried out.

This includes revising wording of questions, ambiguous responses, format of the
questionnaires, and answer codes.
> Step 13. Data entry
■ Data entry programs should be written following the completion of the questionnaires and
should be tested during the survey pilot phase.
■ A training workshop should be held for data entry operators and data entry supervisors to
ensure proper understanding of the instruments, data entry programs, and verification
mechanisms.
■ Data entry should start at the beginning of survey implementation and should be completed
promptly following the end of data collection.
■ A standard state-of-the-art data management program, such as CSPRO should be used
(available for free on-line).
■ Data entry programs should include a unique identification code for each questionnaire and
unit interviewed in order to match responses within districts and regions.

Various controls should be introduced in the data entry program in order to reduce data
entry errors, as well as validation mechanisms to detect data inconsistencies.

In particular, standard tests of fields and inconsistencies should be included in the
programs to identify outliers for each variable.
80
■ Inconsistencies and potential errors in data detected should be verified while the survey is
still being fielded.

A return to the field may be necessary if information in some questionnaires appears to
be doubtful.
■ To minimize data entry errors, double-entry procedure is recommended to identify and
correct errors. A special program is used for double-entry, to allow entering the data a second
time, to check for discrepancies and correct data.
> Step 14. Survey implementation
■ Team assembling: The survey firm needs to assemble a team of high-quality enumerators
and supervisors with adequate experience in facility surveys.
■ Team composition and management structure: Survey teams should be composed of at
least two enumerators per team, who will be administering the various survey modules to the
facilities. In order to oversee the survey, a group of supervisors needs to be present in the field
while the survey takes place. These supervisors should be responsible for managing and
supervising teams of enumerators (generally two enumerators per team) who will administer
the various questionnaires.
■ Survey timing: The survey should be fielded ideally two to three months after the end of the
fiscal year (for accounting books to be closed), and quantitative tracking should cover the last
completed fiscal year.
■ Field deployment strategy: Before survey implementation, the survey firm should submit an
adequate team deployment strategy to allocate teams and supervisors in the field in order to
achieve, in the most efficient manner, the service facilities and other administrative units while
respecting supervision objectives and in the delays set by the time schedule. Possibilities of
return visits need to be accounted for in the survey schedule.
As the modules of PETS/QSDS require different types of information to be collected from
different levels and individuals, the time required is estimated to be about two days per facility
when a repeat visit to collect absenteeism data is applicable.
■ Replacements: Accepted sample replacement methods should be used to ensure that the
sample size is kept as close as possible to that originally defined.
81
■ Coordination with authorities: The survey firm should closely coordinate its activities with
the relevant authorities (Ministry of Education, Ministry of Health, district officials, etc.)

This coordination seeks to facilitate easy access and ensure effective communication
and harmonious relations between the parties involved.

Coordination with line ministry officials at the central level should be the responsibility
of the core PETS/QSDS team to ensure that the survey firm receives adequate support in
the field.

In particular, prior to field work, facilities and administrative levels that are part of the
sample should be informed by their administrative officials of the survey visits to ensure
collaboration.

Introductory letters should also be provided to enumerators and supervisors.
■ Permits and ethical clearance: Well before the survey implementation period, the survey
firm should obtain all the necessary permissions from the authorities for implementing the
survey.

The survey firm is also responsible for adhering to local formalities and obtaining any
required permits related to the logistics of the pilot implementation.

In some countries, obtaining these permissions could take weeks or months, as ethics
committee clearance could be required to collect information in the health or education
sectors.

Delays in obtaining these government permissions have led to delays in field
implementation in some countries (e.g., Senegal 2010).
■ Survey supervision and quality control: In collecting the data, the survey firm must ensure
close supervision and oversight, as this is a significant factor in ensuring the collection of highquality data. The fieldwork supervisors must devote time and attention to supervising the work
of the enumerator teams.

In addition, random visits to enumerators in the field are essential to ensure quality
control and coherence in the interpretation of questionnaires. In particular, random
checks of questionnaires and data quality should be done throughout the survey
implementation.
82

Real-time quality control procedures should be implemented. Every day, completed
questionnaires should be carefully reviewed by supervisors and the main management
team for incompleteness, incoherence, etc. Feedbacks and questions should be relayed
immediately to enumerators for clarifications, corrections, and potential return visits to
collect missing data.

A test of these verifications and quality control procedures should be held and an
account of the results should be required at the pilot exercise stage.

The survey firm should be required to provide a system of ex-post interview verification
and/or random spot checks to ensure quality control of enumerators’ work. For
instance, it could be useful to have a percentage of facilities revisited.
■ Core team supervision: The project manager and PETS/QSDS core survey team should closely
supervise survey implementation in the field.

Some members of the core survey team should be in the country for the whole survey
implementation and data entry period. It is recommended that surprise field visits be
conducted.
■ Launching of the survey: The full-scale survey could be conducted once all the previous
conditions are met.
■ Completed questionnaires: Completed questionnaires should be transmitted to data entry
operators weekly or bi-weekly.
83
Phase 5: Analysis, Report Writing, Recommendations and
Dissemination
> Step 15. Data cleaning, analysis and report writing
■ Data cleaning and analysis should be done shortly after the end of data collection. A Survey
implementation report should be produced discussing the process of data collection and any
problem encountered during the survey and data entry and an evaluation of the quality of the
survey data. Also, an Analytical Report should be produced promptly (drafts and final versions)
clearly identifying and communicating the specific findings of the study.
A. Data Cleaning and Survey Implementation Report
■ Data cleaning procedures should include for instance:





Questionnaires and variable coding
Range checks: out of range values should be reviewed
Skip patterns
Consistency checks across related questions/variables
Standard tests to identify outliers for each variable;
■ See the PETS/QSDS Web Portal for detailed data cleaning procedures.
■ The survey firm should provide the initiating agency with a master data set containing all
survey data (raw and clean data) and secondary data with all associated documentation
including a codebook and data dictionary. All of these sources should be merged using unique
unit identifiers. The data set should be in a standard software format (e.g. Stata, Excel, SPSS).
■ The data cleaning procedures should be described and cleaning program codes provided as
part of the survey implementation report,

The data quality section should include tables of summary statistics for all variables
(number of observations, mean, min, max, etc), including a short evaluation of the data
quality indicating any weaknesses or other issues that will be relevant in the analysis.
Indicators of data quality could include response rates for various key variables.
■ Data should be stored in a secured location and made available rapidly to other researchers.
■ The completed questionnaires in their original paper format should also be submitted by the
survey firm and properly stored.
84
B. Data Analysis and Reporting
■ The research team composed of the project manager, data analysts and experts in
econometrics and statistics should analyze clean primary survey and secondary data using
statistical software such as Stata or SPSS and produce an Analytical report. Rigorous data
analysis techniques should be used and discussed to ensure credibility of the results.
■ The main objectives of the Analytical report are to present survey results and answer the
policy questions initially formulated. The analytical report is a comprehensive report
summarizing all the work related with the study preparation and institutional mapping report
(see Steps 4-5), survey design and implementation and includes detailed description of the data
analysis, econometric specifications and presentation of the results and policy implications. The
analysis report should incorporate key information of the institutional mapping report before
turning to the survey implementation strategy and survey results. The data analysis should
provide convincing evidence on bottlenecks in the service delivery chain, on equity issues
within the sector and on the sources of these problems.
■ The structure of the report will depend on the types of programs, policy questions or sectors
analyzed and the scope and research questions of the study. For instance, among other things,
the study could focus on measuring leakage and delays in resource allocation at various levels,
or how resource equity or service performance vary among sub regions or sub groups.
■ Good examples of Analysis reports are available, for instance (Education PETS/QSDS): Das,
Dercon, Habyarimana and Krishnan (2004a) « Public and Private Funding Basic Education in
Zambia: Implications of Budgetary Allocations for Service Delivery”; (Health PETS/QSDS): Picazo
and Zhao (2009) ”Results of the Expenditure Tracking Components of the PETS/QSDS 2005-06”
in “Zambia Health Sector Public Expenditure Review”; (Education and Health PETS): World Bank
(2008) ”Niger: Public Expenditure Tracking Survey, Education and Health”; (Health QSDS):
Lindelow, Reinikka and Svensson (2003) “Health Care on the Frontlines: Survey Evidence on
Public and Private Providers in Uganda”.
■ Box 13 outlines the suggested content of the Analytical report.
■ Some of the main components of the Analytical report are as follows:



The report should present an overview of the study objectives, scope, methodology,
sampling and data collection strategy.
The section on the sampling should outline the sampling strategy (including expected
and final sample for each types of units) as well as power calculations and population
weights.
The report should also discuss data collection and any challenges faced during survey
implementation.
85




The report should also include background information on the sector or program under
study. It should present an overview of public expenditures, the importance of the
sector or program and the resources officially mobilized. The performance and state of
service delivery in the sector or program should be discussed; the structure of public
hierarchy, roles and responsibilities of main levels and links in the expenditure chain
toward service providers. It should discuss the budget process, timing and the roles of
various actors and levels.
It should describe ways in which resources are mobilized and channeled, including
allocation principles and mechanisms at various levels (official and effective rules);
provide descriptive and analytical information on various units analyzed in the survey
work (regional, district and local governments/administrations, facilities, staff, clients,
etc.),
The section on study finding should examine selected tracking flows (salaries, non
salaries, in-kind); it should identify funding levels from various sources disbursed
through the expenditure chain, the resources received at the various levels down to
frontline providers and beneficiaries (amounts and percentage), and the ways in which
these resources are used at various levels.
Descriptive statistics should be produced on the main components of the contents of
the various primary and secondary data. The report should also provide adequate
information on, among other things:
o
o
o
o
o
o
o
o
Main characteristics of service providers and other units in the delivery chain;
Financial resources in the sector/program and main sources;
Decision making procedures at various levels;
Allocation mechanisms for the main resources tracked;
Main problems in budget allocation and execution;
Supervision and accountability;
Characteristics of personnel and personnel management;
Measures of intermediate output, final output and outcomes.

The report should identify variations in allocation or allocation rules for all tracked
resources relative to the expected or official allocation rules. In particular, at each stages
of the expenditure supply chain, leakage should be identified by comparing amount of
resources disbursed by the higher level in the supply chain to the amount received by
the lower level. If feasible, overall leakage should be quantified and sources of such
resource dissipation analyzed.

It is important to distinguish lack of quantifiable measurement of leakage associated
with weak survey design or implementation, unavailability or poor quality of data, from
lack of leakage associated with an efficient program expenditure system.

Other inefficiencies and bottlenecks in the service delivery chain, such as delays, should
be identified and analyzed;
86
Box 13: Outline of a PETS/QSDS Analytical Report
Executive Summary
1. Introduction:
-Motivations and objectives of the PETS/QSDS
-Organization of the report
2. Methodology
-Overview
- Main sources of information
- Sample strategy and expected versus final samples
3. Description of the sector/program
- Sector/program outcomes
- Objectives of the sector/program
- Organizational structure of the sector/program
- Budget process and allocation rules
4. Resource allocation in the sector/program (PETS)
- Resource allocation flows in the sector/program from various sources
- Budget allocations versus release of resources at various levels (central, district, etc.)
- Overall resource availability at the various levels (District, Local, Facility)
- Measurement of leakage at various levels
- Delays and other inefficiencies in the service delivery chain
- Equity issues across categories
- Other specific themes
5. Frontline service providers and quality of services (QSDS)
- Characteristics of the facilities, infrastructure and equipment
- Staff characteristics
- Human resource management, incentives and absenteeism
- Management of in-kind inputs
- Service output and quality
5. Analysis
- Bottlenecks in the service delivery chain
-Potential sources of inefficiencies and inequities
6. Conclusion and recommendations
- Challenges in the sector/program and at the service delivery level
- Recommendations
ANNEX A: Survey Methodology and Implementation
A. Sampling Strategy and design
B. Field Work
C. Data entry and coding
D. Survey experience
E. Lessons learned and recommendations
ANNEX B: Survey Instruments
REFERENCES
87

The report should also identify inequities in resource allocation (financial, human, inkind) and services among sub-regions or sub-groups examined.

The report should assess information and reporting systems, and accountability
mechanisms at various levels.

Specific research questions and sector or program specific topics analyzed in the study
such as private versus public contributions, user fees levels and management or the
effect of decentralization, should be discussed.
■ Rigorous statistical and econometric techniques should be used for instance multivariate
regression analysis to analyze different components of the sector/program to assess the
contribution of various factors and characteristics, such as governance and accountability
mechanisms, location and distance, etc. in explaining leakage and other inefficiencies such as
delays and test various hypotheses.
■ The report should clearly identify and communicate the specific findings on various
inefficiencies and inequity in resource allocation in the sector or program.
■ The report should also provide detailed recommendations on how to strengthen the sector or
program internal information and oversight system for public expenditures and or to improve
citizen/clients potential power over service providers and service quality (See Step 16).
■ The report should be peer reviewed and subject to broad consultation before a final version
is finalized. In particular, the draft report should be discussed with stakeholders and adequately
edited. They should also be distributed among civil society (See step 17).
> Step 16. Preparing and formulating recommendations
The biggest challenge for tracking surveys beyond these methodological issues and new
applications is to be able to translate their findings and contributions into policy
recommendations, and ultimately into policy reforms and institutional changes, in order to
improve quality of services and population outcomes.
Indeed, tracking surveys are a means to achieve an end. The information on incentives and
deficiencies in organizational structures and rules should ultimately be used to identify policy
reforms and help implement a reform agenda in client countries.
■ Some of the main recommendations that could be put forward on the supply side concern
the reduction of information asymmetry and strengthening the accountability relationships,
88
which are at the root of institutional deficiencies within service delivery systems. The areas
identified for reforms in various previous survey reports include:
i) Speeding up budget execution at various levels of the delivery chain;
ii) Improving communication and information pass-through (dissemination of information
is a general problem between the administrative levels, including with the population);
iii) Increasing inspection and monitoring at all levels of the service delivery system;
iv) Improving internal controls, in particular accounting and reporting systems, in order to
enhance transparency and accountability
v) Improving logistic systems for managing materials, supplies, and in-kinds (e.g., drugs) at
the decentralized level to allow adequate reporting and controls;
vi) Providing additional training for decentralized government levels and service delivery
units (problems often result from a poorly trained personnel);
vii) Establishing mechanisms and incentives in the system to make the service delivery
system more client-driven at all levels.
■ In addition, there is growing evidence that citizen participation in service delivery and better
information can help improve outcomes, especially using mechanisms that enable clients to
monitor and directly discipline service providers (World Bank, 2003; Banerjee et al., 2006).
Various recommendations concerning improvements in governance by giving clients potential
power over service providers should be proposed and implemented. The service delivery
reforms that should be considered include the display of expenditure information at the facility
level, following the successful use of such an approach in Uganda.
For instance, information campaigns in newspapers and on radio about public transfers toward
targeted facilities (as used in Uganda and Tanzania) could be recommended for patients,
parents’ associations, and communities to be better informed and allowed to voice their
demands.
> Step 17. Results dissemination, policy reforms, and future research
PETS/QSDS results should be disseminated promptly among government ministries and units,
as well as NGOs and civil society organizations, following the report completion to increase
impact in terms of service delivery quality and efficiency and population outcomes.
A. Results dissemination
■ Dissemination strategies should be developed. These could include information activities and
take the form of seminars or workshops to present the findings and implications in public
gatherings and among the stakeholders.
89

During the activities, elicit views on how to promote reforms and improvements in
service delivery and population outcomes.
B. Policy reforms
■ Information collected should be analyzed and used for planning and reform processes. Policy
reforms could be targeted to improve the efficiency of public expenditures and quality of
services, and their impact on the wellbeing of the population. These include:

Proposing plans of action to revise budget allocations in order to improve efficiency,
effectiveness, and equity of public expenditure;

Improving budget execution by obtaining information on various problems in budget
execution (capacity, reallocation, etc.) at different stages;

Enhancing government systems of recording, reporting, and information systems at
various levels in the administrative system toward service providers (for financial and
non-financial resources);

Strengthening relationships of accountability between the government, providers, and
citizens by improving information on actual resource flows and quality of service
delivery;

Strengthening domestic capacity, inside and outside government, to conduct monitoring
and evaluation activities.
C. Future research
■ In a multi-year program perspective, PETS could be used as part of a monitoring and followup mechanism. In particular, PETS and QSDS could be part of a larger strategy that seeks to
improve equity and efficiency of public management where the initial PETS could be used as a
baseline to measure the allocation of resources and performance in service delivery and for
paving the way for more comprehensive follow-up surveys. Subsequent PETS focusing on other
elements of service delivery performance could be proposed, for instance, on service quality,
population outcomes, and user satisfaction (World Bank, 2008).


Repeated PETS could allow benchmarking progress in implementation of reforms over
time.
In a long-term perspective, PETS could also be part of a process aimed at improving
public expenditure efficiency and equity by focusing on capacity building.
90
■ The following research questions may especially be examined:
i)
Benchmarking leakage of resources and better explaining capture of funds (including
delays, etc.)
Robust baseline surveys of the sector studied should be devised to diagnose and
analyze the process by which public resources are translated into services and to
identify shortcomings in the supply chain and among frontline providers, particularly
leakage of resources. Factors explaining leakage that could be examined notably
concern users’ bargaining power, service provider remoteness, size of facilities,
access to infrastructure, incentives, etc. Developing a core set of questions for PETS
to be applied across countries should be a priority.
ii)
Benchmarking absence rates and better explaining provider behavior
Service provider absenteeism has been found to be a widespread problem in a
number of countries. It is also an effective way to get policymakers’ attention to
performance in service delivery. It would be worthwhile to document the extent of
the problem and identify factors that are associated with worker absence. Baseline
surveys could seek to measure absenteeism in various facility types, ownership
categories, locations, etc., to measure time on task for different types of workers,
and to better explain reasons for absence rates. For instance, in Zambia, Das et al.
(2004) found that teacher absenteeism had more to do with health problems than
with shirking.
iii)
Evaluating interventions to combat leakage, absenteeism, and other service delivery
failures with a focus on human development and other sectors
Impact evaluation of pilots or reform programs such as information campaigns,
citizen report cards, monitoring mechanisms, and incentive schemes for service
providers could be carried out. Experiments to promote local participation and
collective action through advocacy could be designed and evaluations could be
carried out to assess their impact on local participation, provider behavior, and final
outcomes. Such an experiment could involve, for instance, the establishment of new
teaching supervision programs that could be put forward in schools, while in others
the program, if successful, would be phased in later. Student learning results could
then be compared in the two groups of schools.
 Example 39: Randomized evaluations of potential experiments
Experiments involving monitoring mechanisms and incentives could be implemented
to reduce worker absenteeism and improve outcomes. Such actions or other
mechanisms designed to monitor and provide high-powered incentives to workers
could be examined with a view to improving citizens’ control over service providers.
These programs could involve improving the flow of information between citizens
91
and service providers or between citizens and public officials. Alternately, they could
involve the community in hiring and firing service providers.
Such prospective impact evaluations help identify which community or citizen-based
interventions lead to increased provider time on task, service quality, and outcomes.
Experiments to test worker incentives empirically, community or external
monitoring, or other actions could be carried out to determine which program leads
to lower absenteeism, more time on task, and improved outcomes.
 Example 40: Experiments to test worker incentives
iv)
Explore not-for-profit providers’ motivation and behavior using these types of tools.
In particular, analyze intrinsic motivation of public, NGO, and religious providers as a
major factor in social service provision.
v)
Link demand and supply sides of service delivery to develop a better understanding
of development outcomes.
Future research should target linkages between facility survey data and household data
in order to allow measurement of final outcomes at the household level and to link
service and human development and other outcomes. PETS/QSDS, for instance, can
comprise a household survey module. Alternatively, linkages with another self-standing
household survey can be established.
92
NOTES
 Note 1: Other micro level governance tools
Other micro level governance tools include, in particular, Staff Absenteeism Surveys (SAS),
Citizen Report Cards (CRC), and observational studies such as health vignettes. SAS focus on
the availability of teachers and health practitioners on the frontline and identify problems with
their incentives (see, Rogers and Koziol 2011). CRC are instrument to promote citizen
awareness and participation. They collect information from users about experiences of service
quality, disseminated back to citizens/users so they have information about their community
experiences of quality and efficacy of service delivery. Observational studies aim to measure
the quality of services, proxied for by the level of effort exerted by service providers. Health
vignettes use unblind standardized case studies involving an actor to test the level of
knowledge and competence of service providers. For guidance on using PETS to monitor small
scale programs by NGOs, see Koziol and Tolmie (2010). See also Amin, Das and Goldstein (2008)
for an overview of a range of tools for measuring service delivery.
←
 Note 2: Public Financial Management Economic and Sector Work Products
There are a number of diagnostic economic and sector work (ESW) products that the World
Bank, the IMF, partner organizations and markets have developed in recent years that can help
in assessing the effectiveness of PFM systems. Many of these have been associated with
aspects of fiduciary interests or creditworthiness. These products fall essentially into two
categories:
Non-standardized analytical assessments of a country’s PFM system, using such diagnostic
products as:
•
Public Expenditure Reviews (PERs), which analyze the country’s fiscal position, its
expenditure policies (in particular, the extent to which they are pro-poor), and its
expenditure management systems. While PERs vary considerably in coverage, they may
also examine institutional arrangements for public expenditure management on
national and sub-national levels, touch on the issues associated with the size of the civil
service wage bill, and revenue policy and administration - aspects of direct relevance for
Central Finance Authorities (CFA).
•
Country Financial Accountability Assessments (CFAAs), which evaluate the strengths
and weaknesses of accountability arrangements for managing public resources in areas
like budgeting, accounting, and audits, and also identify the risks these may pose to the
93
use of World Bank funds. CFAAs also often analyze the role of CFAs in the country’s
public and private sector financial accountability framework.
•
Country Procurement Assessment Reviews (CPARs), which examine public
procurement institutions and practices in borrower countries, which in some countries
might include the CFA.
•
Public Investment Management Reviews (PIM Reviews), which examine the efficiency
of the public investment management function. As of now, these have been applied in
six countries, but they are likely to be applied more widely.
•
Reports on the Observance of Standards and Codes (ROSCs), while templated, may not
be prescriptively standardized. Fiscal ROSCs, as most pertinent for CFAs, may examine
the clarity of roles and responsibilities in fiscal management, budget processes, as well
as the disclosure and integrity of information.
•
IMF Reports, particularly ‘Red Cover’ technical assistance reports on PFM, and other
reports such as safeguard assessments of central banks (to be used under confidentiality
arrangements).
•
Client Country and Development Partner Reports, which include reports from official
oversight agencies in borrowing countries (such as parliament or supreme audit
institutions), as well as PFM diagnostic work undertaken by other development
partners, such as the Asian Development Bank Diagnostic Study of Accounting and
Auditing, European Commission ex ante assessments of country financial management,
and UNDP Assessments in Accountability and Transparency (CONTACT).
•
Institutional Governance Assessments (IGAs) that might touch on PFM and the
institutional capacity of CFAs.
•
Institutional and Governance Reviews (IGRs), which evaluate the quality of
accountability, policymaking, and service-delivery institutions within a given country
from a broad governance perspective and often deal with the CFA in diagnosing the
shortcomings of formal PFM systems.
•
Other studies, in which finance functions are dealt with, but may not be the core of the
studies, such as Governance and Anti-Corruption (GAC) diagnostics which look at
aspects of governance and occasional political economy studies, and other analytical
work (see for instance, cash rationing in Zambia, political economy in Uganda).
Standardized assessments, with a quantitative element, of a country’s PFM system, intended
to generate standardized benchmarks for comparative purposes, including:
94
•
Public Expenditure and Financial Accountability (PEFA) indicators, a collaborative effort
of seven development partners to oversee the development of a universally accepted
tool to assess a country’s PFM system against 28 indicators in three areas of budget
credibility, comprehensiveness, transparency, and the stages of the budget process (i.e.,
planning, execution, reporting and audit).
•
Country Policy and Institutional Assessments (CPIAs), which rate a country’s
performance against 16 criteria representing different policy and institutional
dimensions of a country’s poverty and growth strategy, including the quality of
economic and budgetary management.
←
 Note 3: PETS as one instrument within PFM toolkit
PETS helps focus on the links between effective Public Financial Management (PFM) and actual
service delivery. Consequently, it is potentially valuable for the cross-cutting dialogue on PFM,
often conducted with stakeholders in ministries of finance. It is also particularly relevant to the
sectoral dialogues for counterparts in health, education, water, as well as other infrastructural
and administrative service (including legal) delivery sectors. The implementation of a PETS has
itself become part of a benchmark for achieving basic performance scores within the Public
Expenditure and Financial Accountability (PEFA) indicator concerned with frontline service
delivery financing. More compelling and accessible information concerning budget execution
for service delivery can most importantly help draw in enhanced popular demand across service
delivery beneficiaries and civil society organizations (CSOs). Thus, PETS is one instrument
among the overall PFM toolkit which includes PER, PEFA, CFAA, etc.
←
 Note 4. PEFA and PETS
PEFA, which is a multi-donor program assessing the quality and efficiency of recipient country’s
public finance, is composed of 32 “high level” indicators providing a panoramic view of public
finances in a country. One of these indicators, PI-23 “Availability of information on resources
received by service delivery units”, is essentially built on PETS contribution and activities in the
country. The indicator could take four values (A to D) depending on the availability of
decentralized information. A score of A is attributed to a situation where data collection
systems provide reliable information on resources received by primary schools and health
clinics with an annual report. A score of B is attributed to a similar situation in education or
health, or when PETS have been carried out in both sectors in the last three years. C is
attributed for PETS in the last three years in education or health. D is for a situation of no data
collection on resource transfers to service delivery units in the last three years.
←
95
 Note 5: Stand alone PETS?
PETS are useful for drilling down for information on resource flows for one or two selected
sectors. PETS are usually done in conjunction with other PFM diagnostic reviews and studies.
However, there are pros and cons for doing a stand-alone PETS. First of all, PETS usually take
longer to undertake than typical PERs, and the timing of the two needs to be sequenced and
synchronized which is often not easily done. Information gathered by PETS usually feeds into
the PER analysis, thus enriching it with sector and micro level details.
←
 Note 6: Some PETS/QSDS’ motivations and objectives 15
TABLE 1: SOME OF THE MAIN MOTIVATION AND OBJECTIVES OF PETS-QSDS
Purpose
Diagnostic
Diagnostic
Monitoring
Monitoring
Monitoring
Evaluation
Evaluation
Monitoring
Evaluation
Evaluation
15
Objectives
-To measure the difference
between the intended resources
and resources actually received at
various tiers, including service
providers
-To track expenditures for social
services
-To identify delays and leakages
-To demonstrate surveillance and
control of the expenditures to the
civil society and external donors.
- Monitoring of pro poor
expenditures in the context of a
PER
-To assess the efficiency of budget
execution
-To measure leakage
-To identify bottlenecks in resource
flows
-Measure leakage from the two
main education funding programs
(one administered by the World
Bank).
-Evaluate the effectiveness of HIPC
funds on tuitions and on
enrolment
-To shed light on the effectiveness
of decentralization policies
- To evaluate the impact of the
information campaign on leakage
levels
-To assess leakage levels at
different levels in the service
provider supply chain and
procurement process.
-Evaluate delays in budget
execution
-Evaluate clients’ service
satisfaction
-Evaluate impacts of reform
Motivation / Context
• Output and outcome indicators
remained stagnant despite
substantial increase in public
spending on basic services
Examples
Uganda 1996; Rwanda 2000,
2004; Sierra Leone 2000, 2001;
Senegal 2002; Namibia 2003;
Chad 2004; Kenya 2004
• Increase in the budgetary
allocation to the social sectors,
particularly health and education
Rwanda 2000
• Major gaps exist in access to and
utilization of basic services by the
poor
Ghana 2000
Tanzania 2001
• In the aftermath of political crisis,
usage of HIPC funds to pay for
tuition fees of primary school
children to boost enrolment rates.
Madagascar 2003
• Implementation of an
Information campaign to provide
citizens’ voice
• National governance plan to fight
corruption, reduce poverty and
increase efficiency of public
expenditures
• Priority given to access to basic
services
Uganda 1999, 2000
• Reintroduction of user fees in
education
Madagascar 2005
Source: Gauthier (2006) for further details.
96
Cameroon 2003
Following are brief reviews of the motivations, objectives and means associated with some of
the PETS and QSDS implemented in Sub-Saharan African countries.
Tanzania: Tanzania has implemented four PETS/QSDS to date. The first, in 1999, covered the
health sector and primary education, while the second, in 2001, also examined rural water and
roads, judiciary, agriculture research and HIV/AIDS (REPOA/ESRF, 2001). As in Uganda, the
motivation for the tracking surveys was linked to the lack of information at the central level on
resource use through the various levels of the administrative apparatus. It was strongly
suspected that public resources were not used for stated purposes.
The first survey focused on non-wage health and education expenditures to hospitals, health
centers and primary schools, through district administrations. The second survey, which was
part of a Public Expenditure Review (PER), focused on pro-poor expenditures and assessed the
efficiency of budget execution in key social sectors. The objectives were to review and assess
government procedures and channeling of resources in social sectors, as well as administrative
responsibilities and reporting mechanisms at each administrative level. Information was
collected at three levels of the public hierarchy. These two tracking surveys provided
information on disbursement procedures and delays for two types of funds (salaries and other
charges). Leakage between administrative levels was also estimated.
Ghana: The motivation for the Ghana 2000 PETS arose from Ghana‘s Poverty Reduction
Strategy (PRS), which recognized that a significant gap existed in the access and utilization of
basic services by the poor. The objective of the tracking survey was to improve the efficiency of
public spending and improve outcomes in social sectors. As in Uganda and Tanzania, the PETS
implemented in Ghana sought to estimate leakage of public funds in the transfer process from
the central government to public service facilities through district authorities, in basic
education and primary health care. The survey represented a pilot project to measure actual
expenditures in the two social sectors. It collected information at the central level, district
administrations, health clinics and primary and junior secondary schools. Sample selection was
structured to allow linkages with the Ghana Living Standard Survey conducted in 1998, which in
turn allows matching between households and facilities data sets.
Kenya: In Kenya, significant deterioration in health outcomes were observed despite increased
health spending. These results were perceived to be associated with an ineffective health
system as services failed to reach targeted people due to diversion of resources, weak
incentives, poor accountability mechanisms, and lack of demand attributable to low incomes in
a context of increased HIV/AIDS and poverty. Furthermore, in the education sector, despite
bursary funds created to support needy children, poor children had continued to drop out of
secondary schools due to prohibitive school fees. Poor targeting of programs and fund capture
were suspected. The government wanted to continue to shift resources toward core poverty
programs such as primary health care and education. However, it was crucial to track the
expenditures on some key inputs and services to determine where and how allocations were
spent and whether they were benefiting the poor, as intended. A PETS was put forward in
2004. The objective was to provide information on the process of resource allocation to the
97
service providers in the heath and education sectors to improve the effectiveness of the MOE
bursary program and primary health care services.
Specific objectives included: (a) to determine the amount and criteria used in allocation of
funds and inputs for health centers and dispensaries, (b) to assess the quality of services
provided to the patients, (c) assess the extent to which the management of funds in the bursary
program complied with MOE procedures; and (d) to assess the impact of the bursary scheme in
terms of its contribution in improving access and completion rates in secondary schools
(Republic of Kenya, 2004).
Several other surveys were implemented in Africa (e.g., in Cameroon, Senegal, Namibia and
Sierra Leone) and in other regions. Other surveys are ongoing, notably in Burkina Faso in the
education and sector and South Africa in the health sector (see PETS/QSDS web portal).
←
 Note 7: Measuring leakages: issues
In some tracking surveys however, no firm conclusions on leakage could be made due to several
factors, related mainly to the fact that the survey instruments chosen could not specifically
monitor such problems, or because of implementation problems. These relate especially to
methodological issues that affected the survey’s capacity to efficiently measure diversion of
funds and corruption (Gauthier and Reinikka, 2007).
In general, caution has to be exercised with respect to the interpretation of leakage levels (in
particular, but not exclusively, the “narrow” measure), as reasons other than corruption might
explain low arrival rates or observed differences of resources between levels (Reinikka and
Svensson, 2004a; Lindelow, 2006). These include incomplete records or problematic
accounting, data collection problems and data entry error (Gauthier, 2006, p.32)
Furthermore, the normal use of resources at an administrative level has to be accounted for in
the leakage calculation. However, it is often difficult to measure input use at a certain level. For
instance, in the Ghana report an 8% administrative overhead cost was used to measure
(narrow) leakage. 16
←
 Note 8: Average leakage levels in various Sub-Saharan African countries
Table 2 below shows all surveys in SSA countries and presents the average leakage level
observed in the various countries and the flows of resources on which these observations were
based.
←
16
Ye and Canagarajah (2002), however, do not provide justification for their specific choice of overhead costs.
98
TABLE 2: LEAKAGE OF PUBLIC FUNDS
Country
Uganda
Survey
Year
1996
Type
PETS
Sector
Educationa
nd
Health
Resources
tracked
Education:
Capitation grants
Health:
Non-wage
expenditures
Leakage
Education:
-87% (on average) during
1991-1995
-Leakage of salaries much
smaller
Health: Not defined.
Observation
Other comment
Cause
Education:
-Leakage appears principally
at the district level
-Resources either
disappeared for private
gains or were used by
district officials for purpose
unrelated to education
Education:
-Large variations in
leakage across schools:
- Larger schools appear
to receive larger share
of the intended funds
(per student).
- Schools with children
of better off parents
experience lower
degree of leakage
- Schools with higher
share of unqualified
teachers experience
more leakage
Education:
-Asymmetric
information has adverse
effects on the flow of
funds to frontline
providers and service
delivery
-Schools with greater
capacity to influence
local officials are
granted higher shares
-Shows the value of
transparency and
efficiency of mobilizing
civil society against
corruption
-Improvements are
associated with better
information about
school entitlements
through radio and
newspaper campaign
-Excessive drug use in
public and private
health facilities. This
could be associated
with over-prescription
or leakage.
-Procurement
malpractices at District
level
-Inadequate adherence
to accountability
procedures and
guidelines for provision
of safe and cleaned
water to the community
-Incompetent
Contractors
-Greed and Conflict of
Health: Most transfers from
government are in-kind. A
quantitative assessment of
the flow of resources to
health centers or service
delivered could not be
achieved.
-Information campaign is
estimated to account for
about ¾ of the
improvement in leakage
1999 and
2000
PETS
Education
-Capitation grants
-Reduction of leakage from
87% in 1991-95 to about 18%
in 1999 and 2000
2000
QSDS
Health
- Financing
- Drugs, vaccines
and supplies
-Some evidence of drug
leakage, but average figures
not provided.
2009
PETS
Water
- Water and
sanitation funds
allocation
-Evidence of funds leakage but
no specific estimates
99
Country
Survey
Year
Type
Sector
Resources
tracked
Leakage
Observation
Other comment
Cause
Interest
1999
PETS
Education
and Health
- Non wage
expenditures
-Leakage was estimated at
57% in education and 41% in
health care
2001
PETS
Education
and Health
- Non-wage
expenditures
-Average figures for leakage
not provided
-Difference between
disbursement and receipts
between Treasury and councils
is estimated at 18% on average
for non-wage funds for July
1999-June2000 and JulyDecember 2000.
2004
PETS
Education
Ghana
2000
PETS
Education
and Health
- Non wage
expenditures
- Salaries
Rwanda
2000
PETS
Education
and Health
- Recurrent
expenditures
Tanzania
The inflow of development
grant at the school level is 84%
of the central level
disbursement. The inflow of
overall capitation grant at the
school level is in the range of
54%-64% of the central level
disbursement. The inflow of
the cash part of the capitation
grant at the school level is 7686% of the central level
disbursement. The inflow of
books at the school level is
28% of the central level
disbursement.
- Leakage estimated at about
50% of non-wage education
expenditures and 80% of the
non-wage health
expenditures
- Leakage of salaries is
estimated at about 20%
- Some evidence of leakage
between regions and
districts, but no firm
estimates
100
- Leakage appears at the
district level
- Salaries appear less prone
to diversion
-Decentralized funds (OC)
sent to districts are
essentially all consumed at
the district level.
-Only material sent by the
center to districts is partially
redistributed to facilities.
-No cash funding below the
sector/district level, only inkind material
On average, councils only
acknowledge receiving 59%
of the development grant
sent to the regions.
-Large proportion of leakage
seemed to occur between
central government and
district offices (instead of
between district and
facilities), during the
procurement process when
public expenditures are
translated into in-kind
transfers.
-Except for staff salaries,
recurrent expenditures in
health and education do not
reach schools and health
-At the district level,
treasury and sector
heads tend to reallocate
non-wage expenditures
in favor of activities that
benefit the council staff
at the expense of
facilities (e.g. traveling,
vehicles, fuel versus
school material and
medications)
The book part of the
capitation grant is the
main cause of the
leakage in the transfer
of capitation grant.
- Lack of predictability
of the disbursement
promotes leakage
especially at the subnational level.
-Highly aggregated
government records are
reported to undermine
transparency
-Possibilities of leakage
were found to be much
greater when the value
of material distributed
was unknown to
recipients
-In-kind nature of
transfers increase
information asymmetry
and lack of
accountability in the
delivery system and
reduce feedback from
frontline facilities
-Lack of accountability
in the use of public
funds and other
resources contributed
-District offices
accounts are credited at
the discretion of the
regional offices.
Country
Zambia
Survey
Year
Type
Sector
Resources
tracked
Leakage
Other comment
Cause
facilities.
Observation
by users, parents,
NGOs, donors and other
development agencies.
-The lack of budgets
and guidelines for the
use of funds was cited
by health officials as the
source of major
inefficiencies and
causes of delays and
potential leakages.
2004
PETS
Education
-Teachers’ salary
- Three funding
programs:
i) Funds for
Genocide
Survivors (FARG)
ii) Education
support Funds for
Vulnerable and
Poor Children,
iii) Capitation
funds.
- Some evidence of leakage of
capitation grant at the
school level in particular,
but no firm estimates.
- Potential leakage of the
Education Support Fund
program at the central level,
but no estimates are
provided.
-There are no controls for
the utilization of capitation
grants by schools, as neither
the MOE, province or
district have control
mechanisms in place.
-District reports are said to
be unreliable and audits of
MOE does not cover all
schools.
-Anomalies are noted in
the list of beneficiaries
of the Education
Support Fund program
at the central level (the
first names of
beneficiaries are not
listed, neither their
exact birth dates or
name of their parents),
which introduces
potential misreporting.
2001
PETS
QDSD
Education
- Non-wage
funding for basic
education (fixedschool grant,
discretionary nonwage grant
program)
-Leakage of 10% for fixed-rule
grants
-76% for discretionary nonwage expenditures
-Rule-based funding reaches
almost every school, while
discretionary funds are
mainly used at the district
and province levels. About
20% of schools receive any
funding from discretionary
funds
2007
PETS
QDSD
Health
Various resource
flows
(government
budget, basket
funds, vertical
project funds and
internally
generated funds)
A fifth of the health facilities
received resources less than
their intended allocations.
Medical staff experience nonreceipt of the full amount of
salaries (about 15 percent of
staff); unauthorized salary
deductions (15.5 percent of
staff); and staff payment of
“expediter’s fee” to obtain
While allocation to the
districts is equityenhancing, allocation to
hospitals follows principles
of historical budgeting that
tend to perpetuate installed
capacity, even if much of
that existing capacity was
heavily influenced by mining
rather than public health
-Rule based funds are
progressive as greater
per pupil funding is
observed in poorer
schools
- Discretionary
disbursement higher to
richer schools in rural
areas and wealth
neutral in urban areas
-Overall, public funding
is regressive: almost
30% higher allocation to
richer schools.
The allocation rules
manifest themselves
clearly in the highly
inequitable provincial
receipt of resources,
where the poorest,
most remote and least
urbanized provinces
receive the lowest per
capita MOH releases.
101
-For rule-based funds,
delays in disbursement
may a factor.
-For discretionary
funds, the few schools
that received large
amounts have greater
bargaining power with
higher administrative
levels.
Country
Survey
Year
Type
Sector
Resources
tracked
Leakage
Observation
salaries (10 percent of staff).
concerns.
- Unreliable data on district
budget at the district and
provincial levels make it
difficult to assess whether
resources reach their
intended beneficiaries
- Evidence of capture of rent
by local government
officials and important
problems of non payment
of salaries of health workers
-Decentralized local
governments have different
priorities than health and
could use resources
earmarked to health for
other purposes.
Mozambique
2002
PETS
QSDS
Health
- No firm estimates of leakage
- Some evidence of leakage of
drugs in the transfer from
provinces to districts
Nigeria
2002
PETS
QSDS
Health
- No firm estimates of leakage
- But report of high degree of
leakages in the more rural
state (Kogi)
Senegal
2002
PETS
Health
Decentralization
fund
- No firm estimates of leakage
- Some evidence of leakage at
regional and communal levels
in non-wage expenditures
from central level to providers
Cameroon
2003
PETS
Health
- No firm estimates of leakage
2004
PETS
Education
- Non-wage
recurrent
expenditures
Various resource
flows
2003
PETS
QSDS
Health
Madagascar
20062007
PETS
Other comment
Cause
-Lack of accountability
of local government
leads to public
resources capture by
local officials.
- Inconsistent data on
resources sent and
received between
levels
- No firm estimates of leakage
- No firm estimates of leakage
Education
IPPTE
CRESED
- Leakage of cash funds at the
lower echelon of the education
sector is perceived to be low
(10% and 8% for the two main
sources of cash contributions
to schools).
- Leakage of material is more
common as 28% of schools
report receiving less material
than stated by the district.
- Leakage is associated with
remoteness as 56% of
schools in the Communes at
far distance from the capital
of the province show
leakages in comparison with
only 21% of the schools in
the Communes close to the
capital.
Education
and Health
Education:
Cash (caisse
école), school kits,
school equipment,
Education:
-The ratio of non-received
funds to expected funds for
the school year 2005-2006
Education:
-Leakage of the “caisse
école” is high for some
schools: in particular
102
- Schools are
uninformed about
decisions taken higher
up in the education
sector system
concerning their
potential resources and
possibilities.
-Only 35% of schools
reported knowing at
the beginning of the
year what they were
supposed to receive
from the district level
(Cisco).
Education:
-Smaller schools,
schools in poorer and
less educated areas
-Lack of information,
dysfunctional
accounting system and
absence of allocation
rules at the district and
school level increase
the incentives for
leakage.
Education:
-Low financial capacity
at the decentralized
levels
Country
Survey
Year
Type
Sector
Resources
tracked
textbooks and
salaries.
Health:
Current
expenditures, non
medical
consumable,
drugs
Namibia
2003
PETS
QSDS
Education
and Health
Chad
2004
PETS
QSDS
Health
Kenya
2004
PETS
Education
and Health
-Non-wage
recurrent
expenditures
Leakage
Observation
from the Ministry of Education
to Cisco and from Cisco to
school level equaled 3% and
4% respectively. -A high 40% of
the schools did not receive all
the equipment that they were
entitled to during the last two
school years.
Health:
Averaging across survey
rounds, 73% of the commune
pharmacies report leakage in
the drug supply chain from
district to commune levels.
- No firm estimates of leakage
smaller schools and schools
in ethnically heterogenous
areas suffered more from
leakage in 2005/2006.
Health:
Leakages of antibiotics are
especially problematic: half
of the basic health centers
that
ordered this drugs did not
receive what they were
supposed to receive.
seem to suffer more
from leakages.
Health:
The likelihood of
leakages in the drug
supply chain increases
with drug price
- Incomplete records make
it difficult to assess whether
resources reach their
intended beneficiaries
- Potential leakage of
medication and other
material
-Public resources arrival
rates vary considerably
among regions.
-The highest rate of leakage
is observed in the BET
region, the most remote
area of the country
- A large proportion of
leakage seemed to occur
between central
government and regional
delegation during the
procurement process via so
called “centralized credits”
Health:
Leakage is more
pronounced in health
centers than dispensaries.
Education:
-The bursary program is not
thoroughly audited which
increases possibilities of
leakage
-Criteria of selection of
-It was estimated that if
all public resources
officially budgeted for
regional delegations
had reached the
frontline providers in
2003, the number of
patients seeking
primary health care in
Chad would have more
than doubled during the
year
- Only about 27% of non-wage
budgetary officially allocated
to regions by the MOH reaches
regional health delegations
-Less than 1% of non-wage
budgetary officially allocated
to regions reach local health
centers
Health:
-Leakage of total funds
received at the health center
level is estimated at 38%
- Leakage of user fees at the
facility level estimated at 25%
- Leakage of Community
development funds at facility
level is estimated at 37%
Education:
103
Other comment
- An audit trail of the
bursary funds released
was not possible
because of the lack of
proper accounting
system
Cause
- Poor record keeping
and few records of the
delivery of material to
district and school level
are available and could
increase the incentives
for leakage.
-Main factors explaining
low level of resources
received at the local
level are:1) the very
high rate of resource
centralization at the
MOH level, 2) the lack
of supervision and
control of resources,
and 3) lack of planning
in the allocation of
resources as allocations
are arbitrary at every
levels.
Health:
-Provinces and districts
are unaware of budgets
and programs.
-Supervisory capacity of
provincial and district
authorities are
insufficient.
-Financial and
accounting systems are
Survey
Year
Country
Type
Sector
Mali
2005
PETS
QSDS
Education
Niger
2008
PETS
Education
and Health
Resources
tracked
Selected
government
expenditures in
education
(textbooks,
notebooks and
drawing books for
students) and in
health (food
expenditure and
hospital supplies,
and essential
medicines)
Leakage
Observation
-More than 80% of schools did
not receive their entitled
amount of bursary funds
-Evidence that some schools
are receiving more allocation
than required and that funds
are diverted for personal gains
-Total leakage of bursary funds
estimated at 35.8%
vulnerable and needy
students not followed by
many schools
Schools received on average
40% of the total number of
textbooks originally allocated
to them by the MoE.
The leakage rates vary
considerably among CAPS
and can exceed 90% as is
the case for Baraouéli or
Torokorobougou (in
Bamako).
- No firm estimates of leakage
Education:
-General lack of systematic
information recording.
-Budget increases did not
always lead to spending
increases.
- Supplies do not arrive at
facilities on a regular basis.
Health:
The quantity of financial
resource transfers between
MH and the Regional Health
offices (DRSPs) are well
accounted for. The regional
offices are also effective in
procuring materials once
credit becomes available.
Other comment
the leakage rate
increases with the
allocation amount,
resulting in very high
leakage volumes for
schools which have
been allocated large
numbers of textbooks
Health:
Other aspects of the
financial management
at DRSPs, however,
could be improved.
Record keeping could
be more detailed in
terms of exact dates of
financial transactions,
amounts and
breakdown of spending.
Sources: Gauthier 2006, Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006), PETS 1996-2004 tables.
←
104
Cause
inadequate
Education:
-Financial management
of the school is in the
hands of the head
teacher with minimal
influence of the PTA
and BOG.
-Lack of information at
the school level leads to
non accountability of
public resources
-Poor school records
and lack of proper
audits
The absence of
transparency on budget
information facilitates
the capture of the
education budget
 Note 9: Salaries
Salaries are also often paid directly by the central government to individual workers at the
service provider level, without going through the administrative apparatus (for instance, in
Rwanda, salaries are paid directly into workers’ bank accounts). Alternatively, when salaries are
transferred through the administrative structure, they are generally paid by local authorities
directly to workers, thus with the same incentives at the recipient level for ensuring full
transfer.
←
 Note 10: Estimates of delays: examples from Africa
Table 3 below presents estimates on delays in various SSA countries, for certain types of items
and inputs.
←
Country
Uganda
TABLE 3: DELAYS 17
Survey
Year
Type
Sector
1996
PETS
Education
Health
2000
QSDS
Health
1999
PETS
2001
PETS
Education and
Health
Education and
Health
Ghana
2000
PETS
Rwanda
2000
PETS
Tanzania
17
Education and
Health
Education and
Health
Delays
Observation
-Anecdotal evidence that
teacher’s salaries suffer
from delays.
-However, survey
indicates that salary
payments reach schools
relatively well.
-72% of staff faces salary
delays in public facilities
(compared to 28% in for
profit facilities).
-40% of government
facilities report stock outs
of supplies during the FY
- Frontline workers suffer
delays in pays
-Delays in disbursement
and significant delays in
the processing of nonwage funds, ranging from
6 to 42 days at the
treasury, while wage
disbursements are rarely
delayed.
-Delays are also observed
in all districts surveyed.
N.A.
-Evidence of delays in
budget execution at the
central level
-Considerable delays in
transfers between regions
Source: Gauthier 2006, (pp. 52-55)
105
Cause
-In 20% of government
facilities, salary delays are
reported more than 16
weeks.
-Delays are reported
worse for non-wage
expenditures versus
salaries and in rural areas
-In some district, transfers
were not made by
councils to some sectors
for the period 1999-2000
-Linked to cash
budgeting system and
the fact that salaries are
prioritized in the budget.
-This increases volatility
in transfers and increase
asymmetry of
information
N.A.
Health:
-Very low execution rate
as 80% of non-wage
expenditures are released
at the end of the year
-Delays were largely
attributed to the
application of the cash
budgeting system in the
MOF and cash
Country
Survey
Year
Type
Sector
Delays
Observation
and districts
2004
PETS
Education
Sierra Leone
2000
2001
PETS
Zambia
2001
PETS
QDSD
Agriculture,
education,
health,
security
sector, social
welfare, rural
development
and local
development,
water and
sanitation.
Education
2007
PETS
QDSD
Health
2002
PETS
Health
Mozambique
-In particular, delays were
observed in the payment
of capitation grant to
schools
-13% of teachers do not
receive their salaries
regularly.
-82% of teachers have
salary arrears (2003)
-Irregularities in the
payment of the Education
Support Fund program
reported by 43% of
students surveyed
Education: Most schools
in districts covered in the
survey reported high
percentages of teachers
receiving salaries on a
regular basis.
- Salaries: About 5% of
teachers incur delays
-Hardship allowance: for
almost all the provinces,
about 20% of teachers
incur delays
-“double-class allowances
(additional amount paid
for overtime, etc): More
than 75% of recipients of
“experience at least 6
month overdue
Delays in the receipt of
salaries are observed (22
percent of staff).
- Delays and bottlenecks
106
- Only 47% of teachers
knew the amount of
salary arrears.
Cause
constraints of the
government
-Salaries are directly
transferred to teachers’
bank accounts. They
don’t receive detailed
pay slip. They lack
information about their
exact salary and
deductions at the
source.
Education: Only Bonthe
and Moyamba have low
percentages of teachers
who received their
salaries on time.
- Well defined allowances
(hardship and
responsibilities) tend to
be paid on time
-However, less well
defined allowances suffer
important delays.
- Delays in the case of
double class allowances
and student trainees
appear to be due in part
to lag in payroll updating
In contrast to previous
years, the timing of MOH
releases on district grants
is predictable. However,
more than a third of the
DHMTs themselves
admitted delays in
releasing district grants to
facilities, although this is
difficult to understand
given that most of them
received these resources
from MOH on time. Out
of the 20 DHMTs queried,
seven (or 35 percent)
reported delays in
releasing funds to the
health centers and district
hospitals under them, 50
percent did not, and 15
percent provided no
categorical answer.
Only two-thirds of
health facilities have
accurate registers.
Country
Survey
Year
Type
Sector
QSDS
Delays
in budget execution
and supply
management
- Extensive non-payment
of salaries observed in
one of the two states
surveyed, Kogi
- Overall, 42% of staff
experience salary delays,
reporting not receiving
salary for 6 months or
more in the past year at
the time of the survey.
Nigeria
2002
PETS
QSDS
Health
Senegal
2002
PETS
Health
- Evidence of delays in the
decentralization fund
(non wage)
Cameroon
2003
PETS
Health
2004
PETS
Education
2003
PETS
QSDS
Education
20062007
PETS
Education and
Health
-Delays in the notification
of budgetary envelopes
to decentralized units
-Delays in the notification
of budgetary envelopes to
decentralized units are
noticed.
-Significant delays to get
IPPTE and CRESED at the
school level
Education:
-Significant delays in the
arrival of the “caisse
école”, school kits and
textbooks at school level
Health:
-Extensive delays in the
distribution of drugs from
the central purchasing
unit to the facility-level
pharmacies. The
distribution takes on
average one and a half
months.
-There are big delays in
payment of dispensers’
salaries by communes:
60% of the health centers
report irregular
payments.
Madagascar
Namibia
2003
Chad
2004
Education and
Health
PETS
QSDS
Education:
- Delays in the supply of
books at the school level
Health:
- Delays in the release of
funds at the central level
- 14% of the health
workers face delays in
receiving salaries.
- In public clinics, salary
107
Observation
Cause
- Regression results show
significant differences
between Kogi and Lagos
in terms of the extent of
non-payment of salaries.
-Non payment of salaries
had impact on provision
of services : the greater
the extent of non
payment of salaries, the
higher the likelihood that
facility staff behaved as
private providers
- Non payment of salary
was related to problems
of accountability at the
local government level
- Asymmetry of
information between
the local government
and service providers on
funds allocated
-Fund managers have
about 9 months to
execute their budget
- District is slow in
budget execution
Health: Health centers
where payments are
irregular suffer more from
the leakage of antibiotics.
Education:
-Frequent teaching staff
meetings and
inspections lower
leakages of the caisse
école in the education
supply chain
Education
-Mismatch between
MOE textbooks
catalogue and available
books
- Delays in salary payment
are slightly higher 16 % vs
10% in urban areas
- Poor infrastructure and
absence of
decentralized financial
institutions. Workers
Country
Kenya
Mali
Survey
Year
Type
Sector
Delays
Observation
Cause
delays affect 20% of the
often have to travel long
personnel
distances to collect
7 % of health personnel
salaries
doe not receive their full
salaries.
2004
PETS
Health and
-Delays in medical
- Delays in medical
education
supplies delivery
supplies may be
-61% of health centers
explained by top-down
report stock outs of drugs
approach in the
during FY 2003-04
procurement process.
2006
PETS
Health and
Doctors paid by the HIPC
QSDS
education
fund receive salaries with
up to 2 or 3 month delays.
Salaries for doctors paid
by the City are often
delayed by 12 to 20
months.
Sources: Gauthier 2006, Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006); PETS 1996-2004 tables. Note;
(*) Reports not available; N.A.: Not available
←
 Note 11: Measuring equity
Das et al. (2004a) used a household survey in Zambia to develop an index of wealth, which
measured the progressive nature of two types of school funding (rule-based and discretionary).
In addition, their equity analysis accounted for the value of staff inputs at the school level and
private contributions to education expenditures. The study found that rule-based (per-school)
funding had progressive characteristics that led to greater per pupil funding for poorer and
rural schools. This was mainly due to the fact that rule-based funding was a fixed amount per
school, irrespective of enrolment, and enrolment was lower in poorer rural schools. However,
staff expenditures per pupil were found to be regressive; they were higher in urban and richer
schools. This was associated with the fact that poor and rural schools tend to have a larger
share of lower paid teacher trainees on the payroll. Discretionary funding was found to be
regressive in rural areas (whereas it was wealth neutral in urban areas). Overall, (accounting for
all these sources of public funds) public school funding in Zambia was shown to be regressive,
with richer schools receiving almost 30 percent more public resources per pupil. Furthermore,
Das et al. (2004a) also showed that household spending further contributes to inequalities in
education.
Das (2004) examines the question of the effect of government education expenditure on
equity. His model specifically accounts for the difference in the substitution effect between
public and private education expenditures between the rich and the poor. He shows that the
success of government education expenditures at reducing educational inequalities depends on
the types of expenditures and institutional arrangements used. While cash grants were
successful at targeting the poor, they were less successful at modifying overall educational
expenditures given the greater crowding out effect on poorer households. Consequently, public
transfers did not reduce inequality in overall educational expenditures.
108
Gauthier and Wane (2005, 2008a) investigated equity in Chad. They focus on access to public
health resources at the regional level and user costs among quintiles of income. They find stark
inequities in the Chadian health system. The most affluent individuals have better access, both
geographical and financial, to health care and receive a higher quality of care. About 44 percent
of individuals in the richest quintile have another health care provider close to their home,
compared with 13 percent for the poorest quintile. For the most affluent patients, out-ofpocket medical expenses account for 2.1 percent of their monthly household incomes versus 21
percent for the poorest, which thus bear a burden that is ten times heavier
←
 Note 12: Measuring impact of decentralization
Das et al. (2004a) incorporated the question of decentralization in the sample design in the
2002 Zambia PETS/QSDS which surveyed two centralized and two decentralized provinces.
Decentralized provinces were defined as those where a district education board exists and is
responsible for school funding. While the survey indicates that decentralization improved the
flow of funds by decreasing spending at the provincial level, it somewhat reduced the allocation
of funds to schools. Indeed, decentralized provinces presented greater levels of fund capture
than centralized provinces. Overall, while only between 15 percent and 33 percent of total
funding in the system (rule-based and discretionary funding) reaches schools, the record is
slightly worse in decentralized provinces. Schools in centralized provinces receive around 30
percent of total funds in the system compared with about 25 percent for schools in
decentralized provinces. Note that the tracking exercise in Zambia assessed the amount of
resources available in the education system within the four provinces surveyed, distinguishing
between rule-based and discretionary funds, and the originating level of the resources, as well
as funds available at the district level. Financial resources available at the school level were
tracked per pupil (Das et al, 2004a, p.36).
Khemani (2006) examined decentralized delivery of primary health care in Nigeria. Her
empirical study is based on the PETS/QSDS carried out in 2002 in two states (Lagos and Kogi)
covering 30 local governments, 252 primary health care facilities and over 700 health care
providers. The two states differ on a number of aspects that influence accountability
relationships, in particular level of urbanization, availability of alternatives (such as private
providers) and effectiveness in monitoring frontline providers. The study found a high degree of
leakage in the more rural Kogi state. Furthermore, the author found evidence of a general
problem of accountability at the local level in the use of public resources. Although the study
cannot address the question of whether decentralization has a beneficial effect on allocation
and use of public resources — because the two states examined did not differ significantly in
the extent of decentralization of responsibility — the overall policy lesson is that strengthening
local government accountability is fundamental to reducing public resources capture.
←
109
 Note 13: Drug mark ups
Gauthier and Wane (2008b) based on a 2004 PETS/QSDS show that that there is a negative and
significant relationship between average drug mark-up and effective transfers of public
resources to health centers in the health sector in Chad. Local health facilities that receive
government transfers are able to charge lower mark-ups on drugs than centers that do not
receive transfers. Leakage of government resources thus appears to have a significant and
negative effect on user fees and thus constitutes a barrier to access to health care. They also
observe that health centers located in rural areas tend to charge significantly lower mark-up
than centers located in urban areas and that greater competition among health centers tends
to significantly reduce average mark-up on drugs and thus be favorable to users.
←
 Note 14: PETS as part of country dialogue
Macro perspective: PETS/QSDS can well inform the overall country dialogue on economic
management, poverty reduction strategies, development strategies and plans, and public
expenditure policies and implementation.
Poverty dialogue: The overarching objective of many developing countries is the reduction of
poverty, and one of the key strategies to reduce poverty is the use of public expenditure and
related policies. Well-managed public expenditures in the social sectors have proven to be one
of the most effective tools for reducing poverty and improving social sector indicators.
However, while the dialogue on the allocation of public resources, such as the size of the
budget or inter-sectoral allocations, is only the front-end of that discussion, what actually
matters is the implementation of a given allocation strategy and policy. It is not enough to
allocate: it needs to be ensured that resources are spent effectively at the facility levels.
Sector dialogue: PETS/QSDS can be powerful aspects of the sector dialogue. Once the central
finance authority has allocated the sector budgets and resources, it is the responsibility of the
sector ministries to utilize the resources to achieve the desired outputs, outcomes, and results.
PETS/QSDS are the key instruments for diagnostics that ask and answer questions such as: have
the teachers been hired, are textbooks being procured and reaching the students, are drugs
available to the patients, are centrally allocated resources reaching the decentralized levels and
facilities? Once the sector has been allocated the resources, it is the concern of the manager to
ensure that the resources are well used and that there are sector results.
←
 Note 15: PETS Within the programmatic AAA approach
PETS/QSDS can fit in well with the programmatic approach to AAA/ESW, usually within the
overall PFM umbrella. The usual multiyear perspective of programmatic AAA, which groups
related AAA tasks, is particularly suitable for PETS/QSDS which can often span fiscal years to
110
plan, design, implement, disseminate, and translate into policies. A programmatic AAA also
allows for the important task of prioritizing, sequencing, and linking related activities. The TTLs
can decide, in close consultation with the stakeholders, what should be done first and how the
synergies among the tasks can be best realized. However, the realities imposed by capacity
constraints such as limited human resources in the government to do these tasks need to be
factored in. Similarly, the task manager can only do a limited number of tasks at any given
time.
←
 Note 16: Raising the awareness of the government and potential stakeholders
Sensitization activities could also take place to inform and sensitize the governments and the
various potential stakeholders of the usefulness of PETS and QSDS. This could take the form of
workshops or other types of activities (See Example 11).
These activities could help to:

Develop country ownership and build support for conducting the study;

Elicit views and identify problems in service delivery, and refine and identify specific
study objectives.
←
111
 Note 17: Typical policy reforms resulting from PETS/QSDS
TABLE 4: INTERVENTIONS AND REFORMS 18
Country
Uganda
Year
Survey
Type
Sectors
1996
PETS
Education
and Health
1999
and
2000
PETS
Education
2000
1999
QSDS
PETS
2001
PETS
Health
Education
and Health
Education
and Health
Ghana
2000
PETS
Education
and Health
Rwanda
2000
PETS
Health
Tanzania
18
Interventions
-Central government attempted
to remedy the situation by
publishing monthly
intergovernmental transfers of
public funds in the main
newspapers and on the radio,
and required schools to post
information on inflow of funds.
Objective
Reported effects
- To make information
available to citizens and
PTAs
- To signal to local
governments that the central
government had resumed its
oversight role
- Flow of funds improved
dramatically from 13% of
intended capitation
grants reaching schools
in 1991-95 to over 80%
in 1999 and 2000
- Delays in transfers are
still considerable.
Implications
-Results suggest that
efforts to increase
citizens’ ability to
monitor the system,
and to inform them
about their rights and
entitlements, are
important aspect in
controlling corruption
-Treasury initiated dissemination
of itemized local government
budget to members of
Parliament and regular
publication of budget allocation
to various levels of government
in newspapers
-PETS opened an avenue for
inter-ministerial collaboration
and provided a practical
approach to assess service
delivery
Source: Gauthier 2007, pp. 121-122
112
-Early indications are
that the campaign has
reduced leakage
considerably
-Survey results have not
produced a strong
response to reduce
leakage and rent
capture, either through
innovations in
transparencies or
increased oversights at
various government
levels.
Country
Year
Survey
Type
2004
PETS
Sierra Leone
2000
and
2001
PETS
Zambia
2001
Mozambique
2002
Nigeria
2002
Senegal
Cameroon
2002
2003
PETS
QDSD
PETS
QSDS
PETS
QSDS
PETS
PETS
Madagascar
2003
2005
Namibia
2003
Chad
2004
Kenya
2004
PETS
QDSD
PETS
QDSD
PETS
QSDS
PETS
QSDS
Sectors
Education
(Also
realized in
health,
water and
agriculture
projects)
Education,
Health and
various
other
sectors
Education
Interventions
Objective
-PETS has led to modifications
in the MOF procedures,
including tracking of
expenditures to regional
administrative levels
Health
Health
Health
Education
and Health
Education
Health
Education
and Health
Health
Health and
education
Sources: Survey reports. Uganda, Tanzania: Reinikka and Svensson (2004a).
←
113
Reported effects
Implications
 Note 18: Typical PETS objectives
TABLE 5: PETS-QSDS SURVEYS IN SUB-SAHARAN AFRICA: MOTIVATION AND OBJECTIVES
Country
Uganda
Tanzania
Year
Type
Sectors
Purpose
1996
PETS
Education and
Health
Diagnosis
1999
and
2000*
PETS
Education
Followup/impact
evaluation
2000
QSDS
Health
Analytical
2004*
2009
QSDS
PETS
Health
Water
Follow-up
Analytical
1999
PETS
Education and
Health
Diagnosis
2001
PETS
Education and
Health
Analytical
2003
2004
PETS
Information on
rural water
supply and
roads, judiciary
and HIV/AIDS
was also
collected.
Education
Analytical
Education
Analytical
Context/Motivation
Objectives
-To identify the reasons
why despite substantial
increase in public
spending on basic
services, output and
outcome indicators
remained stagnant, in
particular official reports
of primary enrolment.
-Implementation of an
Information campaign to
provide citizens’ voice.
-Locally implemented
surveys
-Health services were
perceived as not
meeting the needs of the
population.
-People preferred
private non-profit and
for-profit health clinics
for curative care to the
less expensive public
health facilities
-Despite increasing
health budget, health
indicators were declining
-To measure the difference between the
intended resources and resources
actually received at various tiers,
including service providers
For the last over two
decades, significant
amounts of money into
the sector to improve
issues of access and
functionality for the
resource poor peoples.
-Suspicion that serious
problems existed in the
flow of funds from the
central government to
frontline providers
-Monitoring of pro poor
expenditures in the
context of a PER
-Analyze the national budgeting process
- Establish and track the chain flow of
funds and budget support to water and
sanitation sector from the centre to the
end-user.
- Analyze local government accountability
mechanisms
-To track all non-wage transfers to
primary schools via local authorities.
The study covered the
Capitation Grant,
textbooks and the
Development Grant, and
also considered data on
enrollment and test
scores. There are no
references to the
previous two PETS.
This study was carried
out in Tanzania as part
of a process of
enhancing accountability
and to respond to
To analyze the flow of resources in a
stand alone development program in
primary education.
114
-To collect data on access to information
and capitation grant reception among the
PETS 1996 school sample in order to
evaluate the impact of the information
campaign on leakage levels
-To assess services provided by different
categories of providers
-To identify problems in facility
performance, including the extent of drug
leakage, staff performance and availability
-To provide information on user charges
and application of user fee policies
-To measure and explain the variations in
cost efficiency across health units with a
focus on the flow and use of resources
-To examine the patterns of staff
compensation, oversight and monitoring
and their effect on performance.
-Tracking down government pro poor
expenditures on priority sectors
-To assess the efficiency of budget
execution
-To make recommendations for improving
monitoring pro poor expenditure
To establish more exact knowledge on
the actual amount of resources disbursed
from the central level that reach local
schools.
Country
Year
Type
Sectors
Purpose
Ghana
2000
PETS
Education and
Health
Analytical
Rwanda
2000
PETS
Education and
Health
Analytical
2004
PETS
Education
(Also realized in
health, water
and agriculture
projects*)
Analytical
Sierra Leone
2000
2001
PETS
Diagnosis
Zambia
2001
PETS
QDSD
Agriculture,
education,
health, security
sector, social
welfare, rural
development
and local
development,
water and
sanitation.
Education
2007
PETS
QDSD
Health
Mozambique
2002
PETS
QSDS
Health
Pilot:
Diagnosis/
Analysis
Nigeria
2002
PETS
QSDS
Health
Diagnosis/
Analysis
Analysis
Context/Motivation
funding leakage issues.
-Major gaps exist in
access to and utilization
of basic services by the
poor
-Budget shares for
health and education
were increased in a
context of a
decentralization policy
Objectives
-To measure leakage between different
points of resources distribution.
-To identify bottlenecks in resource flows
-To explore ways to ensure that funds
reach service delivery facilities
-Increase in the
budgetary allocation to
the social sectors,
particularly health and
education
-Social indicators remain
stagnant
-Indicators, especially in
health and education,
remained stagnant
despite increase
budgetary spending in
social sectors.
- In the context of
conflict, the home-grown
surveys sought to fill an
accountability vacuum
- Increased budgetary
allocation to social
sectors without
noticeable improvement
in outcomes
-To track expenditures for social services
-To identify delays and leakages of
budget transfers in order to improve the
effectiveness of budget spending.
-To demonstrate surveillance and control
of the expenditures to the civil society and
external donors.
-To analyze public funds transfer process,
respect of procedures, account keeping
and evaluate leakages of public funds
-To formulate strategies to improve the
system.
-In a context of
economic crisis, the
government and donors
seek to increase quality
and equity in primary
education.
-Reforms in the
administrative and
institutional structure of
the education delivery
system were introduced
-Need to increase per
capita health spending.
Health indicators remain
weak, signifying possible
resource allocation
and/or service delivery
problems.
-To examine the structure of funding and
implications for service delivery
-To examine the relationship between
expenditures and educational outcomes
-To determine if the changes undertaken
by the MOE had the desired impacts in
terms of education outcomes
-Broad public sector
reform agenda designed
to increase efficiency
and transparency in
public service de livery.
-Health sector reforms,
including institutional
and management
reforms.
-In the context of
decentralization of basic
health services at the
local government level,
-To assess the functioning of the health
service at the primary health care level
with an emphasis on flow of resources
and output.
-To assess if resources allocated to
primary health care reach their
destination.
-To provide baseline data against which
progress can be assessed
-To examine the flow of resources an
incentives at the facility level
-To examines the role of local
governments and community based
115
-To provide diagnostic information on key
poverty sectors
-To provide evidence on leakage in the
service provider supply chain
-To monitor and evaluate government
services in order to increase transparency
-To map sources and uses of funds in the
health sector
-To assess the allocation, release, actual
receipt and use of public expenditure
down to the facility level
-To analyze alternative scenarios for
improving allocation of resources in the
health sector and options for service
delivery.
Country
Year
Type
Sectors
Purpose
Context/Motivation
to understand how
institutional
arrangements work in
practice and impact
service delivery
2006
PETS
Education and
Health
Diagnosis/
Analysis
Senegal
2002
PETS
Health
(Education*)
Diagnosis/
Analysis
Cameroon
2003
PETS
Health
Diagnosis
Madagascar
2004
2003
PETS
PETS
QSDS
Education
Education and
Health
Diagnosis
Analytical
and impact
evaluation
20062007
PETS
Education and
Health
Analytical
2003
PETS
QSDS
Education and
Health
Analytical
Namibia
-Weak correlation
between budgetary
allocation at the national
level and outcomes
motivated the tracking of
resources.
-Despite economic
growth, poverty remains
prevalent
-In the framework of a
national governance
plan to fight corruption,
reduce poverty and
increase efficiency of
public expenditures
-Priority given to access
to basic services
idem
-In the aftermath of the
2002 political crisis, the
government used part of
the HIPC funds to pay
for the tuition fees of all
public primary school
children to boost
relatively low enrolment
rates.
-Also expected to shed
light on the effectiveness
of decentralization
policies
-The PETS seek to
evaluate the
effectiveness of this
measure on school
enrolment
Education: -To improve
the enrollment and
completion rates as well
as the quality of
education, the
Government had
substantially increased
investments in the
education sector in
recent years.
Health: Despite large
efforts by the
government and donors,
little updated information
was available on the
effectiveness of
spending in the public
health sector.
-Despite important
budgetary allocation in
social sectors, outcomes
do not meet
116
Objectives
organizations in the delivery of primary
health care services
Education: To collect information on
capital expenditures for school
construction.
-To track flows of public funds through the
various hierarchical levels to evaluate
resources reaching local service providers
and how long it takes
-To assess the impact of leakages and
delays on service provision
-Evaluate delays in budget execution of
non-wage expenditures toward local
service providers
-To assess leakage levels at different
levels in the service provider supply chain
and procurement process.
-Evaluate clients’ service satisfaction
idem
Education: -The main objective of the
study was to provide information on
expenditure allocations and leakage in the
education system, from the two main
funding programs in the education system
(one administered by the World Bank, the
other by the MOE).
Three surveys were implemented.
-First survey: To evaluate the impact of
the political crisis on economic activities
and on social service delivery.
-Second survey: to track budget at the
school level.
-Third survey: To collect information at the
district level on budget allocation.
Health: -Impact of reinstating user fees
-To evaluate health facilities’ efficiency
and its determinants and to measure
patients’ satisfaction.
Education: - To analyze flow of resources
through the system and measure
absenteeism, leakage and bottlenecks
Health: -To study if and when budgets,
material and salaries arrived at the basic
facility level based on primary data
collected in a nation-wide survey.
-To improve their knowledge on the
functioning of the sector and thus to
address bottlenecks towards better
service delivery.
-To track public resources to local service
providers at different levels in order to
assess leakage
Country
Year
Type
Sectors
Purpose
Chad
2004
PETS
QSDS
Health
Analytical
Kenya
2004
PETS
Health and
education
Analytical
Mali
Mali
Niger
Context/Motivation
expectations
-Hypothesis is that some
funds do not reach
destination
-The government and
donors wanted to ensure
in the context of
increasing oil revenues
and public expenditures
in social programs in the
framework of the PPTE
Initiative, that funding
would contribute to
poverty reduction
objectives by reaching
their destination.
Objectives
-To compare health resource access and
level of services in all regional
delegations.
-To analyze resource allocation and their
impact on performance of health care
facilities
-To provide data on basic characteristics
of health centers
-To provide information on potential
factors explaining differences in
performance between health facilities and
types of ownership
-To identify means to improve the
situation.
-To provide information for improving the
effectiveness of public expenditures in
bursary funds and in primary health care
in order to determine if they benefit the
poor as intended
-Deterioration of health
indicators despite
increase in health
spending.
-In the education sector,
important rates of
dropout despite bursary
programs.
2005
PETS
Education
Analytical
With a consistently
Identification and analysis of potential
QSDS
increasing budget, the
causes of problems encountered in the
Ministry of Education
delivery of public resources to designated
(MoE) is also
recipients.
progressively
decentralizing the
spending of a higher
share of its resources to
the regional level.
2006
PETS
Health
Analytical
In the context of public
- Provide a quantitative assessment of the
QSDS
management reforms
budget execution with a focus on nonunderway to improve
salary expenditures
health outcomes, the
- Assess the delays in receiving health
Ministry of Health has
services and assessment of resource
developed in 2003 the
losses
Medium Term
- Analyze the impact of constraints and
Expenditure Framework
anomalies in the flow spending on the
(Cadre de dépenses à
efficiency and performance of health
moyen terme) of the
facilities.
health sector.
2008
PETS
Health and
Analytical
In order to improve
- To increase the government’s
education
development outcomes
understanding of the link between public
in education and health
spending and service delivery at the
and to ensure that
facility level
resources reach their
- To contribute to improving the
intended beneficiaries,
effectiveness and accountability in the
there is a need for
use of public funds.
increased resource
allocation to also be
accompanied by
improvements in the
efficiency of public
expenditures.
Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006), PETS 1996-2004 tables. Note:
(*) Reports not available.
←
117
 Note 19: Previsions versus effective expenditures data
There are generally various budget laws in a country. Indeed, in most countries, there are
generally three or four budgets: (a) planned (b) revised and (c) executed (final) and (d) audited.
The “Planned” budget is important in itself as it helps identify the capacity of programming of
the administration, as well as for identifying incentives and real priorities of the government.
However, for PETS purposes, one needs especially to make use of the executed budget, which
presents among the three budgets, the best information available on official resource transfers
and which reflects the variations in the levels of revenues and expenditures. Indeed, there
could be expenditure leakage between the MoF and the line ministry. Also, this information
allows the determination of how adjustments are made when there are (positive or negative)
shocks, (i.e., which ministry gets hit?).
While in theory, audited accounts of actual government expenditures are the most reliable, and
are preferable to data on unaudited spending, there is typically a 1–2 year delay between the
end of the fiscal year and the availability of audited data on actual expenditures. Generally, the
most practical solution is hence to use provisional (unaudited) figures on executed budgets.
←
 Note 20: Choice of units in the primary sector in health and education in Senegal and
Tanzania 2010
In Senegal and Tanzania, as reported by Bold, Gauthier, Maestad, Svensson and Wane (2011),
the aim of the 2010 QSDS studies was to measure quality of primary services in health and
education. The unit of observation in education was easy to determine (i.e., all schools with
primary level teaching). However, in health care, the domain of coverage was more complex.
Primary health includes outpatient consultations, family planning, maternal and child health
services, etc.
Generally, these services are provided both in lower level units (e.g., health posts, dispensaries,
and community health centers, etc), as well as hospitals. A key question is whether all levels of
providers should be included, or it is sufficient to include levels for which provision of primary
health services is the main objective. It is more complex to include an analysis of hospitals
services, except if there is a clear separation within hospitals between primary health services
and other services.
118
In the case of the SDI (QSDS) surveys in Senegal and Tanzania, the approach has been to include
lower level facilities and include higher levels until one was confident to have covered a
significant majority of people’s encounters with the primary health services.
A 75 percent coverage was used as a rule of thumb. In Tanzania, the survey included
dispensaries and health centers. In Senegal, the survey included community health posts and
health centers. Hospitals were not covered in either country.
←
 Note 21: Hard and Soft Rules governing resource flows
Resource flows (financial, in kind) within public administrations could be governed by either
hard or soft allocation rules.
(a) Hard/fixed rules: These rules correspond to situations where a specific criteria or allocation
rule is used to determine the transfer of resources toward a level, area, units, or specific
population.
For instance, at the central level, such rules are used when states, regions, provinces or
districts are financed through a specific rule, such as a per capita basis or through a poverty
formula (perequation, etc).
Such formulas could also be found for allocating resources toward service providers
through, for instance, transfers to schools on a per capita or per student basis (i.e.,
capitation grant), or by schools, by types of schools, or mediated by some poverty
criterions.
(b) Soft or discretionary rules: Such rules correspond to a situation where transfers are not
conditional to a specific rule or criteria and the effective allocation is left to the discretion of
bureaucrats or elected officials.
Soft allocation rules are frequent in many developing countries. For instance, at the central
line ministry level in the health or education sectors, it is frequent that allocation of some
budget lines toward various decentralized administrative units are not determined by a
specific (hard) rule, but left to the discretion of the officials.
Similarly, at lower level of the administration, officials at the provincial, regional or district
level could have the discretion to allocate resources toward the units under their
jurisdiction according to their discretion.

For instance, in Chad, allocation from line toward lower administrative levels is
essentially entirely left to the discretion of officials. (Note that there could exist
119
a decision rule established by the officials which could be elicited by the
researchers).

Note that salary flows are allocated according to the contractual obligations of
the ministry or administration in a specific area and could hence be considered
as hard rules.
←
 Note 22: Rule based versus discretionary expenditures
The level of discretion exercised on resource allocation tends to influence leakage levels.
Greater discretionary power granted to particular administrative units, combined with weak
supervision and improper incentives, could lead to large fund capture.
Indeed, differences in leakage levels have been observed between funds allocated through
fixed-rule and those that are at the discretion of public officials or politicians (Reinikka and
Svensson, 2001, Das et al, 2004a, and Lindelow, 2006). In Zambia for instance, rule-based
funding (per-school grants) presented a level of leakage of only 10 percent, versus more than
76 percent for discretionary funding. 19 As emphasized by Das et al (2004a), since rule-based
funding is clearly defined by a simple allocation rule, capture of funds is more difficult
compared with discretionary funds, which are not bound by any specific allocation rule.
Similarly, in the health sector in Chad, where no fixed allocation rules apply for most resource
allocation, it was estimated that only 1 percent of non-wage resources allocated to regional
health administration in 2003 arrived at the facility level.
However, this finding about rule-based versus discretionary expenditure cannot be generalized
yet. Indeed, one of the highest leakage levels among surveys was observed on a rule-based
funding (capitation grant) in Uganda. 20 Of course, this observation does not constitute a
contradiction to the hypothesis that rule-based funding is less prone to leakage. Indeed, in the
Uganda case, discretionary funding, which could have revealed even higher levels of leakage,
was not tracked.
Ultimately, in countries surveyed, it was generally observed that the greater the agent’s
discretionary power, the higher the leakage observed. Furthermore, funds capture, in countries
characterized by relatively important discretionary power and weak monitoring, has been
associated with groups’ bargaining power. In Uganda, for instance, large variations were
observed across schools in reception of public resources, which were explained by relative
bargaining power. Reinikka and Svensson (2001, 2004b) used econometric analysis to explain
these differences across schools. They showed that smaller schools and the greater presence of
19
It should be noted that Das et al (2004a, p. 25) are “agnostic” about the fact that such non-arrival rates at the
school level constitute good or bad service delivery, and whether leakage is indeed taking place given that for
discretionary funding, no fixed-rule governed the share of resources earmarked for service providers. Indeed,
normal administrative use at the provincial and district levels should be accounted for in this figure, as well as
delays in arrival, etc.
20
Average leakage level of 87 percent in Uganda during the 1991-95 period.
120
unqualified teachers were associated with less fund reception per student. However, schools in
better off communities experienced less leakage of funds. In particular, they estimated that a
10 percent increase in household income increases the amount of public funding that reaches
the school by 3 percentage points.
Similarly, in Zambia differences in schools’ bargaining power was put forward to explain the fact
that only about 20 percent of schools receive any resources from discretionary funding. Das et
al (2004a, p.41) argue that the few schools that received large amounts of discretionary funds
have greater bargaining power with higher administrative levels.
Source: Gauthier 2006, p.33
←
 Note 23: Measuring leakage on resources allocated with fixed/hard allocation rule
When the resource for which leakage is to be measured is allocated through a rule-based
formula (i.e., fixed or hard allocation rule) (such as in Uganda for capitation grant, i.e., specific
amount per student is allocated to a school based on its enrolment), then the measurement of
leakage at the facility level is done using the following standard formula:
Leakage is measured as the ratio between how much the facility actually received in financial
transfers during the period under study and the entitled funds, that is, how much it should have
received during that period.
In presence, for instance, of a capitation (i.e., per student) grant formula in education for
instance, to measure leakage one needs only to compare how much the school is entitled to
based on the number of students using the specific formula in place and compare it with the
amount received by the school.
Note that in the case of the Uganda PETS 1996, transfers to schools was measured as the sum
of financial and in-kind transfers. In kind transfers were only measured in quantities and value
of material was obtained through a generic price list (see Ablo and Reinikka 1998, and Reinikka
and Svensson 2001).
←
121
 Note 24: Measurement of leakage in case of discretion/ soft allocation
In the absence of hard/fixed rule of allocation, that is if the resources are allocated
through discretion of officials at the various levels, then the way leakage is measured is
affected as well as the sampling strategy.
 In the absence of fixed-rule, leakage rate is measured as the ratio between how much
the facility actually received and how much the central level (or other hierarchical level)
has sent to the facility.
 The following “narrow” leakage formula applies:
In addition to data on resources received by a sample of frontline providers (as in the case of a
fixed rule), in the absence of fixed rule, one needs to carefully collect data on the resources
disbursed/sent by the central, regional and district levels (Note that in a fixed rule context, the
specific allocation rule provides the denominator).
o Data has to be collected on the amount of financial transfers and value of all
resources sent to a region during the fiscal year and ultimately what reaches
facilities.
For instance, this approach was followed in the PETS/QSDS in the health sector in Chad in 2004.
Similarly, in the education sector in Mali, each school was assigned during the course of a year a
certain number of schoolbooks by the Ministry. The 2005 survey compared the book allocation
number as recorded in government documents (executed) and the number of books received
by the schools as measured in the survey based on school records (See Wane … for details).
←
 Note 25: Rapid Data Assessment
Some past surveys have failed because of skipping this essential step or inadequate RDA. Data
problems are frequent at various levels within the administrative system.
Realizing a rapid data assessment before designing the instruments and piloting a survey should
be a norm in all surveys. Reinikka and Smith (2004, pp. 52-53) write on that respect:
122
“A rapid data assessment may be necessary to determine the availability of
records at various layers of government as well as in the private sector,
particularly at the school level. Some studies have failed as the availability of
records in local government and schools was not adequately assessed
beforehand. It is important to verify the availability of records early on, even if it
means a delay and some extra up-front costs.”
Indeed, records could be badly kept, incomplete, unreliable and hard to understand. It is
important to know what information is recorded in official records, the quality and availability
of data, the aggregation level, periods available, consistency between levels, etc. There could
be widespread absence of adequate account keeping: financial data in particular could
especially be incomplete and uncertain.
Indeed, the most serious problem of PETS is often the nonexistence of records at the facility
level but even at the central level. Record keeping practices are often very poor and often only
involve storage on paper form. In some occasions, there could be electronic data, but it is not a
guarantee of greater accessibility.
In the Chad 2004 health PETS/QSDS, for instance, MoH personnel data was found to be
managed by one person and were on a computer without back up. However to get a copy of
the file was not feasible as it was found that the computer had a virus. The tracking of wage
expenditure was hence not feasible.
It is also not infrequent that newly appointed administrators or elected officials (e.g., health
facility committee) at the local or district levels have no records of previous fiscal year activities.
If information is of poor quality at the local government level (region or district) for instance,
this level could be bypassed and information could be collected only at the facility level (in
order to measure resources available for service) and at the central ministry level (in order to
know how much was officially sent).
■ See also Example 14 for RDA Questionnaires
←
 Note 26: Size of user surveys
The number of users interviewed per facility sampled varied somewhat from survey to survey.
In the 2002 PETS/QSDS in Mozambique for instance, approximately eight outpatients were
interviewed (for a total of 679 individuals). The selection procedure was sophisticated (see
Lindelow et al 2004). In Uganda and Chad, exit polls were used to interview about 10 patients
per facility (for a total of 1,617 individuals and 1850 respectively).
←
123
 Note 27: Rationale for collecting various categories of data
The rationale for potentially collecting these various categories of data is as follows.
(i) Environment and characteristics: Collecting data on the characteristics of the various agents
(i.e., administrative units and service providers) within the supply chain and the environment in
which they evolve is important in order to identify important particularities of the sector
studied. Such data include information on the location, ownership structure, size, etc. Data on
the service provision environment allow the assessment of the competitive environment. These
data could include information on the number and distance of other service providers, the
distance from market or from the capital which could have an effect on resource reception and
information, etc.
(ii) Financing: Financial data are important especially for PETS in order to track resources from
the central government unit through the supply chain down to the service provider. These
quantitative data are taken from the central government budget and records, as well as from
decentralized administrative and facility records. Financial data are also important in QSDS to
assess the efficiency in the usage of the resources available to service providers and equity in
the allocation of these resources.
(iii) Inputs: Measurement of inputs is essential for assessing resource use, transfers and
availability at the various levels. Among the main inputs that need to be collected is information
on material inputs such as staff and salaries, medications, vaccines, text books, blackboards,
equipments, etc.
(iv) Institutional arrangements and production process: One of the distinctive particularities of
PETS and QSDS, as mentioned, is their emphasis on institutional and production process data.
These institutional arrangements and mechanisms determine the incentive structure of the
personnel. This information allows understanding how resources are combined, utilized and
transformed at every level of the supply chain and how services to the population are
produced. These data include, for instance, information on management practices, supervisory
and reporting activities, rewards and penalties, etc. Various proxy of accountability are also
targeted, including information on supervision, reporting, auditing, performance assessment,
rewards systems, etc.
(v) Intermediate outputs: Intermediate outputs are outputs of the production process that are
not directly associated with the services rendered but which affect quality and performance
(OECD, 2000). For instance, staff absenteeism is related to the production process but does not
measure a final objective of the production process. Among the key information that need to
be collected are measures of access to services such as the variety of services offered, as well as
opening and closing time of facilities. It could include the physical distance of the targeted
population (e.g., walking distance to the health clinic or school). Some measures of quality of
124
services could also be proxied by physical and human capital, consultation duration, availability
of material, etc. Also, measures of affordability of services are important elements to capture.
These could for instance include direct costs (such as user fees, transport costs, informal
payments, bribes, etc) and indirect costs (opportunity costs, such as waiting time, etc.) (Amin
and Chaudhury, 2007).
(vi) Final outputs: Information on final outputs is related with what is directly valued by clients
of the service provision (e.g., patients, student). These could be for instance graduation rates or
the number of patients treated.
(vii) Final outcomes: Outcomes are the result of the interplay between government policies,
public interventions, frontline service providers, individuals (e.g., patients and students) and
communities. Final outcome indicators are overall measures of performance within the sector.
These could be based on objective measures of outcomes such as mortality rates or student
achievement, but could also be perception measure such as clients’ satisfaction.
While as in any surveys, an objective of parsimony in data collected needs to be respected in
order to minimize non-sampling errors, collecting information on these core areas could
provide a certain degree of standardization in tracking surveys and allow benchmarking and
cross-country comparability over time (Amin and Chaudhury, 2007).
←
 Note 28: Data sources records versus recalls
In order to minimize measurement errors, it is recommended to use records, accounts or
invoices to collect quantitative or financial data. In exceptional cases, when no other sources of
data are available and data based on recalls are collected, clear indications in that respect
should be reported.
Still, in some past PETS/QSDS surveys, respondent recalls were used because no records were
available at the facility level. Data quality is certainly in doubt in such situations.
In Ghana, for instance, the information collected from schools was based on recall data rather
than data obtained from school records or accounts, making the data significantly less reliable.
←
 Note 29: Length of quantitative data tracking
Completeness of data collected (or targeted) generally detracts from accuracy, as typically,
there will be more missing data for a longer time period. Given that data at the facility level
generally are in quite disaggregated forms, the work of the enumerator is made very tedious, if
not unfeasible, when too much data and too long a period are targeted.
125
In past surveys, there has been much variance regarding the length of data collected. In some
cases, data collection covered period of five years and sometimes only one month for specific
data. Such variations greatly depend, of course, on the difficulty of collection of the specific
data targeted and the recording procedures.
←
 Note 30: Seasonality of data
Short collection periods are generally associated with problems of seasonality, which could bias
the data. If data are collected on a monthly basis, for instance, there is of course a need to
aggregate the data on an annual basis.
While in the education sector, there are few problems of seasonality (except maybe for
dropout levels), in the health sector, seasonality is generally an important issue. If data are
compiled on a monthly basis and only certain months are available (or collected), aggregation
on an annual basis is problematic, as frequentation data and types of illnesses are potentially
quite different from month to month and across regions.
In general, it is better to collect annual data if they exist instead of monthly data, except if the
issue of seasonality of services is specifically targeted as a management or performance issue.
If only monthly data for a short time span are available, a strategy has to be devised to reduce
biases introduced by seasonality.
←
 Note 31: Problems with informal sampling
Some past PETS/QSDS samples have been chosen informally on the basis of their convenience
of access or ease of interviewing, instead of through the use of random selection techniques
(e.g., Tanzania and Namibia). The key problem with such informal methods is that valid
inferences about the whole population of facilities (as well as clients or staff) cannot be made
from the survey results, which considerably weakens its potential policy impact.
←
126
PETS/QSDS EXAMPLES
 Example 1: Some examples of PETS/QSDS motivations and objectives: Mozambique,
Zambia, Chad, Afghanistan
Mozambique 2002. A combined PETS-QSDS was implemented in the Mozambican primary
health care system in 2002. The health system in the country had rapidly expanded in the last
three decades. However, there was evidence of problems in service delivery, including low
quality, shortage of drugs and equipment, low staff morale and informal charging. (Lindelow et
al, 2004). Furthermore, the process whereby resources were allocated between line ministries
to districts and facilities was not well known. A further concern, as in most other tracking
surveys, was that resource allocated to the health system did not reach facilities that provided
services.
The objective of the tracking survey was a) to assess the functioning of primary health care
services, b) to provide quantitative and factual evidence on how the financing and logistical
arrangements for primary health care facilities operated in practice and to determine whether
resources reached their destination, and c) to assess the impact of these arrangements on the
capacity of facilities to deliver services. The survey further hoped to provide baseline data
against which policy reforms and progress could be assessed. The survey collected information
from public health care facilities and district administrations. A staff survey and a patient exit
poll were also administered. The tracking survey focused on the allocation and execution of
non-wage recurrent budgets through the administrative process, facility infrastructure, inputs,
outputs, management, user fees and human resources (Lindelow et al, 2004).
Zambia 2002. A combined PETS-QSDS was carried out in Zambia in 2002 in the basic education
sector. Education in Zambia was recognized by the government as a key component of
development in a PRS paper. In particular, the survey was intended to evaluate the impact of a
specific funding program to schools supported by international donors, the “Basic Education
Sub-Sector Investment Programme” launched in 1998 (Das et al, 2004a).
The survey traces rule-based and discretionary expenditures flowing from the central
government to schools via provincial and district administration, in order to identify leakage of
funds, inefficiencies and delays. The scope of the survey was extended by adding a household
survey, which allows exploration of linkages between frontline providers and users of services.
Furthermore, this was the first attempt by a PETS-QSDS to measure educational outcomes,
which it tries to trace back to educational expenditure. Along with the survey, pupils were
tested in two consecutive years, which allowed derivation of learning achievement measures.
Chad 2004. A combined PETS-QSDS was organized in the Chadian health sector in 2004. The
Chad PRS paper considered the health sector a central pillar of the government’s strategy to
revive growth and alleviate poverty. In a context of increasing oil revenues and public
expenditures in social programs as part of the PPTE Initiative, the government of Chad and
127
international donors wanted to ensure that funding would contribute to poverty reduction
objectives by reaching its destination and benefiting the population.
The Chad survey focused on the health system as a whole (primary, secondary and tertiary
health care providers).The objective of the survey was to compare resource access and level of
services in health centers of all the country’s regional delegations. Specifically, the objectives
were a) to analyze the impact of and bottlenecks and constraints in resource allocation and
their impact on efficiency and performance of health care facilities; b) to provide data on basic
characteristics of health centers, in particular those that significantly affect public service
quality; c) provide information on potential factors explaining differences in performance
between health facilities and types of ownership and identify means to improve the situation;
d) analyze the impact of service quality on households’ demand for health care service
(Gauthier and Wane, 2005).
Afghanistan 2010. A PETS was implemented in the Afghanistan education sector in 2010.
Various analysis, including a World Bank PER in 2009 and a Ministry of Finance’s review of its
experience with provincial budgeting pilots, observed inequities and delays in delivering
resources to provinces and local levels across sectors. The Government of Afghanistan was
interested in addressing these concerns and a Result-Based Budget Approach was to be
introduced to better link public expenditures and outcomes. The Ministry of Education in
particular had been working with key donors at improving its internal budget formulation
especially addressing issues of resource allocation across provinces.
In this context, the PETS in the education sector “aimed at tracking the quantities and timing of
subnational resource flows and identifying bottlenecks and leakage in the system” (World Bank,
2009). The overall objective of the study was to understand the mechanisms of resource flow in
the general education sub-sector. The focus was placed on wages and non wage recurrent
(Operation and Maintenance) expenditures. It seeks at identifying problems in the system
which may result in delays or leakage and addressing aspects of capacity at local levels.
←
 Example 2: What motivates religious not-for-profit health care providers in Uganda
Using a QSDS, Reinikka and Svensson (2010) explore what motivates religious not-for-profit
health care providers. They use a change in financing of not-for-profit health care providers in
Uganda to test two different theories of organizational behavior (profit-maker versus altruistic).
They show that financial aid leads to more laboratory testing, lower user charges, and increased
utilization, but to no increase in staff remuneration. The findings are consistent with the view
that the not-for-profit health care providers are intrinsically motivated to serve (poor) people
and that these preferences matter quantitatively.
←
128
 Example 3: Repeated PETS
Uganda. Repeated PETS were used in Uganda to evaluate the impact of an information
campaign designed to reduce resource leakage in education identified by an initial tracking
survey. (Reinikka and Svensson, 2004b).
Rwanda. Two PETS were implemented in Rwanda: the first was conducted in 2000 in the health
sector, while the second, in 2004, covered education, health, together with water and
agricultural projects. The objective of the first survey was to identify potential leakage and
sources of delays in the flows of funds from the MOF to service providers through the
administrative system, and to recommend measures to improve use and flows of resources.
The second PETS pursued various objectives, especially a) to assess the system of resource
allocation in the social sectors, b) to analyze the degree of compliance with procedures and
rules for public expenditures, c) to evaluate delays and leakage of public expenditures funds
and their impact on the attainment of planned objectives, d) to analyze account keeping and
reporting mechanisms, e) to evaluate the outputs and perception of the beneficiaries
In practice, the Rwandan survey was essentially an impact evaluation of three funding programs
(the Fund for Genocide Survivors (FARG), the Education Support Fund for Vulnerable and Poor
Children, and the Capitation Fund). It also comprised a diagnostic study of salary payments to
primary school teachers.
Madagascar. In Madagascar, three rounds of PETS survey were put forward in the education
sector starting in 2003 following the decision by the government to use part of the HIPC funds
to pay for the tuition fees of all public primary school children in order to try to increase
enrolment (Francken 2003). Three distinct surveys were implemented. The first was put
forward to evaluate the socio-economic situation following the political crisis of 2002. Public
primary schools were surveyed on financial data, especially financial contributions to the
schools. The second survey was organized to track budgets at the school level to quantify the
amount of funds and materials that should arrive and that actually arrive at the local facility
level from the two main funding programs in the education system (one administered by the
World Bank, the other by the MOE). The third survey was organized at the district level to gain
insights into budget allocation (World Bank/INSTAT, 2005; World Bank, 2007).
←
 Example 4: Delays in Zambia, Rwanda, Tanzania, and Nigeria
In Zambia, information was collected on delays (and arrival time) in the reception of salaries
and four types of allowances (compensation and time overdue) at the service provider level.
The four types of allowances studied were (i) Double-class allowance, paid to teachers who
teach more than their contract stipulates, (ii) Hardship allowances, paid as an incentive to teach
in rural areas, (iii) Student-trainee allowance, paid to teachers in their second year of training
who are assigned to rural areas, (iv) other allowances, which include compensation for
129
additional responsibilities at the school (Das et al, 2004a). For the salary component, the
payment system is efficient; over 95 percent of staff was paid on time and less than 3 percent
reported more than six months’ overdue pay. However, allowance disbursements have worse
records, depending on the type of allowance. Well-defined allowances (hardship and
responsibilities) are paid on time, while delays are observed in disbursement of the other types
of allowances. In particular, more than 75 percent of recipients of “double-class allowances”
(additional amount paid for overtime, etc) experience at least six month’s overdue pay.
According to the report, this appears to be partly due to lags in payroll updating.
In Rwanda, delays were observed in the payment of capitation grant funds to schools. About 13
percent of teachers did not receive their salaries regularly. More importantly, 82 percent of
teachers report salary arrears in 2003.
In Tanzania, REPOA/ESRF (2001) described the disbursement procedure between the central
level and facilities via district authorities. It measured the disbursement time between these
levels and delays for salaries and “other charges,” as well as for certain materials (books and
drug kits). Salaries appeared to be less prone to diversion than non-wage expenditures, but
delays in pay for frontline staff were also observed. The most important delays are reported for
non-wage resources in rural areas. The cause was linked to a cash budgeting system, which
lead to volatile transfers of resources due to fluctuations in revenue. Significant delays in the
allocation of non-wage funds at the central level were documented. In fact, delays were
observed in all districts studied. In some districts, no transfers at all were reported in some
sectors for the period under study.
In Nigeria, Das Gupta et al (2003) and Khemani (2006) found evidence of long delays in the
payment of salaries to health providers. In particular, extensive non-payment of salaries of
public health personnel was observed in one of the two states surveyed (Kogi and Lagos) where
42 percent of staff respondents reported not receiving a salary for six months or more in the
past year at the time of the survey. Results of regression analysis indicated significant
differences between Kogi and the other state surveyed (Lagos) in terms of the extent of nonpayment of salaries. It was further shown that non-payment of salaries could not be explained
by lack of resources available to local governments. Evidence suggested that non-payment of
salary was related to problems of accountability at the local government level in the use of
public resources (Khemani, 2006). Furthermore, it was observed that non-payment of salaries
had an impact on provision of services as the greater the extent of non payment of salaries, the
greater the likelihood that facility staff acted as private providers (more service provided
outside the facility, expropriation of drugs from facility stocks, etc.).
←
130
 Example 5: Teacher and Health Worker Absenteeism in India
In India, one-quarter of government primary school teachers were absent from school, but only
about one-half of the teachers were actually teaching when enumerators arrived at the schools
(Chaudhury et al, 2006). The authors analyze the high absence rates across sectors and
countries; investigate the correlates, efficiency, and political economy of teacher and health
worker absence; and consider implications for policy.
←
 Example 6: Teacher Absenteeism in Zambia
Das et al. (2005) explored the relationship between teacher absenteeism and students’
learning. Absenteeism in Zambia was shown to be associated with shocks, caused often by
illness or death rather than overall lack of motivation. The authors showed that teachers work
harder to compensate for such absences but that children with a frequently absent teacher
may fail to improve in their test scores. The findings suggest that programs to allocate
substitute teachers could significantly improve education outcomes in such an uncertain
environment.
←
 Example 7: User fees in Chad
In the Chadian health sector, contrary to the conclusions of previous studies (Ministère de la
Santé publique, 2001; World Bank, 2004), the tracking survey discovered that the most
important source of health center financing is user fees. As most public health expenditures in
Chad are consumed by the central and regional administrations that do not provide direct
services to the population, very little is left for frontline service delivery. Once labor resources
are allocated, health centers are left to their own devices to finance their activities through user
fees. Indeed, government transfers account for only 2 percent of health center revenues
(excluding salaries) and for one quarter of their revenues, including salaries. The impact in
terms of access is significant, since the health centers tend to charge higher user fees to make
up for their lack of resources (Gauthier and Wane, 2008b).
←
 Example 8: User fees in education: Uganda, Zambia, Rwanda
In the education sector in Uganda, the tracking survey found that private contributions
represented more than 60 percent of education finance at the school level in the early-1990s.
Subsequently, fees were abolished (although the repeat PETS showed that some fees continued
to be collected) (Reinikka and Svensson, 2001).
131
Similarly, in Zambia despite an official policy of abolishing Parent Teacher Association (PTA) fees
in order to increase enrolment (PTA fees were abolished for primary and basic schools in April
2002, three months before the fielding of the survey (Das et al, 2004a, p. 51)), the survey
estimated that private education expenditures were prominent in education costs. Das et al.
(2004a) estimated that education fees and other private non-fee expenditures (textbook,
uniforms, etc) represent on average between 54 percent and 67 percent of total education
costs for rich and poor households, respectively.
In Rwanda the ministry of education introduced a policy of “education for all” to improve
access to basic education. Under this program, the ministry pays education fees for all students.
Basic education has thus become free and compulsory. However, the survey report notes that
the disbursement of public funds to schools is irregular and insufficient to cover school
expenditures. Schools then ask parents to make additional payments, which far exceed public
funding. (In the capital, these school fees ranged from 400-2000 Rwf per student per term,
while in the province, they ranged between 5 and 500 Rwf (Government of Rwanda, 2004)).
School authorities reportedly send home children who were unable to pay school fees
(Government of Rwanda, 2004).
←
 Example 9: Measuring efficiency
Lindelöw et al. (2004) examined the question of productivity of health centers using PETS/QSDS
data in Mozambique. They analyze seven categories of service output and devise a composite
index of output to deal with the problem of multi-output production. They observe significant
urban-rural and regional differences in service output per capita. The authors note important
variations in output per health worker across districts (8 to 1 ratio). They find that the low
productivity observed in certain districts may be due to low density, and consequently, that low
productivity may be a cost of extending service to sparsely populated areas in the interest of
equity.
Lindelöw et al. (2003) examined the question of efficiency in Ugandan health clinics using QSDS
data. They make use of an output weighted index similar to the one used in Mozambique to
measure health workers’ output for different categories of facilities. They note very important
differences in outpatient-equivalent service units per workers across facilities in the country (in
a 50 to 1 ratio). Important differences in output per worker are also observed across ownership
categories, with lower levels observed among non-profit facilities.
Gauthier and Wane (2005) also examined differences in productivity among health facility
ownership categories in Chad using PETS/QSDS data. They observe that religious not-for-profit
facilities are the most efficient providers for a variety of performance indicators (including the
average number of outpatients and the number of patients per staff member). A typical
religious not-for-profit facility in Chad treats 2,300 more patients annually than the average
government provider of primary health care. Employees in religious not-for-profit facilities are
132
also the most productive, offering health care to over 270 patients more per year than the
average health worker. This difference was 970 patients annually when the sample is restricted
to qualified workers.
Over et al. (2006) examine efficiency of health service delivery in six countries, focusing
especially on HIV prevention. They apply a technical efficiency approach based on stochastic
production frontier to estimate the determinants of relative productivity. They evaluate costeffectiveness of service delivery, controlling for quality of health care.
←
 Example 10: Impetus in Uganda, Ghana and Chad
In Uganda, the central government had very little information about resource use and reasons
for poor outcomes, but was suspecting that local governments, recently created through a
decentralization policy, might be diverting resources for other purposes. The objective of the
1996 PETS was therefore to track expenditure flows in the hierarchical structure in order to
identify factors explaining these poor results, and to measure potential leakages in school and
health facility funding.
In Ghana, the motivation of the 2000 PETS arose from Ghana‘s Poverty Reduction Strategy
(PRS), which recognized that a significant gap existed in the access and utilization of basic
services by the poor. Issues that were identified as crucial were quality, equity, efficiency and
financing gaps (Ye and Canagarajah, 2002). With these considerations in mind, the objective of
the tracking survey was to improve the efficiency of public spending and improve outcomes in
social sectors. Decentralization policies had been put in place to improve the availability and
access to services. The PETS was consequently intended to enhance transparency and
accountability of public expenditure allocation. As in Uganda and Tanzania, the PETS
implemented in Ghana sought to estimate leakage of public funds in the transfer process from
the central government to public service facilities through district authorities, in basic
education and primary health care.
In Chad, in the context of increasing oil revenues and public expenditures in social programs as
part of the PPTE Initiative, international donors wanted to ensure that increased funding would
contribute to poverty reduction objectives by reaching its destination and benefiting the
population. The objective of the 2004 health survey was to analyze the impact of and
bottlenecks and constraints in resource allocation and their impact on efficiency and
performance of health care facilities; to compare resource access and level of services in health
centers of all the country’s regional delegations; and analyze the impact of service quality on
households’ demand for health care service (Gauthier and Wane, 2005).
←
133
 Example 11: PETS/QSDS Workshops in Afghanistan and Morocco
In Afghanistan, a workshop was organized in September 2009 to discuss basic education subsector challenges in preparation of a PETS put forward in 2010. The “Workshop on Provincial
Resource Allocations” sought to support the Government of Afghanistan’s budget process
reforms to better connect and align the national budget to the Afghanistan National
Development Strategy (World Bank, 2009:7). The overall aim of the workshop was to bring
together key actors (in particular the Ministries of Finance and Education and sub national
levels) to develop a common framework for addressing issues of efficiency and equity in
resource allocation. More specifically, the workshop sought at identifying problems and causes
of inefficient delivery of education services and infrastructure, especially at sub-national levels.
(World Bank, 2009)
In Morocco, a seminar was organized in Rabat in June 2010 in preparation of a PETS/QSDS in
the health sector to present the various service delivery tools. It assembled people from various
ministries (e.g. Ministry of Finance, Ministry of Public health) and civil society. The theme
examined was “Are You Being Served?” based on the book of the same name. It helped inform
and assembled a group from various ministries to discuss key challenges facing health services
and examine new tools and methodologies to assess efficiency and equity in service provision.
←
134
 Example 12: List of Tasks and Project Timeline Chart
PETS/QSDS List of Tasks and Activities
Tasks
1.Background
and
Institutional
Mapping
Report
Activities
Contents
a. Collect
relevant
documentation
and data
Collect and review the relevant documentation related to the sector and programs under study.
Documents to be collected include:
 Government sector documents and program reports;
 Government financial and budgetary documents from the Ministry of Finance and line ministries
 Sector administrative data and reports, such as routine information system data and annual
reports;
Review the main aspects of the sector/programs, the core strategies and policies, the variety and levels of
funding flows, and identify the main challenges facing the sector/program.
Following the review of documents, conduct a thorough analysis of institutional arrangements and policy.
Through interviews with key informants within central and sub-national levels down to frontline providers
and beneficiaries, and field visits, identify and describe the policy environment and administrative
arrangements and rules governing the allocation of resources within the program.
In particular:
i. Identify resources mobilized in the sector/programs: Identify the sources of funding and
importance; main components of the programs, the types of transfers and benefits, targeted
beneficiaries, the types of resource transfers and benefits.
ii. Ways in which these resources are mobilized and channeled; allocation mechanisms of resources
in the sector/programs and various channels toward users; Describe the modes of allocation, how
the public hierarchy is structured, and the roles and responsibilities of various administrative
levels and units in the budget execution processes toward frontline providers and beneficiaries;
criteria for allocation. Identify the allocation rules or mechanisms used to allocate the resources
within the sector/programs at the various administrative levels;
iii. Ways in which these sector/program funding are used at various levels: usage at central, provincial
and decentralized levels (transfers to beneficiaries vs other usage of the resources (salaries, goods
and services etc). For instance, flows could be divided into three categories: Monetary (cash
transfers); Materials (such as textbooks, drugs, or equipment); Payroll (remuneration of staff).
Further subcategories, such as investment expenditures, could be introduced along functional
classifications in the budget.
b. Analyze
institutional
arrangements
and policy
environment
135
Tentative
timeline
iv.
c. Identify
tracking flows
d. Draw
resource flow
diagrams and
identify risk
areas
e. Formulate
questions and
hypothesis
f. Rapid Data
Assessment and
Assembling of
existing data
Results obtained by these resources: transfers and services provided (impact and outcomes if
feasible).
v. Accountability and information systems. Describe the information system and reporting
mechanisms at each level for the sector/programs and delivery paths for the different sector
inputs; and assess the accountability framework.
vi. Identify the main issues and challenges facing the program, which affect service quality;
During the analysis, ensure the distinction between official and effective rules and procedures (e.g., for
resource allocation, accounting, recording, or monitoring) that are observed in practice for the different
flows and levels. Rules and procedures effectively used to allocate resources (or recording funding flows)
could differ from official rules and may vary from one location to another (i.e., district), and should be
understood and analyzed.
Identify cash flows (or resource flows) for monitoring/tracking. Select funding or resource flows for which
financial and quantitative data (primary and secondary) will be collected and at which levels.
For each resource to be tracked, draw resource flow diagrams illustrating how various resources flow
within the sector/programs through each administrative node of the public administration toward service
providers and beneficiaries. Identify key “decision points”, that is, places where resource allocation,
deployment, and procurement are made and decision rules.
Identify higher risk decision points and weak nodes in the supply chain that could affect the quality and
availability of services within the program to focus the analysis and data collection strategy on these risk
areas and resources.
If expenditure allocation at some levels are made through centralized computer based accounting and
payment systems (e.g. BAS), the budget procedures at the various levels need to be detailed in tables
(including the budget and accountability procedures, specific budget lines, source of information used,
actors involved and time period during the year)
Once institutional arrangements understood and main risk areas identified, translate the study’s broad
objectives into specific objectives and research questions that will drive the data collection strategy.
Formulate hypotheses to explain the problems facing the service delivery system or program based on
evidence and understanding of the system. These tentative answers to each research question will help
determine the specific data required to test these hypotheses.
-Identify main primary data to be collected and verify their availability. Verify the existence of records
(receipts of resources sent by the government, services rendered, etc.);
-Verify data availability, quality, and consistency given that there could be differences across geographical
locations (e.g., rural vs. urban)
136
2.Design PETS
instruments
3.Sampling
strategy
4.Field
deployment
strategy
5. Survey
manuals and
protocols
6.Pre- test of
instruments
7.Training
8.Finalization
of
instruments
-Identify and evaluate the sources of existing/secondary data and documents on sector financing,
spending, activities, and output. Copies of relevant documents and electronic records should be collected.
These data are dispersed among various services and government agencies. If expenditure allocation at
some levels are made through centralized computer based accounting and payment systems (e.g. BAS),
specific budget lines have to be reviewed and detailed. Collect budgetary data as well as routine
administrative data (e.g. Management Information Systems, MIS).
Design PETS survey instruments to collect the relevant data required at each level to verify the hypothesis
formulated to explain inefficiencies or iniquities in the system and to systematically measure leakage
levels, etc.
There should be a module to improve awareness of beneficiaries on the government’s assistance as well as
government’s accountability/responsibility to realize this entitlement
Develop a sampling strategy to obtain a representative sample of population under study; collect sample
frame data: computer-based lists of various units and households, vulnerable children etc using census or
statistical services within province.
Develop a field deployment strategy to collect data within time and resource constraints. The deployment
strategy seek to allocate teams and supervisors in the field in order to achieve, in the most efficient
manner, the service facilities and other administrative units while respecting supervision objectives and in
the delays set by the time schedule. Possibilities of return visits need to be accounted for in the survey
schedule.
Prior to the training, a detailed enumerator and supervisor manual, including a survey protocol describing
the main procedures for survey implementation, should be prepared. The manual should discuss the
questions of each survey instrument one by one, explaining the rationale of each question and potential
interpretation problems. The manual should be used in the training sessions and will also support the
teams’ field work.
Pre-test all questionnaires among a small number of units (i.e. various administrative levels covered in the
survey (including provincial/regional or district, facilities, staff, users, households, etc.) using all the
instruments developed in urban and rural areas.
Classroom and on-site training of all survey enumerators and supervisors to ensure that all questions and
procedures are well understood. Training workshop includes field pre-pilot of the questionnaire to be
carried out by the enumerators and supervisors to test all questionnaires and to train enumerators and
supervisors in the field among a small number of units at various levels.
Following the pre-pilot, a final revision of instruments should be carried out. This includes revising wording
of questions, ambiguous responses, format of the questionnaires, and answer codes.
137
9.Data
collection
10. Data entry
11.Data
cleaning
12. Data
analysis and
reporting
13.
Consultations
14.Disseminat
ion of results
The data collection/pilot could be conducted once all the previous steps are completed. The fieldwork
supervisors must devote time and attention to supervising the work of the enumerator teams. Completed
questionnaires should be transmitted to data entry operators weekly or bi-weekly.
Data entry programs should be written following the completion of the questionnaires and should be
tested during the survey pilot phase. A training workshop should be held for data entry operators and data
entry supervisors to ensure proper understanding of the instruments, data entry programs, and
verification mechanisms. Data entry should start at the beginning of survey implementation and should be
completed promptly following the end of data collection.
Data cleaning should be done shortly after the end of data collection. Data cleaning includes: Standard
tests to identify outliers for each variable; A brief data cleaning report summarizing the data cleaning and
including a statistical description of variables; The presentation of a table of summary statistics for all
variables, including a short evaluation of the data quality indicating any weaknesses or other issues that
will be relevant in the analysis.
Reports should clearly identify and communicate the specific findings, in particular: Measures of leakage at
each level, delays and other inefficiencies and ineffectiveness; Recommendations to be implemented;
At minimum, report contents should include background information on the state of service delivery in the
sector, descriptive and analytical information on various units analyzed in the survey work (local
governments, facilities, staff, clients, etc.), evaluation of efficiency and equity in service delivery, funding
disbursed at the central level and received at the decentralized level, etc.
Consultation with different stakeholders
Dissemination strategies should be developed. These could include information activities and take the
form of seminars or workshops to present the findings and implications in public gatherings and among
the stakeholders. During the activities, elicit views on how to promote reforms and improvements in
service delivery and population outcomes.
Source: Adapted from UNICEF (2011) “PETS Pilots in Dien Bien and HCMC in Vietnam”
138
Project Timeline Chart 21
Month 1
1
Week
2
3
Month 2
4
1
2
Month 3
3
4
1
2
3
4
Phase 1: Background analysis
Review of documentation
Analysis of the policy framework and
institutional arrangements
Phase 2: Preparatory stage
Research questions and consultation with
key stakeholders
Rapid Data Assessment (RDA)
Phase 3: Instrument design
Design of survey instruments
Preliminary pilot of instruments
TOR for survey implementation
Month 4
1
Week
2
Month 5
3
4
1
2
3
Month 6
4 1
2
3
Month 7
4 1
Phase 4: Fieldwork
Training and field pilot of the
instruments
Data collection
Phase 5: Analysis / report writing
Data cleaning and variable
construction
Statistical analysis
Budget data analysis
Report writing
Phase 6: Dissemination
Publication of draft report
Presentation at dissemination
workshops
Revisions and final report
21
Source : Adapted from ODI (2011)
139
2
Month 8
3
4 1
2
Month 9
3
4
1
2
Month 10
3
4
1
2
3
4
←
140
 Example 13: South Africa PETS/QSDS
In South Africa where a PETS/QSDS on Early Childhood Development programs was put forward
in 2009-10, important differences were noted regarding record keeping quality across the three
provinces surveyed (Western Cape, Limpopo, and North West).
In Western Cape, electronic data existed at the provincial level with disaggregation at the
district level and good quality data at the facility level. In Limpopo, electronic data could not be
compiled below the provincial level. In this context, following the RDA, the empirical strategy
was adapted to bypass data collection from district level and only collect information from the
provincial and facility, using electronic records at the provincial level and questionnaires based
on facility records at the provider level
←
 Example 14: Objectives of field Visits and Contents of Rapid Data Assessment
Objectives of Field visits part of the institutional analysis
The field visits seek to gather information for the institutional mapping report and Rapid Data
Assessment (RDA). Meetings should be organized with all the ministries, departments,
administrative levels, districts, communes, frontline providers (e.g. schools or health centers)
and other actors involved in the planning and implementation of the sector/programs.
Meetings should also be held with a few beneficiaries of the programs: patients, households,
community representatives, etc. The main objectives for the field visits include:
a) Identifying the source of funds for the sector/programs under study;
b) Developing an understanding of the responsibilities of key actors;
c) Obtaining greater details on the sector/programs administrative and expenditure system and
procedures;
d) Exploring the types of transfers and how they're implemented in practice;
e) Observations on the stated vs. actual implementation of the program in areas such as
identification of beneficiaries, budget allocation and receipt etc;
f) Identifying sources of secondary and primary data;
g) Gathering detailed budget information for the last two complete fiscal years for the
sector/programs under study;
h) Undertaking a rapid data assessment to identify available data and what isn't to help refine
the aim and scope of the tracking exercise.
141
Vietnam PETS pilots 2011
Potential questions for Rapid Data Assessment and Institutional Mapping
visits22
District and Commune levels
Target respondent: Head of the District or commune, most senior officer or officer in charge
0. name of respondent, title and phone number (in case further information is required)
1. What are your agency’s main roles and responsibilities in the planning and
implementation of the programs?
2. How many communes (or boarding schools) are part of your jurisdiction?
3. Please provide a list of the personnel of your district/commune currently involved in the
sector/program planning or implementation:
-Category of personnel, number of staff in each category, share of their salaries paid by
the programs
4. Do you keep accounting data of the programs’ resources receipts and expenditures? Are
these data available for FY 2009/10 and FY2010/11?
5. What are the resources received by your district/commune part of the programs in FY
2009/10 and FY 2010/11: specify the value of the support by sources (provincial
ministry, NGO etc) and categories:
Budget categories
Amount
FY2009/2010
FY2010/11
Financial resources (non
wage)
Salaries and wages
Goods and services
Infrastructure
Other (specify)
Note : One table by source if applicable
6. At what frequency do you receive these resources (monthly, weekly, daily, etc)?
22
Source : Adapted from UNICEF (2011) PETS Pilots in Dien Bien and HCMC”
142
7. What are the mechanisms through which you receive resources? (Ex: electronic
payment systems from provincial treasury to districts, communes accounts, central
payment system, etc.).
8. What were the expenditures of the district/commune part of these programs by
category (for both FY 2009/10 and FY 2010/11)?
Expenditure categories
Amount
FY2009/2010
FY2010/11
Salaries and wages
Goods and services
Capital expenditures
/infrastructures
Transfers
Other (specify)
9. Could you specify the value of the support (resource transfer) provided by your
district/commune toward lower levels/service providers during FY 2009/10 and FY
2010/11
Amount of support
Name of commune or school
FY2009/2010
FY2010/11
10. At what frequency do you provide these supports (annually, monthly, etc)?
11. How do you provide such support/transfer of resources? (Ex: electronic payment
systems from district/commune to commune/school account, etc) and how do you
account for these expenditures? (Ex: electronic system, paper record, etc.)
12. What data on financial transfer, activities and services part of the programs do you
keep? (For instance: financial transfers, number of beneficiaries by categories, number
143
of schools, households, grants provided, % of target population covered, graduation
from program over time, etc). (Identify and collect for last two FY)
13. How are accountability and supervision exercised? Are there field supervision visits of
communes or schools under your jurisdiction (if yes, purpose, intervals, etc.)? Do you
verify the list of beneficiaries, if they meet requirements, etc?
14. Do you report administrative and financial data to higher level jurisdiction periodically?
If yes, what is reported and at what periodicity? (take a copy of a report or information
if available)
15. In your view, what are the main challenges/problems currently facing the
sector/program?
- Inefficiency: too high administrative costs, delays, etc;
- Inequity among groups or communities, etc;
-Inadequate incentives provided to schools, households, students;
- Inadequate targeting of beneficiaries: target population not reached;
- Other (specify).
Frontline Provider: ex. Primary school
Target respondent: Head of the school or most senior officer or officer in charge
0. Name of respondent, title and phone number (in case further information is required)
1.
2. Characteristics of the school(community)
-Name of the school
-Location: city or village name
-Urban or rural area (mountainous area, etc)
-Type of school (ex. primary, secondary, etc)
3. Number of students (by levels)
Level
Number of students
2009/2010
2010/11
1
2
3
4
5
6
144
4. Number of students benefiting from program’s support (by levels and type of support if
applicable)
Number of students benefiting of the program
Level
2009/2010
2010/11
1
2
3
4
5
6
5. What are your school/organization’s main roles and responsibilities in the planning and
implementation of the program?
6. What are the resources received by your school part of the program (for FY 2010/11 and
FY 2009/10): specify the value of the support by sources (commune, district, NGO etc)
and categories:
Budget categories
Amount
FY2009/2010
FY2010/11
Financial resources (non wage)
Salaries and wages
Goods and services
Infrastructure
Other (specify)
Note : One table by source if applicable
7. At what frequency do you receive these resources (annually, monthly, etc)?
8. What are the mechanisms through which you receive resources? (Ex: electronic
payment systems from provincial treasury to districts, communes accounts, central
payment system, etc.).
9. Type and value of support provided by your school to beneficiaries part of the program
(by category of beneficiaries and type of support)
Type of
beneficiaries
(sub program)
2009/10
Value of
support
Number of
beneficiaries
145
Type (sub
program)
2010/11
Value of
support
Number of
beneficiaries
10. What data on financial transfer, activities and services part of the program do you keep?
(For instance: financial transfers, number of beneficiaries by categories, number of
schools, households, grants provided, % of target population covered, graduation from
program over time, etc). (Identify and collect for last two FY)
11. How are accountability and supervision exercised? Are there school supervision visits by
communes (if yes, purpose)? How many last year?
12. Do you verify the list of beneficiaries, if they meet requirements, etc?
13. Do you report administrative and financial data to higher level jurisdiction periodically?
If yes, what is reported and at what periodicity? (take a copy of a report or information
if available)
14. In your view, what are the main challenges/problems currently facing the program?
- Inadequate targeting of beneficiaries: target population not reached;
- Inefficiency: too high administrative costs, delays, etc;
- Inequity among groups or communities, etc;
-Inadequate incentives provided to schools, households, students;
- Other (specify).
←
← Return RDA
 Example 15: Mozambique PETS/QSDS 2002 health funding flows
In Mozambique, as reported by Lindelow et al (2004), the health sector is composed of a threetier administrative hierarchy responsible for the management of health services (Figure below).
Health providers (hospitals, health centers and health posts) are under the administrative
responsibility of District Directorates of Health (DDH). Their responsibilities include planning
human resources and budget management, provision of medical supplies and supervision of
service providers at the district level.
DDH are under the responsibility of Provincial Directorates of Health (PDH), which coordinates
the administration of health activities in the province. They enjoy considerable autonomy with
respect to allocation of staff, medical and non-medical resources and supervision. Provincial
directorates report to the Ministry of Health (MOH), which allocates their resources.
Public health care providers in Mozambique, with the exception of hospitals, do not receive any
financial resources from the public administrative system. The only sources of financial
resources at the provider level are user fees charged for consultation services and drugs. Inkind resources (medical and non-medical materials, drugs and vaccines) are allotted to service
providers through a complex administrative and logistical system. The DDH has a central role:
distributing in-kind resources to health centers and paying salaries to health workers. Some of
these resources are procured directly at the district level through a district logistic unit, but
146
other non-wage recurrent resources (e.g., drugs, vaccines, medical equipment) are procured
directly by the provincial or central levels. Further, the investment budget is the responsibility
of higher levels.
FIGURE : FLOW OF RESOURCES IN THE HEALTH SECTOR IN MOZAMBIQUE
Donors
Central govt.
Central logistics units
Financial
support
In-kind
support
Salary
budget
Procurm.
Procum.
drugs,
vaccines,
equipment,
etc.
Non-slry
recurr.
recur.
budget
Provincial admin.
Vaccines
Med.
equip.
Drugs
Provincial logistics units
Financial
support
Salary
budget
In-kind
support
Procurm.
equipment,
material,
etc.
Non-slry
recurr.
budget
District admin.
Vaccines
Med.
equip.
Drugs
District logistics units
Financial
support
Procurm.
equipment,
material,
etc.
Salary
budget
Health facility
Source: Lindelow (2006)
←
147
Vaccines
Med.
equip.
Drugs
 Example 16: Chad health funding flows
In Chad, as reported by Gauthier and Wane (2005), the health system is composed of a mixture
of public, private for-profit and non-profit service providers. As it is the case in most countries,
the public health system is structured around four levels of responsibility (see Figure below). In
some countries (such as Nigeria or Uganda), the administrative structure governing social
service providers’ resources flows comprises only two layers (e.g., central and district/local
government) instead of three.
At the central level, the MOH is in charge of formulating health policies and allocating financial
resources. At the intermediary level, 14 Regional Health Delegations (RHDs) are in charge of the
implementation strategy and managing health personnel. At the decentralized level, 49 sanitary
districts are in charge of implementing services within responsibility zones and overseeing
health centers (Gauthier and Wane, 2005).
Public health care providers in Chad (except some hospitals) do not receive any financial
resources from the public administrative system, only in-kind resources. The user fee is the only
source of financial resources at the local provider level. The central government ministries
allocate resources (human, financial and/or in-kind) to the various layers of the administrative
structure. These resources pass through various channels. Financial and in-kind resources might
be the responsibility of a line ministry while human resources could be the responsibility of a
civil servant ministry. Public resources arriving at decentralized levels from the Ministry of
Health (MOH) are the sum of four components, namely (1) centralized credits, (2) decentralized
credits, (3) ad hoc requests, and (4) medication and vaccine delivery. Centralized credits are
resources purchased by the MOH (essentially materials and medications), intended for regional
and district administrations and health centers. Delegated credits are budgetary resources
given to regional or district administrations. Resources are also transferred through ad hoc
requests; these materials are directly requested from the MOH and collected by regional health
delegates and district heads in the capital.
At the decentralized level, regional health delegates or district head doctors are responsible for
managing these financial resources, and for redistributing purchased materials and medications
to health providers under their jurisdiction. Moreover, the MOH administers a budget for
medications and vaccines destined for health centers and hospitals. Medications and vaccines
are formally included in the centralized credits, but follow a different path through the CPPA
and PPAs (Gauthier and Wane, 2005).
The budget is administered by the Central Pharmaceutical Procurement Agency (CPPA), which
has a monopoly over drugs and medical products sold to the 14 Prefectoral Purchasing
Pharmacies (PPP), as well as to the public and non-profit health facilities.
148
FIGURE : FUNDING FLOWS IN THE HEALTH SECTOR IN CHAD
Donors
MOF
MOH
MHR
CPPA
Decentralized
Credit
Regional
Health Delegation
PPA
District
Health Delegation
Service
Providers
Source: Gauthier and Wane (2005)
Funds
In-kind Transfers
Staff Compensation
←
 Example 17: Zambia education
In Zambia, in the education sector, public resources also flow through three administrative
levels before reaching primary schools (Das et al, 2004a).The administrative structure
comprises the Ministry of Education (MOE), province office and district offices (Figure below).
Two distinct types of provinces are observed, decentralized and centralized provinces. District
education boards were created in the first type, but not in the second. Resource flows in the
administrative system do not follow a simple top-down approach. At each level of the
hierarchy, funds may be received directly from the central government or donors. In the case of
centralized provinces, all funds are first allocated to provinces, which then determine the
allocation among districts, while in the case of decentralized provinces, a portion of the
discretionary funds are allocated directly from the MOE to districts, which then redistribute the
funds to schools.
In both centralized and decentralized provinces, donor funding administered by the MOE is
partly allocated directly to districts (BESSIP program).
149
There is no movement of rule-based funding from the provinces to districts; all of these
transfers are made directly from the MOE to districts. Discretionary funding (cash transfers)
transit through the province in centralized provinces (including MOE and case IV donor fund),
but is received directly by the districts in decentralized provinces. At the school level, tuitions
and other fees are paid by households.
FIGURE : FUNDING FLOWS ACROSS CENTRALIZED AND DECENTRALIZED PROVINCES
IN THE EDUCATION SECTOR IN ZAMBIA
Case IV donors
(primarily the Program
for Advancement of
Girl’s Education)
Ministry of
Education
Province
(Centralized)
Province
(Decentralized)
District
(Centralized)
District
(Decentralized)
Discretionary flows
Households
Schools
Rule-based flows
Source: Das et al (2004a)
←
 Example 18: Senegal and Cameroon
Among the most complex administrative processes observed in tracking surveys realized to
date in Sub-Saharan Africa are found in Senegal and Cameroon. The PETS reports (République
du Cameroun, 2004; World Bank, 2006) describe in some detail the budgetary and
administrative processes, which in Senegal involve nine procedures, including a Presidential
committee for the approval of the district level budget.
150
At the other end of the spectrum, the administrative structure of the education hierarchy in
Uganda examined in the first PETS was relatively simple. The structure of responsibility involved
the MOE and district education boards, which are directly responsible to the MOE and
supervise school facilities. As in Zambia, private funding to education was considerable at the
time of the survey; parents were expected to pay for textbooks, uniforms, school supplies, PTA
dues, etc.
←
 Example 19: Zambia classification of funding flows
In Zambia, as reported by Das et al (2004), funding to schools is composed of five types of funds
and resources:
Cash flows: Fixed grant component
The fixed-grant is a rule-based component of school funding. In Zambia, it is a per-school
allocation of either $600 or $650, and no level of the administration has discretion over the
amount disbursed. No information is required for the disbursement of funds as in the per-pupil
allotment in Uganda, where data is required on enrollments in the school, and which may be
subject to distortions at the level of the school or district.
Cash flow infrastructure grant
In Zambia, schools also receive money for rehabilitation of classrooms or for new construction
(infrastructure). This money is disbursed through the microprojects unit of the Zambian Social
Investment Fund, but the district retains considerable discretion over disbursement. Such
grants are part of the discretionary component of school funding.
Cash flow other grants
Schools may receive additional money from the GRZ or Case IV donors (mostly funding through
the Program for Advancement of Girls’ Education), and these resources are distributed entirely
at the discretion of the district. Consequently, as with (2) above, these grants are classified as a
discretionary component of school funding.
Flows of material
Schools may receive educational materials such as textbooks and chalk, again allocated at the
discretion of the district, with some input from the provinces and the Ministry of Education.
Although it is technically possible to track such resource flows, the procurement of educational
materials had not yet been completed at the time of the survey (midway through the academic
year). As a result, schools received such materials sporadically and in very small quantities. This
component is excluded from the exercise.
Payment of staff
Staff remuneration can be thought of as arising from rule-based and discretionary components.
Specifically, the salaries and allowances of teachers are paid through the centralized payroll
151
system. Neither provinces nor districts have discretion over the salary or allowances that a
teacher receives, so this is identified as a rule-based component of staff payment. On the other
hand, payments of one-time benefits are left to the discretion of districts and provinces, so this
is identified as a discretionary component. Since most staff remuneration is a direct flow from
the center to the teachers, payroll funds are omitted from the tracking exercise.
←
 Example 20: Chad health PETS/QSDS
In Chad, for instance, the information flow system at the decentralized level is characterized by
the existence of four main flows (personnel, centralized credits, decentralized credits and ad
hoc deliveries). Regional health delegates (as well as regional hospital heads) are informed at
the beginning of the year of their annual decentralized financial credit allocation as well as the
personnel allocated to their region. In contrast, centralized funds, which are converted at the
MOH level into in-kind transfers (medical material, medication, etc) are discretionary and thus
not accompanied by information about their official allocation to regional levels. Note that
district health delegates and facility heads do not receive any budgetary information about
resource transfers except for the quantities that actually arrive at their administrative unit
level.
Indeed, regional delegates are thus responsible for allocating financial or in-kind resources to
lower levels of the administration, but without any information flows. In-kind transfers sent
from the MOH central warehouse arrive at regional warehouses. A “Bon de sortie” is used at
the central level and a “Bon de livraison” is used at the regional level to confirm reception of
specific quantities of materials. A copy is sent to the center, which is then dated. In addition,
some information also flows bottom-up, as regional and districts heads are allowed to transmit
specific resources requests to the MOH, through the ad-hoc delivery channel. As mentioned
above, ad-hoc deliveries are specific supply pickups made by health officials (regional or district
officers) and thus do not involve any other information. Information about services performed
at the local facility level is provided monthly by facility heads through a monthly activity report
(RMA) to the district official. Information compiled at that level is then sent to regional health
administrations and then to the MOH.
←
 Example 21: Flow Charts in Ghana education
The 2002 PETS education survey in Ghana was designed to capture quantitatively specific
resource flows (materials or financial) at each administrative office where the public
expenditures are handled toward primary and scondary schools
Four expenditures were tracked:
(1) Text books and Stationary
(2) Capitation Grants
(3) Investment Expenditures in Basic Education.
(4) Service Activities (Item 3 Expenditures)
152
Note that expenditures go through different channels before reaching schools, even for the
same expenditures. Ex: while the schools books are distributed by District Education Office
(DEO) to basic schools, second cycle schools receive books distributed directly by the Ghana
Education Service (GES) and the Ghana Education Trust Fund (GETFund)
Figure A1 Tracking Text Books and Stationary
Ministry of Finance and
Economic Planning
Statutory Budget
allocation
Budget Allocation
Procurement
Ghana Education
Services
GES Headquarters
GETFund
GES Supplies and
Logistics
Books sent to DEO
and 2nd cycle schools
Books Sent
Books Sent
District Education Office
(DEO)
Books sent
2nd cycle
schools receive
books from the
GOG
Basic Schools
receive books from
Central Government
Tracking Capitation Grants for Basic Schools and Student Subsidies for 2nd Cycle schools
MOESS
Capitation Grants
Budget
Allocation
Budget
Allocation
Ministry of Finance and
Economic Planning
Statutory
allocation
Ghana Education
Services
GES Headquarters
GETFund
Budget Allocation
HIPC Fund
Budget Transfer
Budget transfer
MOESS
District Education Office
(DEO)
Budget Transfer
GES HQ
Capitation Grants Transfer
Student Subsidies Transfer
Basic Schools
receive CG
2nd cycle
schools receive
Student
Subsidies
153
Tracking Investment Expenditure in Basic Education
Ministry of Finance and
Economic Planning
Statutory Budget Transfer
District Assembly
Common Fund
(DACF)
Budget Transfer
District Assemblies
(Fixed % goes to
school investment.
District Education
Office (DEO)
Procurement
Contractors
Services provided
Basic Schools
new
constructions
Source: Ghana PETS 2007
←
 Example 22: Bottlenecks and Risk Areas
It is important to identify especially higher risk decision points and weak nodes in the supply
chain that could affect the quality and availability of services within the sector. The following
figure illustrates such risk areas. It shows a typical resource flow system in a social sector
(health or education) between the central government and service providers through the
various layers of the public administration system.
In general, in each of the various branches of the supply chain system toward frontline
providers, there are possibilities of bottlenecks and leakage: funding and supplies (e.g., drugs,
equipment or materials) could be delayed or be captured. Similarly, salary expenditures could
be delayed or leak through the creation of fictitious (ghost) workers.
However, in the specific sector under study, some delivery channels, programs or areas will be
more at risk of bottlenecks, inefficiencies and capture than others. It is important to identify
these potential higher risk flows and programs and higher risks areas in order to focus the
analysis and data collection strategy on these risk areas and resources.
154
Figure : Resource Flows with Leakage Risk Points
Source: Adapted from Transparency International (2009)
In the specific example presented which illustrates the case of Sub-Saharan African countries,
risk areas were identified as procurement of textbooks as well as (….)
In every survey, the identification of specific risk areas of a system should help in determining
the specific focus of the PETS/QSDS study and condition the design of the survey instrument
and specific data to be collected. Indeed, to design robust data collection instruments adapted
to the particular type of inefficiencies and corruption in place in order to identify leakage and
other inefficiencies, it is important to thoroughly understand potential bottlenecks in the
system and the way resource capture is potentially achieved.
←
 Example 23: Examples of hypothesis
In the 1996 Uganda PETS for instance, the hypothesis for the poor health and education
outcomes observed was that public resources did not reach the schools and health facilities.
155
Survey instruments were developed to compare official budget allocations with actual spending
at various tiers of government, including primary schools and dispensaries. In education, the
focus was on the main non wage transfer, capitation grant.
In Mali, a hypothesis put forward to explain poor education results was that books were not
reaching schools. Data collection focused on tracking books and other school material.
←
 Example 24: Some features and lessons of successful PETS
1996 Uganda PETS in education The initial intentions in Uganda were to track all public
spending in education through the entire delivery system. However, a pilot survey revealed
important data availability problems: It was discovered that at both the central government
and district levels, official records (for both wage and non-wage expenditures) were very poor,
if not simply non-existent. Quality of information at the district level in particular, both on
transfers from the MOE and disbursement to schools, was very poor. Decision was made to
exclude the district level from the tracking exercise and to limit data collection to the central
government and service provider levels
-Furthermore, the pilot survey revealed that at the central government level, data were not
available on salaries paid to primary school teachers, either by districts or schools. The only
systematic information on primary education found to be available and of good quality at the
central level was capitation grants for non-wage spending. Fortunately, financial records were
also relatively comprehensive at the school level. The tracking exercise then ultimately focused
on a single flow: capitation grant. The survey objective thus became: to determine how much
of the capitation grant allocated by the MOE actually reached primary schools.
Data (financial and in-kind) were collected at the school level on the reception of this single
program. To estimate leakage (between the entitled budgetary allocations and actual reception
at the school level), the only other information required was enrolment data at the school level,
which was also available
2002 Zambia PETS/QSDS in education. Another clear success story is the Zambian PETS/QSDS
in education in 2002. Through a very thorough preliminary study of the education sector and indepth analysis of the administrative process, the team was able to grasp the nature and
characteristics of resource flows in the public education system. Six main flows were identified
and categorized in terms of sources, types and administrative levels’ discretionary power in
fund allocation (see Table below).
Funds were further classified into four categories, (i) Rule-based allocation to schools consisting
of a fixed-grant ($600 or $650 depending on the type of school) allocated to schools
independent of enrolment; (ii) Discretionary allocation to schools by provinces or districts; (iii)
Rule-based allocation to teachers: salaries and monthly allowances paid directly to teachers
156
through a centralized payroll; (iv) Discretionary allocation to teachers: Other teachers’
remunerations, such as transfer, leave or funeral benefits, pass through the province or district
office, at their discretion.
Once these flows were identified, the team chose to track non-wage cash flows from the MOE
and donors at the provincial, district and facility levels. In-kind transfers, as well as salary
transfers, were excluded from the tracking exercise. The specific objective of the tracking
exercise then became to determine whether: (a) schools received the fixed-rule component of
the MOH budget (lump-sum payment per school); (b) provinces and districts supported schools
further through discretionary expenditures; and (c) decentralization had an effect on fund
allocation behavior (Das et al, 2004a, p.29).
157
The rationale for excluding staff financial data (salary, allowances etc.) from the tracking
exercise was that staff remuneration is disbursed directly from the central government payroll
to individual teachers and hence does not pass through the hierarchical administrative system.
Still, although it did not track salary flows, the survey analyzed staff compensation in terms of
delays and overdue for various types of teacher compensation. Furthermore, financial data on
staff compensation were collected at the facility level among a sample of teachers (from grades
5 and 6) in order to measure the value of staff input in each school, which was then used in an
equity analysis. (These data were used to calculate the average salary of teachers interviewed
in a school and to construct per pupil staff bills).
The exclusion of in-kind materials and equipment from the tracking exercise was justified by the
fact that the procurement of the school materials was not completed at the time of the survey,
which was fielded in July in the middle of the school term (which started in January) (Das et al,
2004a, p.23). This could have introduced some biases in the data collected if provinces or
district systematically provided more in-kind resources than cash transfers to certain types of
schools or districts. However, as argued by Das et al (2004a, p.50) data on receipts of such
materials at the school level tended to be small at the time of the survey, which reduced the
potential risk of such problem.
Lesson
Having restricted the tracking exercise to cash flows has facilitated data collection in Zambia, as
good quality financial records were available. Furthermore, the distinction introduced between
rule-based and discretionary funding allowed the use of a measure of leakage comparable to
the one developed by Ablo and Reinikka (1998) in the first Uganda PETS. Indeed, in the original
(or “strict”) definition, leakage was defined with respect to a fixed rule. In addition, the
Zambian survey examined the discretionary components of funds transferred to schools using
an equity analysis.
←
 Example 25: Features and lessons of less successful PETS
1996 Uganda PETS in health. Contrary to the education sector where, after a pilot survey, the
focus was specifically restricted to a particular program (capitation grants), in the health sector
the focus remained on all public resources and administrative levels. However, lack of almost
any financial information at the facility level and the heavy reliance on in-kind measures were
not anticipated at design and pilot stage of the survey. Consistent quantitative data could not
be collected. No systematic facility level information on financing, or inputs were found.
Ultimately, survey did not produce any reliable quantitative measures of expenditure flows or
leakage assessment
2002 Mozambique PETS/QSDS in health. Primary health care facilities in Mozambique are not
allocated funding from higher administrative levels, but only receive in-kind transfers:
complicates tracking. The survey could have focused: (i) on specific resource flow or program in
158
order to collect detailed data to permit reliable assessments of leakage; (ii) take a broader focus
and attempt to measure most health expenditures, but incur the risk that the data would not
allow firm conclusions on leakage.
The second option was chosen: tried to track all non-wage recurrent expenditures, drugs and
other supplies, and human resources, at three levels: provinces, districts and facilities.
However, data quality turned out to be a serious concern at both the provincial and district
levels. Large gaps in information were observed in about 75 percent of the districts between
District Health Offices’ financial information records and those provided by the Provincial
Health Offices. Furthermore, complete district level financial data could be collected for only
about 40 percent of districts. Similar discrepancies between provincial and district records were
also found in the case of medication transfers and health worker data.
Mali education PETS/QSDS 2005. The PETS-QSDS in the education sector in Mali in 2005 faced
a relatively similar data inconsistency problem to that encountered in Mozambique. Following
data collection, it was observed that district and schools reported very different resources
levels, and surprisingly schools often reported receiving more resources then reported to be
sent by districts. A post-survey evaluation was realized in a certain number of schools and
districts to identify the source of these inconsistencies. It was found that schools often received
supplies (books and other material) not just from the district level but also from the MOE
(through private suppliers).
Furthermore, schools had difficulty separating supplies between the two sources and tended to
incorrectly register these supplies as simply arriving from the district. Similarly, district records
were inaccurate. Districts sometimes received materials from the MOE which were then sent to
schools, but not properly recorded.
Also, in some cases district records were based not on what had actually been sent, but on the
reports of what schools reported having received. The survey team concluded that the
information system was greatly deficient, and that important reforms needed to be
implemented (Wane et al, 2006).
Lesson
As the Mali and Mozambique experiences reveal, it is important to identify and make sense of
the information actually recorded in official accounts and records. Such identification would
need to be accomplished during a data and institutional assessment phase or pilot phase,
before the launching of the full scale survey.
←
 Example 26: Recommendations of potential tracking choice strategies
If the entire spectrum of expenditures does not lend itself to tracking, either because of the
large number of disaggregated line items and large number of programs or sources of flows,
159
poor recording procedures or data quality, or even survey time or budget constraints, at least
three potential strategies for tracking selection are available:
(i) Pick one or a few specific programs for which fixed allocation rules are in practice (such
as per capita grants). Once the program is identified, all the items associated with the
program have to be tracked on the variables used on the allocation formula and on the
receiving end (on at least two levels including service provider), in order to estimate
leakage. This was the approach chosen successfully by the Uganda 1996 PETS in primary
education, focusing on the capitation (per student) grant program at the central
government and facility levels.
(ii) Track a large budget line, if transfers are essentially all in cash. Good quality financial
data must be available on at least two levels of the administrative structure for the
tracking period. All financial flows during a certain time period have to be tracked. This
is the approach successfully chosen in Zambia in the primary and basic education sector
for non-wage education financial transfers.
(iii) Track a large budget line, such as non-wage recurrent expenditure, which could consist
of cash and in-kind transfers. For financial data, good quality financial data must be
available on at least two levels of the administrative structure for the tracking period. All
financial flows during a certain time period have to be tracked. For in-kind transfers
(such as drugs, medical material, books), instead of tracking all the items, adopt a
sampling strategy. Especially if there is a large number of items received by the facility
or if records are in a disaggregated state, as is typically the case at the facility level, this
approach of sampling a few items instead of targeting wide (census type) coverage is
probably indicated. The strategy entails selection of a few small items with a high
frequency in the flow. While a census would involve tracking all budgets, a sample
choice requires selection of a share of the (in-kind) budget to track, sufficiently large to
be representative but manageable enough to be realistically surveyed. This was the
approach successfully chosen in Chad and Mali. If this approach is followed, information
on the share of these items, for instance medical materials, in the total budget has to be
obtained in order to estimate total leakage in the budget, for instance.
←
 Example 27: Chad: Health PETS: data collection
In Chad, for instance, health facility data sheets using the same structure as the official monthly
activity reports (RMA) sent by facilities to the MOH were utilized to facilitate data collection at
the facility level. Furthermore, the RMA electronic files compiled by the MOH (through district
administrations reporting) were also collected at the MOH level. It was thus possible to verify
(“triangulate”) the validity of information provided at the facility level. In several other
countries data sheets were customized to facilities’ recording practices.
←
160
 Example 28: Niger PETS 2008 Sampling education
In the 2008 education PETS in Niger, for budgetary and logistic considerations, sample design
was chosen to be regionally but not nationally representative. Indeed, a nationally
representative sample would have required to survey sparsely populated remote regions.
Gowever, given that this was the first PETS in Niger, the National Statistical Office (Institut
National de la Statistique, INS) decided to limit geographical coverage. Three regions, Dosso,
Tillabéri and CU Niamey were selected based on their close proximity to the capital. These
account for about 38 percent of total population.
Within each region, districts and service providers were randomly selected to ensure data were
regionally representative. The table below presents the number of facilities surveyed by type.
Although the findings of the PETS are not nationally representative, some lessons could be
applicable to other regions given that expenditure distribution mechanisms are similar across
regions. Still, patterns of inefficiencies could vary between regions due to various factors, such
as differences in remoteness, population density and poverty levels.
Source World Bank 2008, pp. 6-7
←
161
 Example 29: Senegal QSDS 2010 sampling strategy in education and health 23
In the framework of the Service Delivery Indicator Pilot project, QSDS type surveys were put
forward in the education and health sector in Senegal in 2010. The sampling strategy was as
follows.
Education sector
In the education sector, a stratified sample weighted by student population was used with
stratification by regions and urban/rural locations. The country was divided into 10 regions
(Ziguinchor region was excluded due to security issues). A total of 20 strata were constructed by
dividing each region into an urban and a rural stratum. Within each strata health facilities and
schools were selected randomly, with the number of units from each stratum defined by its size
(see below). A lower bound for the number of observations by stratum was, however, imposed.
Sample frame. The target population in the education sector was all children attending grade 4,
therefore all schools with at least one grade 4 class formed the sample frame. The list of
schools as well as the number of students enrolled in grade 4 per school was provided by the
Direction de la Planification et de la Réforme de l’Education (DPRE) database of schools as of
2009.
Sampling strategy. The final sample size was fixed to 151. The number of schools visited within
each stratum was determined by the total enrollment of students in grade 4 in the stratum.
Table 1 shows the number of schools, enrollment and sample size by stratum.
Table 1: Sample Frame and Sample Size for the Education Sector
Number of Schools
Enrollment
Sample Size
Rural
Urban
Region
Rural
Urban
Rural
Urban
41
371
Dakar
15146
204347
2
23
345
59
Diourbel
51134
30958
6
3
644
43
Fatick
109197 16449
12
2
746
87
Kaolack
98742
49023
11
5
945
39
Kolda
147664 24942
17
3
727
49
Louga
61047
22445
7
3
353
19
Matam
50475
9423
6
1
92
Saint-Louis 523
71784
43893
8
5
802
43
Tamba
84341
22375
9
3
601
146
Thiès
120550 95819
14
11
5727
948
Total
810080 519674
92
59
Total
25
9
14
16
20
10
7
13
12
25
151
Finally, schools were randomly selected within each stratum with probability proportional to
enrollment in grade 4, using the DPRE list.
23
Source: Bold et al (2011) pp. 56-58
162
Health sector
Sample frame. In the health sector, the target population for primary health was the whole
population. The sample frame for the health sector was all primary health care providers. There
are two distinct populations of primary care providers in Senegal, the health post (poste de
santé) and the health center (case de santé). It was not possible to obtain from the Ministry of
Health a list of all health posts and centers in Senegal along with the population they serve. The
National Service for Health Information (SNIS) provided for the year 2009 the list of health
posts along with the region, district, and urban/rural location.
Sampling strategy. The final sample size was fixed to 151 with 110 health posts and 41 health
centers. The number of health post to be selected from each stratum was proportional to
population. Population estimates per district were obtained from the latest population
projections for 2010 provided by the national statistics agency (ANSD) using the latest census
(2002) data. Tables 2 & 3 provides information on the sample frame and size for health centers.
Table 2: Number of health posts, population size and sample per stratum
Number of Health Posts
Population Size
Sample
Region
Rural
Urban
Rural
Urban
Rural
9
103
Dakar
49505
2728578
0
11
Diourbel 60
1411802
30513
14
67
2
Fatick
727052
4725
7
34
Kaolack 70
846205
200867
8
56
24
Kolda
585634
103622
5
65
13
Louga
848099
79874
8
65
7
Matam
803211
26416
8
54
38
St-Louis
501147
321364
5
67
22
Tamba
356470
64516
3
80
53
Thiès
734143
910551
7
593
307
Total
6863268
4471026
67
Urban
26
0
0
2
1
1
0
3
1
9
43
Table 3: Final sample health posts and health centers
Sample of Health Centers
Sample of Health Posts
Region
Total
Rural
Urban
Rural
Urban
0
26
26
1
2
Dakar
14
0
14
7
0
Diourbel
7
0
7
4
0
Fatick
8
2
10
5
0
Kaolack
5
1
6
3
0
Kolda
8
1
9
5
0
Louga
8
0
8
2
0
Matam
3
8
3
0
St-Louis 5
3
1
4
2
0
Tamba
7
9
16
6
1
Thiès
67
43
110
38
3
Total
163
Total
3
7
4
5
3
5
2
3
2
7
41
Total
27
14
7
10
6
9
8
8
4
16
110
Health posts were randomly selected within each stratum with probability proportional to
population of the district where the health post is located.
The selection of health centers was done as follows: (1) a sub-sample of 41 health posts was
randomly drawn among the 110 health posts selected, (2) a list of all health centers attached to
the 41 health posts was been established, (3) a random selection of 41 health centers was then
done from this list.
←
 Example 30: Chad: mixed sample-census approach
The alternative strategy of combining a stratified sample in the first stage (to choose districts,
for instance) and a census in the second stage (within districts), allows evaluation of the use of
all resources within districts, and hence measurement of leakage in the case of fixed allocation
rules. It should be noted that this strategy presents the further advantage of allowing obtaining
representative results without the need for weights (Turner et al, 2001, p. 49).
This strategy was used in Chad, where a two-stage sample-census strategy was utilized. First, in
each of the country’s 14 delegations, either one or two districts (depending on the number of
districts in the region) were selected at random. Second, in each of the selected districts, all the
health centers were identified and visited. Also, given the importance of the capital,
N’Djamena, all its health centers were included in the sample. The original health center list
was provided by the MOH Division of Sanitary Information and Statistics. In addition, surveyors
were instructed to identify and visit all health centers not on the initial list in a selected district,
especially the private clinics. The final sample included 281 health centers, of which
approximately two thirds were public, 14 percent private for-profit and 19 percent private nonprofit.
Recommendation: The choice between pure sampling strategies and mixed sampling-census
strategies should be considered. The second strategy could be more practical in the absence of
fixed allocation rules and where “narrow” leakage is measured, for which all resources in an
area have to be assessed.
←
164
 Example 31: Various sampling methods used
TABLE 4 SAMPLE AND RESOURCES MONITORED IN SUB-SAHARAN AFRICA PETS/QSDS
Country
Uganda
Year
Survey
Type
1996
PETS
Sectors
Education
and Health
Sample strategy
Education:
-Two criteria for sample
selection: a) broad regional
coverage
b) Representative of the
school population in the
district.
-Stratified random sample:
For each of the 5 regions of
the country, 2 or 3 districts
were drawn with probability
proportional to the number
of primary schools in the
district. The selection of
schools in the district was
based on school
performance in the primary
leaving examination results
Health: In each district
selected, 5 primarily
government facilities were
visited.
1999
and
2000
PETS
2000
QSDS
Education
Health
N.A.
-Two-stage stratified sample
Three principles: a) focus on
dispensaries; b) all regions
included; c) all categories
(public, private for-profit and
non-profit) should be
surveyed.
Sample
Resources monitored and
years
Levels/Units tracked
Education:
Education:
2 levels:
-18 districts (out of 39)
- Central government (enrolment)
-250 public primary
schools
-Annual capitation (per student)
grant from the central
government (financial and inkind transfers)
Health:
-Panel data 5 years: 1991-1995
-19 districts
-100 health facilities
Health: non-wage expenditure
- Facilities
-16 districts
-218 public primary
schools
-Data for 1999 and 2001
-10 District
administrations (out of 45)
Data for 1999-2000:
-Medical consumable
-Districts
-155 (public, private for
profit and non profit)
health facilities
-Contraceptives
-Health facilities
-non medical consumable
-Patients
-1617 patients
-capital inputs
-First stage: 10 districts
were randomly selected.
- Central government (enrolment)
- Facilities
-Vaccines: 6 months data:
-Drugs (6) : 1 month data
-From the selected districts,
a sample of public and non
profit facilities was randomly
drawn. Private for profit
facilities were identified
based on information given
165
2 levels:
3 units:
Country
Year
Survey
Type
Sectors
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
by public facilities.
2009
Tanzania
1999
2001
2004
Ghana
2000
PETS
PETS
PETS
PETS
PETS
Water
Education
and Health
Education
and Health
Education
Education
and Health
- Water and sanitation funds
allocation from Finance
Planning and Economic
Development to districts
5 levels:
-Central level
-Districts,
-Sub-counties
-Communities water points
facilities
-Water users
-Selection of 8 districts from
each of the 4 regions.
-At least 8 facilities per
district
- From the selected districts,
random selection2 sub –
counties.
-983 water users
N.A.
-3 districts (out of 115)
-45 primary schools
-36 health facilities
-Non-wage education and
health expenditures
2 levels:
-Data for 1998
-Facilities
-Choice of districts based on
geographical balance (ruralurban) and whether or not a
financial management
system (FMS) was in place.
-At the council level, 2
schools and health facilities
chosen from the ward that
houses the council
headquarter and 2 others
from a ward considered
remote by the council
-Three-stage s stratified
sample: First stage:
Selection of 7 regions
according to a stratification
based on the Human
Development.
Second stage: Selection of 3
councils according to their
proximity to the regional
headquarters
Third stage: within each
council 10 schools were
sampled.
-5 districts
-16 primary schools
- Non-wage expenditures
3 levels:
- Data for FY 1999-2000 and
first half 2000
- Districts
-Designed with the aim of
matching data with the
Ghana Living Standard
Survey Round 4 (GLSS4) of
1998.
-10 regions
-40 districts
-119 primary schools
-79 junior secondary
schools
-172 primary health
clinics
-Two-stage stratified
sample: First stage: From
each of the 10 regions, 4
districts were chosen: 2
-Data from FY 2004-2005 to FY
2008-2009
-15 health clinics
- District
-Central government
- Health facilities and schools
-7 regions
- 3 councils
- 210 schools
166
-Capitation grant (cash and
book), development grant and
capacity building grant for
school communities
-Data for 2002-2003
-Total recurrent expenditure
(wage and non-wage)
-Data for FY 1997-1998 and
1998-1999
3 levels:
-Central government
- Councils
- Schools
3 units:
- District offices (health and
education)
- Health facilities and schools
-User perception survey was
carried out
Country
Year
Survey
Type
Sectors
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
depressed, 1 average and 1
better off (based on criteria
of natural resources
endowment, infrastructure
and level of developments).
Second stage: In each
selected district, 3 primary
and 2 junior secondary
schools were selected, as
well as 3 health posts, 1
clinic and 1health center.
Service facilities located in
the EA of the GLSS4 were
automatically part of the
sample as well as facilities
reported used by GLSS4
household.
Rwanda
2000
PETS
Education
and Health
Education: All administrative
units (provincial and district
levels offices) were visited.
400 primary schools were
selected using a two-stage
stratified random sampling
method. 43 and 357 schools
were sampled in urban and
rural areas respectively, with
probability proportional to
the number of schools in the
area. For the urban area, 9
of 43 schools were sampled
from Kigali.
Health: Nation-wide survey
of facilities and
administrative units
2004
PETS
Education
and
Health*
Education: Two-stage: In a
first stage, for each 12
provinces, 3 districts were
selected. Second stage:
primary schools were
chosen in selected districts
as well as 1 secondary
school per district. In
addition, 1 secondary school
per province which was
determined to be the main
beneficiary of the FARG
Education:
- Recurrent expenditures
5 units:
-12 Provincial Education
Offices (out of 12)
(cash, in kind contribution and
equipments)
-Provincial
-151 District Education
Offices (DEOs) (out of
154)
-Data for 1998 and 1999
-Facilities
-Central government
-Districts
-Users
-390 primary schools (out
of 2100)
Health:
-11 Regional Health
Offices (out of 11)
- 37 District Health
Offices (out of 40)
-250 health centers (out
of 351)
-36 districts (out of 106)
- Teachers’ salary
-107 primary schools
- Three funding programs:
-Central government
(out of 2203)
i) Funds for Genocide Survivors
(FARG)
-schools
-banks
-36 Banks
ii) Education support Funds for
Vulnerable and Poor Children
-48 students
iii) Capitation funds.
-48 secondary schools
(out of 339)
-Data for FY 2003
167
4 units:
-beneficiary students
Country
Year
Survey
Type
Sectors
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
program was selected
Sierra
Leone
Zambia
2000
PETS
2001
2001
PETS
QDSD
Agriculture
,
education,
health,
security
sector,
social
welfare,
rural
developme
nt and
local
developme
nt, water
and
sanitation.
Two-stage: In a first stage,
selection of the enumeration
areas for the 8 districts and
12 constituencies in the
Western Areas were
selected.
Education
-Stratified random sample
from urban/rural location.
The sampling frame for
the Enumeration areas
(EAs) was therefore
made of 102 chiefdoms
-Financial and material
resources
-Data for the second half of
2001
-176 health facilities
-477 primary schools
-Schools were chosen from
4 provinces (2 richest and 2
poor with enrolment rates
just marginally better than
the worst performer)
-33 districts
-182 primary schools
(grades 1-9)
- Non-wage funding for basic
education (fixed-school grant,
discretionary non-wage grant
program)
-Data for June 2001-June 2002
6 units:
-Central level
-Provinces
-Districts
-Schools
-Households
-Student achievement
2007
PETS
QDSD
Health
18 hospitals, 90 rural
health centers, 40 urban
health centers.
Various resource flows
(government budget, basket
funds, vertical project funds and
internally generated funds)
FY05-06 financial data
4 units :
-central government
-district
-health facilities
-patients
Mozambique
2002
PETS
QSDS
Health
-Sample was selected in 2
stages: random selection of
districts followed by random
selection of facilities within
districts
-Sampling weights were
used to provide for
nationally representatives
estimates
- Allocation: recurrent budget
panel data 2000 to 2002
5 units:
-35 Districts
-90 Public primary health
centers
- Execution of district recurrent
budget 2000 and 2001
- District directorate of health
-167 workers
- Drugs and other supplies
-679 patients
- District and facility data on
user fees:
-11 Provinces (out of 11)
- Service output:
- Users and staff were also
selected randomly
168
-Provincial directorate of health
- Health facilities
- Staff
- Patient exit polls
Country
Nigeria
Year
Survey
Type
2002
PETS
Sectors
Sample strategy
Health
Sample
Resources monitored and
years
-30 local government
-252 health facilities
-700 staff
QSDS
Levels/Units tracked
3 units:
- Local government
- Facilities
- Staff
Senegal
2002
PETS
Health
-Stratified sampling
methodology was used to be
representative of urban/rural
level.
-10 districts
-37 local governments
-100 facilities
-5 regions were first
randomly selected.
2003
PETS
Health
- Investment program
-Covers all 10 provincial
health delegations and the
two main urban areas,
Douala and Yaoundé.
-36 district (out of 36)
-Stratified multistage.
-2952 patients (in-patients
and out-patients)
-In a first stage, 3
departments (UP) in each
province are selected.
- Decentralization Fund
(recurrent non-wage
expenditures)
- Equipment Fund
-Then 2 departments from
each region, and 10 health
posts and 1 district within
each department were
selected.
Cameroon
- Panel data 6 years: 19972002
-143 health facilities
(including 36 hospitals
and 34 private facilities)
-Non-wage recurrent
expenditures
- Provinces
- Data for 2001-2002 and 20022003 (9 months)
- Health facilities
12 health facilities are
selected in each department
including the district hospital
and 1 private clinic.
-User survey: 6 users are
selected per “zone de
dénombrement” in each
district for the main urban
areas and 3 in others.
2004
PETS
Education
-Covers all 10 provincial
education delegations and
the two main urban areas,
Douala and Yaoundé.
-Stratified multistage.
-In a first stage, 3
departments (UP) in each
province are selected.
720 primary schools and
432 secondary schools
(of which 324 general
secondary schools and
108 technical secondary
schools)
-12,000 households
-User survey: 8 households
are selected per “zone de
169
Various resource flows
- Districts
- Patients
Country
Year
Survey
Type
Sectors
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
dénombrement” in each
district for the 20main urban
areas and 14 in others.
Madagascar
2003
PETS
Education
and health
Education:
Education:
Education:
Education:
First survey: stratified
random sample covering
144 communes in 36
districts (out of 111)
First survey: 36 districts
and 326 primary schools
-Two main funding programs to
schools : CRESED and IPPTE
2 levels:
Second survey: 185
primary schools were
surveyed, about half of
the schools included in
the first survey.
-District allocation data collected
for two years, 2001-02 and
2002-03
Second survey: track budget
at the school level. The
purpose was to provide
nationally representative
figures on budget and
leakages from Cisco to the
school level.
Third survey: budget
allocation at the district level
Health:
First survey: selection of the
public health facility the
most visited in each “zone
de dénombrement”
Second survey: household
survey to track their living
conditions and their
attendance to health
facilities.
Third survey: individual
survey (user perception
survey carried out)
20062007
PETS
Education
and Health
Stratified random sample:
-For each the 6 biggest
regions (out of 22), 2
districts were randomly
selected.
-From selected districts, 3
communes were randomly
selected giving greater
weight to the communes
with more schools.
-Within each commune, 3
public primary schools were
randomly selected.
In the selected communes,
all public health centers of
-Districts school authorities
(Cisco)
-Schools
Third survey: 24 district
administrations (out of
111)
Health:
First survey: 53 FS
(“formations sanitaires”)
in urban areas and 100 in
rural ones (in total, 84
public health centers and
69 private ones (of which
19 faith-based ones)).
Second survey: 12
households in each UPS
in urban areas and 14 in
rural ones
Third survey: 1350
individuals (10 individuals
per FS)
229 schools
113 health centers
Education:
2 units:
Cash, caisse école, school kits,
school equipment, textbooks
and salaries.
-district
Health:
Current expenditures, non
medical consumable, drugs
170
-facility level
Country
Year
Survey
Type
Sectors
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
Type II were visited. If public
health centers of type I were
present in the commune,
one was visited based on
random selection.
Namibia
2003
PETS
Education
and Health
- Convenient sample of
regions (7 out of 13)
- Random sample of schools
and health facilities within
regions
Education:
Education:
Education: 5 units:
113 public and private
primary and secondary
schools
Non-wage recurrent
expenditures
-Regions
School Development Fund
Health:
-Schools
2002-03
9 district hospitals
-School board
Health:
10 health centers
-Students
22 clinics
Non-wage recurrent
expenditures
Health: 5 units:
192 patients
2002-03
-Central government
-Central government
-Regions
-Districts
-Facilities and Patients
Chad
Kenya
2004
2004
PETS/Q
SDS
PETS
Primary
health care
Education
and Health
-Mixed stratified samplecensus strategy
-14 Regional heath
delegations
-Non-wage recurrent
expenditures
7 units:
In the fist step, either 1 or 2
districts depending on the
number of districts in the
region were selected at
random in each of the 14
delegations. In the second
step, all health centers were
identified and visited on a
census basis.
-13 Regional pharmacies
-Data for 2003
-Regional heath delegation
-21 District health
delegations
-8 medical material were traced
-Regional pharamacies
-10 drugs were traced
-District health delegations
-1801 Patients
Facilities
Education:
Education
Stratified sample: In each of
the 8 provinces, 2 poor and
1 rich districts were selected
using poverty index.
Facilities were selected in
each districts using
systematic random sampling
-26 districts and 3
divisions in Nairobi
province
Bursary funds
-Central government
2001-02, 2002-03, 2003-04
-Districts
Health
-Facilities
Non-wage expenditures (funds
and in-kind)
-Users
Users: random exit
interviews of non serious
cases in health centers.
Random selection of 2
students who had received a
bursary in each class. 2 of
-598 parents
-281 Health Facilities
(public, private for-profit
and private non-profit)
-Health Facilities
-Staff
-Patients
-1274 workers
-330 public secondary
schools
- students
Health:
-27 districts and 3
divisions in Nairobi
province
171
-Central government
2003-04
4 units:
Country
Mali
Year
2005
2006
Niger
2008
Survey
Type
Sectors
PETS
QSDS
Education
PETS
QSDS
Health and
education
PETS
Education
and Health
Sample strategy
Sample
Resources monitored and
years
Levels/Units tracked
the nearest parents in each
sampled school were
interviewed.
-214 public health
facilities
A 3-stage sample strategy: selection of 3 regions
(Bamako and 2 others
randomly selected)
-In each academy, two CPC
were randomly when the
academy has four or less,
and three CPC were
selected throughout
academy with at least five.
-Finally, in each CAP, thirtyfive schools have been
investigated.
A one-stage strategy for
urban health facilities..
A two-stage strategy for
rural health facilities:
selection of 7 out of 35
districts and then selection
of a facility within this
district.
Three regions, Dosso,
Tillabéri and CU Niamey
were selected based on
their close proximity to the
capital. Within each region,
districts and service
providers were randomly
selected to ensure data
were regionally
representative.
-3 regions
-15 CAP
-525 schools
-1255 teachers
Selected government
expenditures
4 units:
7 rural community
facilities, 3 urban ones
and 2 benchmark facilities
out of a total of 504
institutions
Selected government
expenditures in health
(mosquito nets, medicines,
medical supplies, salaries in
community health facilities,
operating costs)
Health
-772 patients
-Direction d’Académie
d’Enseignement
-Centre d’Apprentissage
Pédagogique
-Schools
-Teachers
3 units:
-Central government
- Regional
- Facilities
Education:
Education:
Education: 6 units:
3 regional education
offices out of 8, 21 district
education offices out of
23, 299 primary schools
out of 2,736 in the 3
regions surveyed
Selected government
expenditures in education
(textbooks, notebooks and
drawing books for students)
-Book suppliers
-Central government
-Regional
-Districts
-Schools
Health:
-Students
3 regional health offices,
14 district health offices
out of 14, 11 district
hospitals out of 14, 60
clinics out of 233
Health : 5 units:
-Central government
-Private drug suppliers central
and regional
-Regional
-Districts
-Facilities
Sources: Gauthier 2006, Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006). Note : N.A.: Non Available.
←
172
 Example 32: Sample stratification in Mali education PETS 2004 24
In the PETS 2004 in education in Mali, two strata were used in order to specifically focus
on the supply side in the capital. A first strata was constituted of the capital Bamako
and the other strata contains all the other regions.
The steps for drawing the sample were as follow:
For the capital:
(1) Choice of local administrations (CAP). Two or three CAP were randomly selected
for the right and left bank.
(2) Within each local administration (CAP), a choice of 35 schools were drawn.
For the other stratas
(1) Choice of the regions to be part of the sample. Two regions were chosen based
on their characteristics
(2) All the academies of a region are visited. And among each academy, a local
administration (CAP) are drawn. In each region, five CAP are visited
(3) 35 schools are drawn in each CAP
See Wane (2005) for final sample and further details on the sampling strategy.
←
 Example 33: Uganda education PETS 1996: sample strategy
In the 1996 Uganda education PETS, a relatively large sample was used. The sample
selection of schools was governed by two criteria (a) broad regional coverage, and (b)
representativeness of the population of schools in the district (Ablo and Reinikka, 1998).
A stratified random sample was utilized. For each of the five regions of the country, two
or three districts were drawn with probability proportional to the number of primary
schools in the district. In the 18 districts selected (out of 39), the number of schools
visited was based on the total number of schools in the districts. The total sample
comprised 250 public primary schools.
←
 Example 34: Uganda 2000 health PETS: sample strategy
The sample strategy for the QSDS in the health sector in Uganda in 2000 also entailed a
two stage stratified sample. The design was governed by three principles: (a) for
homogeneity purpose, the focus was to be placed only on dispensaries; (b) to capture
24
Source: Wane (2005)
174
regional differences and subject to security constraints, all regions should be included;
(c) to capture ownership categories differences, all categories (public, private for-profit
and non-profit) should be surveyed (Lindelow et al, 2003).
For public and private non-profit dispensaries, the initial sample frame was based on the
1999 MOH facility registry. For private for-profit facilities, given that no census existed,
the strategy retained was to ask sampled public facilities to identify the closest private
for-profit facility. Data were collected at the district administration and health facility
levels. In addition, a patient exit poll was carried out.
←
 Example 35: Mozambique PETS/QSDS: sample strategy
The overall sampling strategy in the 2002 PETS-QSDS in Mozambique was quite
sophisticated. Five units of observation were considered in the survey: provincial and
district administrations, public primary level facilities (health posts and health centers),
staff and patients. A two-stage stratified sample was used for the facility sample
selection. The sample was stratified in terms of urban and rural facilities, in order to
contain sufficient urban facilities to allow adequate analysis. Given that no reliable
sample frame existed, an initial list of primary facilities provided by the MOH was
updated by enumerators during field work (Lindelow et al, 2004, pp. 8-9). A random
selection of districts was first drawn. All 15 urban districts were automatically selected,
along with a sample of 20 rural districts (selected with a probability proportional to the
number of first level facilities in the initial sample frame).
In the second stage, two considerations were taken into account in selecting facilities,
(a) in each district, several facilities needed to be selected in order to assess the extent
to which facility performance could be explained by local management instead of
factors at higher levels; (b) to reduce implementation costs, facilities needed to be
clustered in some areas. In this second stage, three facilities were selected (with equal
probability) in rural districts and two in urban districts. The choice of facilities was made
by enumerators in the field using “preprinted forms with a random numbers series and
clear criteria for listing facilities” to minimize the risk of manipulation (Lindelow et al,
2004, pp. 8, 11).
According to the report, the sample in the rural area is approximately self-weighted,
that is, all rural facilities have the same chance of being selected. Overall, sampling
weights were used to ensure that estimates are nationally representative.
A staff questionnaire was administered. The sample design was as follows. In each
facility, up to three workers were interviewed. The head of the health center was always
interviewed, while the other two workers (or less for facilities with less than three
175
employees) were selected randomly among staff
responsibilities (excluding helpers, etc).
members with
technical
Furthermore, an exit poll of patients was carried out. The selection of patients was done
randomly to reduce selection biases (which could be introduced if the interviews are
restricted to certain hours of the day, if discretion is left to the interviewers, facility staff
or patients themselves). For patients, only those attending “normal” outpatient services
were surveyed and all other types were excluded (maternal and child health
consultations, family planning and similar services). To select patients randomly,
numbered tokens were handed out to all outpatients who were asked to see the
enumerators after the consultation. Patients were then selected randomly with intervals
between the users, which were determined by estimates of the total number of users
expected on a particular day (Lindelow et al, 2004).
The final patient sample presents variations in terms of number of patients interviewed
in each facility because of errors in predicting the number of patients and idiosyncrasies
in ways the sampling process was implemented. Still, Lindelow et al (2004) notes that
the resulting sample of patients is more likely to be representative than ad-hoc sampling
alternatives.
Recommendation: Sophisticated sampling procedures, such as the one adopted for
patients in Mozambique, have to be weighted in terms of their costs and benefits
compared to more simpler approaches in terms of potentially less biased samples, but
also in terms of variability in number of patients by facilities and in differences in
strategy implementation among enumerators.
←
 Example 36: Ghana PETS 2000 Linkage method
Few tracking surveys in Africa have chosen the route of linkages with other surveys. The
Ghana 2000 PETS is an exception. Sample selection was structured to allow linkages
with the Ghana Living Standard Survey (GLSS 4) conducted in 1998, and thus the
matching between households and facilities data sets. The PETS sample covered 143 of
the 300 GLSS 4 enumeration areas. Schools and health facilities located in the
enumeration areas of the household survey were automatically part of the sample,
together with facilities used by GLSS 4 households (Ye and Canagarajah, 2002)
There is a trade-off with such an approach as a choice has to be made for the sample to
be representative of the population of individuals or of the facility population. In the
Ghana case, the final facility sample is probably representative of the household
population and not of the facility population. In such situations, care should be taken to
assign the proper weights to the facility sample to avoid biases. In the Ghana case,
although the report does not provide details on the question, it could be inferred that
176
the weights used were most probably based on individual population instead of
facilities, which in such case would introduce a bias.
←
 Example 37: Chad PETS/QSDS 2004 linkage with household survey
In Chad, efforts were made to ensure linkages between the tracking survey and a
household survey (ECOSIT) that was about to be launched in 2004 at the time of the
design of the PETS-QSDS. A health module was devised by the tracking survey team and
included in the household survey. Furthermore, facility survey codes were used in the
household survey in order to be able to identify the facilities used by households.
However, the linkage is not perfect; given that the household survey is much broader in
scope, it includes enumeration areas (EA) that were not included in the facility survey.
←
 Example 38: Zambia 2004 Matching school and household data
The Zambian PETS/QSDS took a different path to link supply and demand. Das et al
(2004) innovated in devising a household survey which was then linked with their
sample schools’ students in order to assess schooling achievement. The household
survey was conducted in parallel with the school survey in a sub-sample of schools in
two consecutive years in four provinces. The sub-sample of schools was chosen among
schools sufficiently far apart (as measured using a Geographical Information System that
was used to map the location of every school). This technique allowed matching
household and school data. However, their results are limited to the sub-sample of
schools matched with rural households (sample size not provided), given that the
method chosen disqualifies schools in urban areas which are not widely enough
separated. Furthermore, it should be noted that the household stratification chosen was
not based on a preexisting survey. In that respect, the household sample may not be
representative of the country’s population. However, the objective in Zambia was to
develop direct linkages between schools and households. Indeed, by collecting these
household panel data, Das et al (2004a) were able to incorporate household assets and
spending in a detailed equity analysis, which revealed important elements on public and
private spending and learning achievements.
←
 Example 39: Potential experiments and randomized evaluations
Various other experiments involving the participation of citizens could be considered.
Randomized impact evaluations of such interventions could help find answers to
questions, such as: How can awareness about service provision and resource allocations
be raised? Are decision-making mechanisms such as school boards or health committees
having an impact on service providers and outcomes? Similar to the field experiment in
rural India conducted by Banerjee et al. (2006), the role and impact of the village
177
education committee on school functioning and student learning could be examined.
While these authors found that members of these education committees often had little
knowledge of their role and had little impact on children’s learning outcomes,
experiments could be devised to make these committees more effective, apart from
identifying their impacts on schools and learning in sub-Saharan Africa.
←
 Example 40: Experiments to test worker incentives
Duflo et al. (2006) tested whether incentives linked to teacher presence in class could
reduce absenteeism, and whether they promote teaching and student learning.
Monitoring was introduced through the use of a camera operated by a student, who
photographs the teachers as well as other students at the beginning and end of the
school day. Financial incentives were given to teachers in 60 randomly selected schools
out of the 120 schools surveyed. They find that the program positively affected the
absence rate from 43 percent in the control group to 24 percent in the treatment
schools. Furthermore, student test scores were .17 standard deviations higher in the
treatment schools.
Chaudhury et al. (2005) found that community-hired teachers and contract teachers
have significantly higher absence rates than those of regular government teachers.
Similarly, Banerjee et al. (2004) and Banerjee and Duflo (2005) found that communitybased monitoring did not reduce health worker absenteeism in rural India. In contrast,
Chaudhury et al. (2005) found that teachers had a lower absence rate in more
frequently inspected schools.
←
178
APPENDIX A
PETS and QSDS Indicators
Presented below is a list of PETS/QSDS indicators. Section 1 briefly discusses the various
uses of indicators. Section 2 presents potential indicators and discusses their
motivations. Section 3 illustrates these indicators using the case of the Chad 2004 health
sector PETS/QSDS.
1. Uses of indicators
Indicators are synthetic measures representing important components of the service
delivery system and social sector under study. They play an important role in monitoring
and performance measurement. They allow tracking of results and progress, which
would potentially allow the holding of some parties accountable for results (World Bank,
2002).
In order to measure changes, an indicator specifies what is to be measured along a
scale, but in itself does not indicate the desired direction of change (Binnendijk, 2000). A
choice and definition of indicators must then be associated with baseline values and
potential objectives of expected results over time. 25
One can distinguish between input and output indicators as well as between process
and performance indicators. Process indicators help assess whether “human and
procedural interaction… lead to a defined direction” while performance indicators help
assess whether the level of attainment of certain goals has improved relative to their
initial point. Performance indicators measure “deliverables or substantive outcomes”
(Morgan and Qualman, 1996, p.42).
One can also distinguish between qualitative and quantitative indicators, the former
being a subjective description of categories of satisfaction or appreciation of a subject.
In contrast, quantitative indicators provide “objectively and independently verifiable
number or ratios” (Binnendijk, 2000, p.28).
The collection of a standardized set of PETS and QSDS indicators would allow
comparability over time and countries.
25
For evaluation purposes, that is, in order to better understand why and how a program or strategy
works, baseline studies need to be carried out before the implementation or during the initial phase of a
reform or project. Identifying adequate indicators for observing effects and monitoring results is crucial.
179
2. Potential PETS and QSDS Indicators
Table A1 presents a list of 20 potential PETS indicators with their purpose, definition,
and administrative level examined. A list of 30 potential QSDS indicators is presented in
Table A2. The choice of these indicators is dictated by the need to measure key
elements of the service delivery system along the various objectives identified for PETS
and QSDS studies, units of analysis, and categories of data.
The proposed list of PETS indicators comprises four categories: (i) budget/financing, (ii)
personnel, (iii) materials, and (iv) characteristics and environment. The first category
measures various components linked to the line ministry budget and dissipation of
resources. The second category encompasses measures related to personnel inputs,
while the third category concerns materials and equipment. The fourth category
pertains to the characteristics and environment of the social sector under study.
Some of the indicators listed in Table A1 encompass assessment of overall resource
allocation and availability. For instance, the index of per capita resources at different
levels seeks to objectively reflect the level of resources present at each administrative
level and frontline. Leaving aside the question of efficiency in resource usage at
different levels, it indicates the level of resources available for services.
180
Table A1: Potential PETS Indicators by levels (Central, Regional and Service Provider)
Indicator number
and name
Motivation/Purpose
Central level
This indicator illustrates the
relative importance of the line
ministry in the government
priorities and its capacity to
correctly plan and execute its
budget
This indicator measures the
respective shares of capital
investments, total recurrent
expenditures and personnel
expenditures in the ministry’s
budget.
Proportion of the government
budget that is allocated to the
line ministry:
1a) Official share
1b) Executed share
1c) Execution rate
Proportion of the line ministry
budget devoted to:
This indicator reflects the
importance of the line ministry
budget relative to the size of the
economy. It could allow
comparisons with other
countries and relative to
international norms in the
specific sector.
This indicator illustrates
importance and priority given to
that sector in government
budget.
This indicator illustrates the
importance of financial
decentralization in the line
ministry budget, officially and in
practice, in the total budget and
by types of expenditures
Line ministry’s overall budget as
a percentage of GDP
Budget
1. Line Ministry allocation
(proportion of the
budget)
2. Line ministry budget by
functions
3. Line ministry’s budget
as a percentage of the
GDP
4. Rate of growth in line
ministry budget
5. Fiscal Decentralization
2a) Investments;
2b) Recurrent expenditures
2c) Personnel expenditures
Growth rate of line ministry
budget relative to the preceding
year
Proportion of the budget of the
line ministry that is
decentralized (or centrally
executed) officially and
executed:
5a) Officially (total)
181
Decentralized level
(regional and district)
Service provider level
And decentralization by
expenditure types;
5b) Investments
5c) Recurrent
5d) Personnel
6. Discretionary Budget
7. Budget Usage
8. Proportion of
resources reaching
decentralized levels
This indicator illustrates the
importance of discretion in
budgetary allocation at the
various levels
This indicator measures the
internal usage of the budget at
the various levels and
importance of resource
transfers to lower
administrative levels in the
service delivery system
This indicator illustrates from
every dollar allocated in the
budget for a specific line
ministry, how much arrives at
the local level and how much is
used for other administrative
purposes by each level, i.e. the
proportion of the budget used
at the central, provincial, district
and provider levels.
5e)-f) Idem Executed
6a) % of line ministry budget not
governed by fixed allocation
rules
6b) % of regional level budget
not governed by fixed allocation
rules
6c) % of service provider
budget not governed by fixed
allocation rules
Proportion of the line ministry
recurrent budget that is:
Proportion of the line ministry
recurrent budget that is:
Proportion of the line ministry
recurrent budget that is:
7a) Used
7b) Transferred
7c) Received
7d) Used
7e) Transferred
7f) Received at the provider
level
8a) Proportion of resources in
the sector at the regional and
district level (Total value of
resources received divided by
the total budget of the
Ministry).
8d) Proportion of resources in
the sector reaching the
provider level: (Total value of
resources received divided by
the total budget of the
Ministry).
Idem by categories:
8b) recurrent expenditures
8c) non-wage recurrent
expenditures).
Idem by categories:
8e) recurrent expenditures
8f) non-wage recurrent
expenditures).
Note that this indicator does not
try to determine if there is
leakage or not, as these figures
are a somewhat objective
portrait of the usage of
resources in a specific
administrative system compared
to another.
182
9. Resources per capita
This indicator gives an indication
of the level of real public
intervention in the sector. Real
expenditure data as well as
population in each delegations
and district are required
Line Ministry budget per capita
(executed)
9a) Total
9b) Recurrent
9c) Non-wage recurrent
Average per capita allocation
received in regional delegations
(and districts)
9d) Recurrent
9e) Non-wage recurrent
Average per capita allocation
received at the service
provider level
9f) Recurrent
9g) Non-wage recurrent
10a) Central-Regional leakage:
Average leakage at the regional,
and district levels (proportion of
resources not received from
among resources sent by the
central level)
Average leakage at the
provider levels.
10b) Regional-Provider
leakage (proportion of
resource not received from
among resources sent by the
regional level)
10c) Central-Provider leakage:
(proportion of resource not
received from among
resources sent by central
level)
Coefficient of variation of the
allocation of resources in per
capita terms among providers
11c) Recurrent
11d) Non-wage recurrent
(and by regions: urban and
rural)
10. Leakage
This indicator assesses the level
of leakage at the various levels:
between the central and
regional (district) level, between
the regional level and frontline
providers and the overall
leakage in the supply chain from
the central level to the
providers.
11. Equity:
This indicators illustrates the
level of inequity in resource
allocation among regions,
districts and types of providers
12. Delays in budget
disbursement:
This indicator illustrates the
efficiency in budget
management and disbursement
at the various levels of the
service delivery chain
.
Coefficient of variation in the
allocation of resources in per
capita terms by regions (or
districts)
11a) Recurrent
11b) Non-wage recurrent
12a) Number of days between
budget approval and budget
disbursement (release) at the
central level (MoF-Line Ministry)
183
Alternatively, it could be
measured as the ratio of the
most financed district over the
least financed district (in per
capita terms)
12b) Number of days between
budget approval and budget
disbursement (release) between
central and regional (district)
levels
12c) Number of days between
budget disbursement
(release) at the regional level
and reception at the provider
level
12d) Total number of days
between budget approval at
the central level and funds
reception at provider level
Personnel
13. Leakage of salary:
This indicator assesses the
overall level of leakage in
salaries as well as the
proportion of staff affected by
the problem
13a) Proportion of frontline
staff salaries not reaching
destination (difference
between salaries budgetized
and received)
13b) % of staff that does not
receive full salary
14. Ghost workers
This indicator measures
proportion of non-existing
staff
14) % of workers on official
payroll not working in
facilities
14. Delays in salary
This indicator assesses the
delays in salary payments
15. Delays in allowances:
This indicator assesses the
delays in payments of
allowances (weighted average
by proportion of allowances)
14a) % of frontline staff
receiving salary late (more
than 1 month)
14b) Average delays in salary
payment (number of days)
15a) % of frontline staff
receiving other allowances
late (> 1 month)
15b) Average delays in
allowance payments (nbr of
days)
Material
16. Delays in material
17. Leakage of material:
This indicator assesses delays in
arrival of key resources (a
sample of materials should be
used)
This indicator assesses the
importance of materials leakage
in the supply chain (when total
value of materials cannot be
tracked, a sample could be
chosen, e.g. 10 materials)
Delays (in days) in reception
of key materials and supply
(e.g. books, drugs)
Proportion of materials and
supplies not reaching
destination (in value)
184
Environment and
Characteristics
18. Public ownership
This indicator illustrates the
importance of public ownership
in the sector as well as other
categories
19. External support
This indicator assesses the
importance of donor support at
various levels in the sector
20. Number of Donors
active in the sector
Number of donors supporting
the line ministry
Proportion of external funding
in line ministry budget (By
categories, official and actual)
19a) Total contribution (grants
and loans) in total budget
(Official)
19b) Grant contribution in total
budget (Official)
19c) Total contribution in
Recurrent expenditures (Official)
19d) Total contribution in
investment expenditures
(Official)
19e)-h) Idem Actual
185
19i). Proportion of external
funding in regional level
resources (By categories, actual)
18a) % of public facilities
18b) % of private for-profit
facilities
18c) % of private non-profit
facilities
19j). Proportion of external
funding in service provider
resources (By categories,
actual)
PETS
The various purposes of the proposed PETS indicators are as follows.
1. Line ministry’s budgetary allocation. Within the financing category, this indicator illustrates
the relative importance of the line ministry under study in the government priorities and its
capacity to correctly plan and execute its budget. It is composed of three subcategories
accounting for the official proportion of the budget and its executed share in terms of total
budget.
2. Line ministry budget by function. The second indicator measures the respective shares in the
ministry’s budget of capital investments and recurrent expenditures. It is composed of three
subcategories, the proportion of the line ministry budget devoted to investments, recurrent
expenditures and personnel expenditures.
3. Line ministry’s budget as a percentage of the GDP. This indicator reflects the importance of
the line ministry budget relative to the size of the economy. It could allow comparisons with
other countries and relative to international norms in the specific sector. In the health sector,
for instance, the World Health Organization has set a norm of 4 percent of GDP for the health
budget (World Bank’s World Development Report 1993).
4. Rate of growth in line ministry budget. This index illustrates the importance and priority given
to the sector in the government’s budget. The growth rate of the line ministry official budget
corresponds to the base year of data collection and the preceding year.
5. Fiscal decentralization illustrates the importance of financial decentralization in the line
ministry budget overall and by types of expenditures. Given that there could be differences
between official decentralization and what is done in practice, both figures are computed.
There are four subcategories of official decentralization which are measured: the proportion of
the line ministry total budget that is officially decentralized, as well as the officially
decentralized proportion of line ministry investment, recurrent expenses and personnel budget.
The equivalent four subcategories measure decentralization in practice, specifically the actual
control of funds by decentralized administrations.
6. Discretionary budget. This indicator illustrates the existence of explicit rule for allocating
resources and hence the importance of discretion in budgetary allocation at the various levels.
Fixed allocation rules allow more transparency in budgetary decisions, and their absence has
generally been associated with greater problems of leakage of resources.
7. Budget usage. This indicator assesses budget usage at the various levels within the line
ministry administrative system. It measures resource use at the central level and the
importance of resource transfers to lower administrative levels, in particular service providers.
More specifically, it is composed of 6 subcategories: the proportion of the line ministry
recurrent budget that is centrally used and the share transferred to lower administrative levels.
186
The same assessment is done at the regional level (share received, used and transferred to
lower levels), and finally the proportion of the line ministry recurrent budget that is received at
the provider level
8. Proportion of resources reaching decentralized levels. This indicator illustrates, from every
dollar allocated in the budget for a specific line ministry, how much arrives at the local level and
how much is used for other administrative purposes by each level, i.e. the proportion of the
budget used at the central, provincial, district and provider levels. Note that this indicator does
not try to determine if there is leakage or not, as these figures are a somewhat objective
portrait of the usage of resources in a specific administrative system compared to another. It is
composed of 3 subcategories at the decentralized level (proportion of resources in the sector at
the regional and district level in terms of Ministry’s total budget, in terms of recurrent
expenditures and non-wage expenditures. Similar indicators are constructed at the provider
level.
9. Resources per capita. This indicator illustrates the level of real public intervention in the
sector and resources available at various levels in the system. It could also allow direct cross
country comparisons. It is composed of three sub categories at the line Ministry level, total
budgetary resources per capita (executed), recurrent expenditures available per capita and
non-wage recurrent expenditures. Two subcategories cover the regional and provider levels,
recurrent resources and non-wage recurrent resources per capita received.
10. Leakage. This indicator assesses the level of leakage at the various levels in the supply
chain. Leakage is defined as the proportion of resources intended for identified beneficiaries
that does not reach them. The estimation of leakage rates then implies the ability to measure
exactly how much the intended beneficiaries received versus how much they should have
received as given by resources earmarked for them (that is officially planned resources). When
no resources are earmarked for a level or provider, the estimation of leakage makes use of a
broader definition, that of the share not reaching the specific level. Leakage is assessed in three
nodes: between the central and regional (district) level, between the regional level and
frontline providers and the overall leakage in the supply chain from the central level to the
providers.
11. Equity. This indicator illustrates the level of inequity in resource allocation among regions,
districts and types of providers. In several countries, large variability of health and school
spending across regions and districts, as well as within districts, is observed. Equity is proxied by
the coefficient of variation in the allocation of resources in per capita terms by regions (or
districts). The indicator is composed of two sub-categories (total recurrent and non-wage
recurrent expenditures) at the regional level and among providers Alternatively, it could be
measured as the ratio of the most financed over the least financed district (in per capita terms).
12. Delays in budget disbursement. This indicator illustrates the efficiency in budget
management and disbursement at the various levels of the service delivery chain. Delays are
assessed at four nodes: the number of days between budget approval and budget
187
disbursement (release) at the central level between the Ministry of Finance and the line
Ministry; between the line ministry and regional (district) levels, between the regional level and
reception at the provider level, and finally the total number of days between budget approval
at the central level and fund reception at provider level.
13. Leakage of salary. Within the Personnel category, this indicator assesses the overall level of
leakage in salaries as measured by the proportion of frontline staff salaries not reaching
destination (difference between salaries budgeted and received). It is also composed of a
subcategory of the proportion of frontline staff affected by the problem, i.e., percentage of
staff that does not receive full salary during the period.
14. Delays in salary. This indicator assesses the delays in salary payments. This issue could have
important effects on the quality of services, staff morale and the capacity of providers to deliver
services. It is composed of two subcategories, the % of frontline staff receiving salary late (i.e.,
more than one month) and the average delays in salary payment.
15. Delays in allowances. Similar to the previous indicator, this indicator assesses the delays in
payments of allowances (weighted average by proportion of allowances) and is composed of
two categories, the percentage of frontline provider staff receiving other allowances late (> 1
month) and the average delay.
16. Delays in material. This indicator assesses delays in arrival of key resources. Adequate
availability of medical and educational materials is essential for service provision. Delays in
reception of key materials and supplies (e.g., books, drugs) are measured in days. A sample of
materials should be used to construct this indicator.
17. Leakage of material. This indicator assesses the importance of materials leakage in the
supply chain. In cases where the total value of materials cannot be tracked, a sample should be
chosen (e.g., 10 materials). The index is built as the proportion of materials and supplies not
reaching destination (in values).
18. Public ownership. In the category Environment and Characteristics, this indicator illustrates
the importance of public ownership as well as other categories in the sector. Three categories
are constructed: the percentage of public facilities, private for-profit facilities, and not for-profit
facilities.
19. External support. This indicator assesses the importance of donor support at various levels
in the sector under study. It is defined as the proportion of donors’ contribution in line
ministry’s budget. These indicators measure external donors’ budgetary involvement in the
country and priority given to the specific sector under study. It is composed of four
subcategories: donor community contribution to the line ministry budget as measured in the
budget in terms of total contribution (grants and loans) in line ministry budget for the past year,
as well as the proportion of grants, and the total contribution to recurrent and investment
188
expenditures. The equivalent four categories are assessed in terms of total contributions,
including those outside of budget support, as measured by PETS data.
20. Number of donors active in the sector. Finally, an index of the number of donors supporting
the line ministry in constructed. It seeks to proxy the level of fractionalization in donor support
to the social sector under study.
QSDS
Table A2 presents a list of 30 potential QSDS indicators. This list comprises seven categories: (i)
Characteristics and environment, (ii) inputs, (iii) financing, (iv) institutions, incentives and
production process, (v) intermediate output, (vi) final output and (vii) outcomes. The various
purposes of the proposed QSDS indicators are as follows.
1.Ownership. Within the first category of Characteristics and environment, the first indicator
illustrates the ownership structure of the sector. It is composed of three subcategories
accounting for the proportion of public, private for-profit and private not-for-profit ownership
in the sector.
2. Access to electricity. Various indicators are defined to measures service providers’ access to
basic infrastructure. The first basic infrastructure service considered is access to electricity. It is
measured as the percentage of frontline service providers with electricity.
3. Access to water. Another indicator of basic infrastructure services is the access to water
supply. It is measured as the percentage of frontline service providers with drinkable water
4. Access to sanitation. This other indicator of basic service is the access to modern sanitation.
It is measured as the percentage of frontline service providers with sanitation
5. Access to telephone. The final indicator of basic infrastructure illustrates access to
communication. It is measured as the percentage of frontline service providers with telephone.
6. Facilities per capita. This indicator seeks to measure the availability of basic facilities within
the sector in the country. It is measured as the number of primary facilities per capita (health
clinics or primary schools). Three subcategories are constructed, the per capita index at the
national level and broken down between urban and rural areas.
7. Personnel per capita. This indicator seeks to illustrate the availability of specialized personnel
at the frontline level. It is composed of various subcategories reflecting the services provided in
the social sector. In the health sector, the indicator is composed of the number of doctors per
capita, and the number of nurses per capita. In the education sector, it is composed of the
number of qualified teachers per capita.
189
8. Absenteeism. Several recent studies have shown that counting the number of employees
may not provide an accurate measure of health workers’ labor supply in the facility because of
high absenteeism rates (e.g. Chaudhury et al, 2006). We therefore include the rate of
absenteeism in the facility to complement personnel indicators. This indicator seeks to measure
the level of staff absenteeism, which translates into low quality of services. It is measured as
the % of frontline staff which is absent (not motivated).
9. Ghost workers. This indicator measures the proportion of ghost workers, that is, teachers or
health workers who continue to receive a salary but who no longer are in the government
service, or who have been included in the payroll without ever having been in the service. It is
measured as the % of workers on official payroll not working in the frontline facilities.
10. Staff education. This indicator seeks to proxy some elements of the quality of services by
measuring the level of education of service providers. It is measured as the average level of
education of the personnel (teaching or health providing staff). Three subcategories are
constructed, the average education level at the national level and education broken down
between urban and rural areas.
11. Proportion of women. This indicator assesses the proportion of women in the workforce.
Preference for gender is frequently an important element for usage and client satisfaction. In
particular, the presence of the female staff in sufficient numbers is an important ingredient for
maternal health in general and the fight against maternal mortality. It is measured here as the
% of women among the personnel. Three subcategories are constructed, the % of women at
the national level and the same % broken down between urban and rural areas.
12. Proportion of qualified staff. This indicator illustrates the level of qualification among
frontline personnel. It is measured as the % of qualified staff within total frontline staff (health
personnel and education staff). Three subcategories are constructed, the % of qualified staff at
the national level and broken down between urban and rural areas.
13. Equipment access. This indicator measures the access to some of the pieces of main
equipment used for frontline service delivery. Two subcategories are constructed, the % of
pieces of main equipment functioning (by categories: e.g fridges, cars), and the number of
pieces of main equipment per capita.
14. Stock-outs of key materials and supplies. This indicator measures the key materials for
service delivery out of stock during a certain period (last year). It is measured as the stock-outs
by categories of supplies (e.g. drugs, books). Subcategories are constructed for stock-outs by
categories of service providers at the national level and broken down between urban and rural
areas.
15. Compensation. Various indicators reflecting the incentive structure for staff are
constructed. This indicator assesses the incentive structure by measuring the level of
compensation of the staff relative to the country’s level of income. It is measured at the ratio of
190
average staff compensation (salary and allowances) to GDP. Two subcategories are constructed
for qualified staff and unqualified staff.
16. Salary retention. This indicator assesses the level of salary retention at the service provider
level. It is measured as the % of frontline staff not receiving their total salary during the period.
17. Delays in Salary. This indicator assesses the importance of delays in salary payments at the
frontline level. It is measured as the % of staff receiving their salary late (more than 1 month).
18. Supervision. This indicator illustrates the frequency of supervisory activities. It is measured
as the number of supervisory visits per period (year) from the various administrative levels.
19. Supervision frequency. This indicator illustrates the coverage of the supervisory activities by
the different administrative levels. It is measured as the % of service providers supervised
during the period (one year).
20. Public resources’ share in revenues. This indicator assesses the level of support from the
public administration by measuring the public resources available at the provider levels. It is
measured as the proportion of public resources in proportion to service provider total
revenues. Two subcategories are constructed: public resources in proportion to total revenues
and public resources excluding wages.
21. Public resources per capita. This indicator assesses the level of support from the public
administration by measuring public resources per capita. It is measured as the amount of public
resources available at the provider level per capita (in $). Two subcategories are constructed:
total public resources per capita and public resources per capita excluding wages.
22. External support. This indicator assesses the role of external donors in providers’ financial
resources. It is measured as the % of service providers receiving direct external support from
donors.
23. Donors’ contribution to revenues. This indicator measures the importance of donor’s
funding in service provider’s revenues. It is measured as the proportion of donor support in
service provider revenues.
24. User fees as a proportion of revenues. This indicator assesses the importance of user fees in
provider’s revenues. It is measured as the amount of user fees in proportion of service
provider’s total revenues during a period.
25. User fees relative to clients’ income. This indicator illustrates the problem of public access to
basic services. It is constructed as the amount of user fees in proportion of clients’ average
monthly income. This information could be based on exit polls.
191
26. Extra payments for services. This indicator measures the importance of informal payments
in the service providers’ environment. It is measured as the % of patients reporting “extra
payments” to receive services.
27. Reasons for giving extra payments. This indicator identifies the reasons given by clients for
providing informal payments. It is composed of two subcategories, the % of clients reporting
payments for better access (e.g. reduction in wait time) and % reporting payments to receive
better services.
28. Service outputs. This indicator assesses various measures of service provider outputs in the
social sector under study. Various types of output could be measured. In the education sector,
indexes are built for enrolment (average primary level), dropouts (average primary level) and
graduation rates (primary leaving exam). Subcategories are constructed for the national level
and broken down between by gender. In the health sector in particular, there is considerable
heterogeneity in terms of service output due for instance to variations in case mix across
facilities (types of patients, complexity of cases, etc.) and range of services offered. Nonetheless
the most frequently used indicators that measure performance of health centers include the
number of consultations by specialized staff and the total number of consultation per
employee. Here, service output is measured as the number of consultations per employee. It is
further broken down as the number of consultations per qualified employee These indicators
do not control for the quality of services offered, an important factor in explaining efficiency,
but one which is even harder to measure.
29. Service outcomes This indicator seeks to provide an objective measure of outcomes. It is
composed of various measures of population outcomes in the social sector under study. In
primary education, it is measured as the proportion of student achievement based on test
scores
30. Satisfaction. This indicator seeks to provide a subjective measure of service provision
outcomes. It is composed of various measures of client satisfaction in the social sector under
study, such as student or patient satisfaction.
192
Table A2: Potential QSDS Indicators
Indicator number Motivation/Purpose
and name
Characteristics and
Environment of the
sector
1.Ownership
This indicator illustrates
the ownership structure
of the sector
2. Facilities per
capita
This indicator illustrates
the availability and
potential access to basic
facilities
3. Competition
This indicator illustrates
the level of competition
in the sector
Infrastructure
2. Access to
electricity
Service provider level
1a) % of public ownership in the sector
1b) % of private for-profit ownership in the
sector
1c) % of private not for-profit ownership in
the sector
Number of primary health care facilities (per
capita) (or primary schools)
6a) National
6b) Urban
6c) Rural
2a) Number of competitors in a radius of 2
kms
National, urban, rural
2b) Proportion of facilities which are sole
providers in a radius of 2 kms
National, urban, rural
This indicator measures
facilities’ access to basic
infrastructure services
Idem
% of frontline service providers with
electricity
4. Access to
sanitation
Idem
% of frontline service providers with
sanitation
5. Access to
telephone
Inputs
This indicator illustrates
access to communication
% of frontline service providers with phone
7. Personnel per
capita
This indicator illustrates
availability of specialized
personnel
7a) Number of doctors per capita
7b) Number of nurses per capita Or :
7a) Number of qualified teachers per capita
This indicator measures
the level of staff
absenteeism
% of frontline provider staff absent (nonmotivated)
This indicator measures
% of workers on official payroll not working
3. Access to water
Personnel
8. Absenteeism :
9. Ghost workers
% of frontline service providers with
drinkable water
193
10. Staff education
proportion of nonexisting staff
This indicator illustrates
the level of education of
service providers
11. Proportion of
women
This indicator assesses
the proportion of women
in the workforce
12. Proportion of
qualified staff
This indicator illustrates
the level of qualification
among frontline provider
personnel
Equipment
13. Equipment
access
HIV/AIDS test
Material
14 Stock-outs of key
materials and
supplies
Incentives
15. Compensation
16. Salary retention
17. Delays in Salary
18. Supervision
19. Supervision
frequency
in facilities
Staff average level of education (teaching
staff or health providing staff)
10a) National
10b) Urban
10c) Rural
% of women among the personnel
11a) National
11b) Urban
11c) Rural
% of qualified staff to total frontline staff
12a) National
12b) Urban
12c) Rural
This indicator measures
the access to some of the
pieces of main
equipment for service
delivery
This indicator measures
the access to HIV/AIDS
tests
13a). % of equipment found functioning (by
categories: e.g fridges, cars)
13b) # of clients per main piece of
equipment
This indicator measures
the materials out of stock
during a certain period
(last 6 months)
14a) Stock-outs by categories of supplies
(e.g. drugs, books)
14b) Stock-outs by categories of service
providers (national, urban, rural)
This indicator assesses
the level of
compensation of the staff
relative to the country’s
level of income
This indicator assesses
the level of salary
retention
This indicator assesses
the importance of delays
in salary payments
This indicator illustrates
the frequency of
supervisory activities
This indicator illustrates
the coverage of the
Ratio of average staff compensation (salary
and allowances) to GDP
15a) qualified staff
15b) unqualified staff
% of centers offering HIV/AIDS test
% of frontline staff not receiving total salary
% of staff receiving salary late (more than 1
month)
Number of supervisory visits per period
(year)
% of providers supervised (year)
194
20. Public resources’
contribution to
revenues
21. Public resources
per capita
External Support
22. External support
23. Donor
conribution to
provider revenues
supervisory activities by
the different
administrative levels
This indicator illustrates
the importance of public
resource support
This indicator illustrates
the importance of public
resource support at the
population level
This indicator assesses
the role of external
donors in providers’
funding
This indicator measures
the importance of
donors’ funding in
providers’ revenues
User fees
24. User fees’
proportion of
revenues
This indicator assesses
the importance of user
fees in provider’s
revenues
25. User fees’
This indicator illustrates
proportion of clients’ the problem of access to
basic services
income
Informal system
26. Extra payments
This indicator measures
the importance of
for services
informal payments
27. Reasons for
This indicator identifies
the reasons for informal
giving extra
payments
payments
Output
28. Service outputs
This indicator assesses
various measures of
service provider outputs:
Outcomes
29. Service
outcomes
This indicator seeks to
provide some objective
Public resources as a proportion of service
provider total revenues
a) total revenues
b) Revenues excluding wages
Public resources per capita ($)
a) total resources
b) excluding salaries
% of service providers receiving direct
external support from donors
Amount of donor support for service
provider revenues
User fees as a proportion of service provider
total revenues
User fees in proportion of clients’ average
monthly income (based on exit polls)
% of patients reporting “extra payments” to
receive services
22a) Reduction in wait time (access)
22b) Better services
25a) Enrolment (average, primary) (total,
boys, girls)
25b) Dropouts (average, primary) (total,
boys, girls)
25c) Graduation rates (primary leaving exam)
(total, boys, girls) Or:
25a) Number of consultations per employee
(or per qualified employee)
Measure of population outcomes
195
measure of outcomes
30. Satisfaction
This indicator seeks to
provide some subjective
measure of outcomes
-Student achievement (based on test scores)
Student or patient satisfaction
196
3. Illustration of the Proposed Indicators for Chad
To illustrate the potential usefulness of the proposed PETS/QSDS indicators, provided below is
an illustration for the case of Chad using the data collected in the 2004 PETS/QSDS in the health
sector. 26 Tables A3 and A4 illustrate the proposed indicators using the PETS/QSDS data in Chad.
For each of the 20 PETS indicators and 30 QSDS indicators, the tables present the values, when
possible, and contain comments about the calculation, when useful.
Table A3: Illustration of the proposed PETS indicators in the case of Chad
I. Budget
Indicator
1. Line Ministry
budgetary allocation
(proportion of the
budget)
2. Line ministry budget
by functions
Definition
Score
Proportion of the
government budget that is
allocated to the line
ministry:
1a) Proportion of planned
budget
1b) Proportion of
executed budget
1c) Line ministry budget
execution rate
Share of the line ministry
budget devoted to:
2a) Investments;
2b) Recurrent
expenditures
2c) Personnel
expenditures
8.4%
8.4%
100%
Brief explanation
This indicator illustrates the relative importance
of the line ministry in the government priorities
and its capacity to correctly plan and execute its
budget. In Chad, the Ministry of Health (MoH)
budget in 2003 was 33 billion CFA Francs (US$57
million). This represents 8.4% of the total
government budget. The planned health budget
corresponds largely to the executed budget.
Line ministry official budget by functions
for the past year (corresponding to data
collection)
57%
43%
16%
3. Line ministry’s budget
as a percentage of the
GDP
Line ministry’s overall
budget as a percentage of
GDP
4. Rate of growth in line
ministry budget
Growth rate of line
ministry budget relative to
26
2.1%
23.6%
This indicator reflects the importance of the line
ministry budget relative to the size of the
economy. The World Health Organization has
set a norm of 4% of GDP for the health budget.
In Chad, the health budget in 2003 of 33 billion
CFAF (US$57 million) represents 2.1% of GDP.
This indicator illustrates the importance of the
sector in the government budget priorities. In
See Gauthier and Wane (2005, 2008) for details of the combined PETS/QSDS survey in Chad.
197
the preceding year
5. Fiscal Decentralization
6. Discretionary budget
7. Budget usage
Proportion of the budget
of the line ministry that is:
decentralized (or centrally
executed) officially and
executed:
5a) Officially (total)
24%
and by expenditure types;
5b) Investments
5c) Recurrent
5d) Personnel
0%
60%
48.4%
5e)-h) Idem Executed
8.5%
7a) % of line ministry
budget not governed by
fixed allocation rules
7b) % of regional
administration budget not
governed by fixed
allocation rules
7c) % of service provider
budget not governed by
fixed allocation rules
8a) Proportion of the line
ministry recurrent budget
that is:
7a) Used
7b) Transferred
Proportion of the line
ministry recurrent budget
at the regional level that
is:
7c) Received
7d) Used
7e) Transferred
Proportion of the line
ministry recurrent budget
198
0%
14.3%
48.4%
80.75 %
68.1%
100%
40.1 %
59.9%
30%
10.4%
19.6%
19%
Chad, the health budget was 27 026 million
CFAF in 2002 and 33 408 million CFAF in 2003, a
23.6% increase over the preceding year.
These indicators illustrate the importance of
financial decentralization in the line ministry
budget, officially and in practice. In Chad, the
proportion of the total health budget officially
allocated to the 14 regional health delegations
(RHD) was 8 billion CFAF, representing 24% of
the budget in 2003, mainly in the form of
recurrent expenditures, corresponding to about
60% of the MoH recurrent budget. However,
only 14% of the regional health budget is
managed at the regional level. Hence, in
practice only 8.5% of the total MoH budget is
decentralized, representing 14.3% of the
recurrent budget. As for personnel
expenditures, health personnel are paid directly
by the MoH but their salaries are posted in the
MoH regional budget representing 48.4% of
MoH wage expenditures.
This indicator illustrates the existence of explicit
allocation rule for resources. In Chad, the only
two items for which a form of fixed rule is in
place are personnel expenditures and
allocations to regional health administration,
which are established in the MoH budget.
Accounting for these two fixed rules, the
discretionary expenditures at the central level
thus represent 80.75%. At the regional level,
only personnel expenditures could be seen as a
fixed rule in the regional administration budget
(31.9% of the official regional budget) the
discretionary budget represents 68.1%. At the
service provider level, all expenditures are
discretionary.
This indicator assesses budget usage at the
various levels within the line ministry
administrative system. In Chad, the proportion
of total recurrent expenditures allocated to the
regional level represented 59.9% of recurrent
expenditures (wage and non-wage) in 2003, and
the share used at the central level 40.1%.
However, the share received at the regional
level represents only 30% of recurrent
expenditures and 19.6% at the local level. The
equivalent figures for non-wage expenditures
are as follows: 32.6%; 67.4%; 18%; 17.1%; 0.9%;
0.9%
8. Proportion of
resources reaching
decentralized levels
9. Resources per capita
10. Leakage
that is:
7f) Received at the
provider level
11a) Proportion of total
public resources at the
regional and district levels
11b)
Proportion
of
recurrent expenditures at
the regional and district
levels
11c) Proportion of nonwage
recurrent
expenditures
at
the
regional and district levels
11d) Proportion of total
public resources reaching
the provider level:
11e)
Proportion
of
recurrent expenditures at
the provider level
11f) Proportion of nonwage
recurrent
expenditures
at
the
provider level
Line Ministry budget per
capita (executed)
9a) Total
9b) Recurrent
9c) Non-wage recurrent
Average
per
capita
allocation received in
regional delegations (and
districts)
9d) Recurrent
9e) Non-wage recurrent
Average
per
capita
allocation received at the
service provider level
9f) Recurrent
9g) Non-wage recurrent
10a) Central-Regional
leakage: (proportion of
resources not received
relative to resources sent
by the central level)
10b) Regional-Provider
leakage (proportion of
resource not received
relative to resources sent
199
12%
30%
10.9%
7.9%
19.6%
0.9%
7.6 $
3.05 $
1.84 $
This indicator illustrates, from every dollar
allocated in the budget for a specific line
ministry, how much arrives at the local level and
how much is used for other administrative
purposes by each level, i.e. the proportion of
the budget used at the central, provincial,
district and provider levels. To estimate the
value of resources reaching health centers in
Chad, the survey collected information on the
financial resources received by health centers,
as well as medical materials, drugs and salaries.
The survey traced eight medical materials
received by health centers as well as 11 drugs
and medical consumables. The total value of
resources reaching the health centers was
calculated as the sum of the financial resources,
medical materials, drugs and salaries received
by health centers from the health
administration accounting for the proportion of
these items.
This indicator measures the level of real public
intervention in the sector and of resources
available at various levels in the supply system.
It could also allow direct cross-country
comparisons. In Chad, we make use of the
budget information reported above to construct
these indicators of available resources per
capita at the different levels.
0.91 $
0.33 $
0.59 $
0.02 $
73.3 %
95%
This indicator assesses the level of leakage at
the various levels. Leakage is generally defined
as the proportion of resources intended for
identified beneficiaries that does not reach
them. In the absence of fixed allocation rules as
in Chad, leakage at the facility level is estimated
as the proportion of the total health budget
earmarked for the regions that does not reach
the primary care providers given that most of
the public resources for health should end up in
11. Equity
by the regional level)
10c) Central-Provider
leakage: (proportion of
resources not received
relative to resources sent
by central level)
Coefficient of variation in
the allocation of resources
in per capita terms by
regions (or districts)
11a) Total Recurrent
11b) Non-wage recurrent
Idem among providers
11c) Total Recurrent
11d) Non-wage recurrent
12. Delays in budget
disbursement
13. Leakage of salary
14. Delays in salary
Ratio of the most financed
district over the least
financed district (in per
capita terms)
12a) Number of days
between budget approval
and budget disbursement
at the central level (MoFLine Ministry)
12b) Idem between
central and regional
(district) levels
12c) Idem between
regional level and provider
levels
12d) Total number of days
between budget approval
at the central level and
fund reception at provider
level
13a) Proportion of
frontline staff salaries not
reaching destination
(difference between
salaries budgeted and
received)
13b)% of staff that does
not receive full salary
14a)% of frontline staff
receiving salary late (more
than 1 month)
200
98.7%
0.30
0.25
0.59
0.36
the facility, i.e. the service delivery point.
Estimations presented are for non-wage
recurrent expenditures. The corresponding
figures for total recurrent expenditures are as
follows: 49.95; 35% and 67.2%
This indicator illustrates the level of inequity in
resource allocation among regions, districts and
types of providers. Based on the estimates of
recurrent resources reaching service providers,
it is possible to evaluate the coefficient of
variations (i.e. standard-deviation/mean) of the
distribution in resource allocation across
regions and across providers in per capita
terms. In Chad, public resources available in the
health sector vary considerably among regions
as well as between urban and rural areas.
16
---
This indicator illustrates the efficiency
of budget management and
disbursement at the various levels of
the service delivery chain. In Chad,
these data were not collected.
6.6%
This indicator assesses the overall level
of leakage in salaries as well as the
proportion of staff affected by the
problem. In Chad, information was
collected among staff on reception of
salaries but tracking of salaries was not
performed.
13.7%
This indicator assesses the delays in
payments of salaries.
---
15. Delays in allowances
16. Delays in materials
17. Leakage of material
14b) Average delays in
salary payment (number
of days)
5a)% of frontline provider
staff receiving other
allowances late (> 1
month)
15b) Average delays in
allowance payments
(number of days)
Delays (in days) in
reception of key materials
and supplies (e.g. books,
drugs)
17b) Leakage between
regional level and
providers
---
96%
17c) Leakage between
central level and providers
99.3%
18a)% of public facilities
68%
Proportion of external
funding in line ministry
budget (By categories,
official and actual)
19a) Total contribution
(grants and loans) to total
201
This indicator assesses the delays in
payments of allowances (weighted
average by proportion of allowances).
This information was not collected in
Chad.
---
82.2%
18b)% of private for-profit
facilities
18c)% of private nonprofit facilities
19. External support
---
Proportion of materials
and supplies not reaching
destination (in values)
17a) Leakage between
central level and regions
(in proportion of line
ministry materials budget)
18. Public ownership
---
14%
18%
51 %
This indicator assesses delays in arrival
of key resources (a sample of materials
should be used). In Chad, information
on arrival dates of materials was not
collected.
To estimate the value of resources
reaching health centers, the survey
traced eight medical materials
received by health centers. The total
value of materials received by health
centers was estimated at 1.75 million
CFAF. Given that these eight materials
make up 7.4% of the value of
centralized credit deliveries, it was
estimated that health centers received
approximately 23 million CFAF in
centralized credits or 50 million CFAF
for the entire country given that the
survey covers half the health centers’
population. This represents only 17.8%
of the total value of materials sent to
delegations. Materials leakage
between regional delegations and
local service providers is then 82.2%.
Overall, materials leakage between the
central administration and local
providers is estimated as a proportion
of the goods and materials budget of
the MoH
This indicator illustrates the
importance of public ownership in the
sector as well as other categories. In
Chad, the survey collected information
on about half of the primary health
facilities among public, private forprofit and not-for-profit facilities.
This indicator measures the
contribution of external donors to the
line ministry as well as various
contributions to the sector. Donors’
official contribution to the line
ministry budget, the 2003 rate is 51%,
of which grants represent 27.6% of the
budget (Official)
19b) Grant contribution to
total budget (Official)
19c) Total contribution to
recurrent expenditures
(Official)
19d) Total contribution to
investment expenditures
(Official)
19e)-h) Idem Actual
20. Number of Donors
active in the sector
19i). Proportion of
external funding in
regional level resources
(By categories, actual)
19j). Proportion of
external funding in service
provider resources (By
categories, actual)
Number of donors
supporting the line
ministry
Note: --- : Data non available in the Chad PETS/QSDS survey
202
27.5 %
0%
89.5%
54 %
30.6%
8%
89.5%
budget. Most external contributions
are intended for investment
expenditures representing 89.7% of
total investments. In addition to
donors’ contributions to the central
ministry, direct donor support to the
sector is measured using PETS data.
These represent about 1 billion CFAF
(US$200 000) in 2003. These direct
contributions are relatively small
compared to the total health budget,
but represent about 18% of the
regions’ official budget for goods and
services.
--6.2 %
25
This indicator measures the degree of
foreign participation. In Chad, at least
25 external donors or NGOs were
active in the health sector in 2003.
Table A4: Illustration of the Proposed QSDS indicators for Chad
Indicator
Characteristics
1.Ownership
Infrastructure
2. Access to electricity
3. Access to water
Definition
Score
Motivation/Purpose
1a)% of public ownership in
the sector
1b)% of private for-profit
ownership in the sector
1c)% of private non-profit
ownership in the sector
68%
% of frontline service providers
with electricity
% of frontline service providers
with drinkable water
33.8%
This indicator measures access to
basic infrastructure services
65.1%
Idem
This indicator illustrates the
ownership structure of the sector
14%
18%
4. Access to
sanitation
% of frontline service providers
with sanitation
60.1%
Idem
5. Access to
telephone
Inputs
6. Facilities per capita
% of frontline service providers
with phone
13.9%
This indicator illustrates access to
communication
7. Personnel per
capita
Personnel
8. Absenteeism
9. Ghost workers
10. Staff education
11. Proportion of
Number of primary health care
facilities (per 1000 inhabitants)
6a) National
6b) Urban
6c) Rural
7a) Number of doctors (per
1000 inhabitants)
7b) Number of nurses (per
1000 inhabitants)
Or :
7a) Number of qualified
teachers (per 1000
inhabitants)
This indicator illustrates the
availability and potential access to
basic facilities
0.03
0.05
0.02
0.03
This indicator illustrates availability
of specialized personnel
0.04
% of frontline provider staff
absent (non-motivated)
% of workers on official payroll
not working in facilities
Staff average level of
education (teaching or health
providing staff, in years)
10a) National
10b) Urban
10c) Rural
% of women among the
203
21%
----
This indicator measures the level of
staff absenteeism
This indicator measures proportion
of non-existing staff
This indicator illustrates the level of
education of service providers (in
years)
8.6
9.1
8.2
This indicator assesses the
women
12. Proportion of
qualified staff
Equipment
13. Equipment access
Material
14 Stock-outs of key
materials and
supplies
Incentives
15. Compensation
16. Salary retention
17. Delays in salary
personnel
11a) National
11b) Urban
11c) Rural
% of qualified staff to total
frontline staff
12a) National
12b) Urban
12c) Rural
25.5%
38.6%
19.3%
41.8%
49.4%
38.7%
13a).% of providers with
functioning equipment (by
categories: e.g fridges, cars)
13b) # of clients per pieces of
main equipment
14a) Stock-outs by categories
of supplies (e.g. drugs, books)
14b) Stock-outs by categories
of service providers (national,
urban, rural)
Ratio of average staff
compensation (salary and
allowances) to GDP per capita
15a) total
15b) qualified staff
15c) unqualified staff
% of frontline staff not
receiving total salary
% of staff receiving salary late
(more than 1 month)
77.6% (fridge)
38% (microscope)
---
15.4% (9 drugs)
19.2% (5
vaccines)
42%
54%
18.6%
6.6%
This indicator assesses the level of
salary retention
This indicator illustrates the
frequency of supervisory activities
19. Supervision
frequency
20. Public resources’
share in revenues
% of providers supervised
(year)
Public resouces as a
proportion of service provider
total revenues
a) total revenues
b) excluding wages
Public resources per capita ($)
a) total resources
b) excluding salaries
76%
24.1
2.3%
0.66
0.02
18.9%
204
This indicator measures the
materials out of stock during a
certain period (last year)
This indicator assesses the
importance of delays in salary
payments
2.3
% of service providers
receiving direct external
This indicator measures the access
to some of the pieces of main
equipment for service delivery
13.7%
Number of supervisory visits
per period (year)
22. External support
This indicator illustrates the level of
qualification among frontline
provider personnel
This indicator assesses the level of
compensation of the staff relative to
the country’s level of income
18. Supervision
21. Public resources
per capita
proportion of women in the
workforce
This indicator illustrates the
importance of public resource
support
This indicator illustrates the
importance of public resource
support at the population level
This indicator assesses the role of
external donors in providers’
funding
23. Donor proportion
of revenues
User fees
24. User fees
proportion of
revenues
support from donors
Proportion of donor support in
service provider revenues
User fees as a proportion of
service provider total revenues
a) total revenues
b) excluding wages
8%
This indicator measures the
importance of donor funding in
provider revenues
This indicator assesses the
importance of user fees in provider
revenues
69.7%
89.7%
25. User fees relative
to clients’ income
Informal system
26. Extra payments
for services
27. Reasons for giving
extra payments
User fees relative to clients’
average monthly income
(based on exit polls)
% of patients reporting “extra
payments” to receive services
22a) Reduction in wait time
(access)
5.6%
4%
This indicator illustrates the problem
of access to basic services
This indicator measures the
importance of informal payments
---
This indicator identifies the reasons
for informal payments
---
This indicator assesses various
measures of service provider
outputs:
22b) Better services
Output
28. Service outputs
Outcomes
29. Service outcomes
30. Satisfaction
25a) Enrolment (average,
primary) (total, boys, girls)
25b) Dropouts (average,
primary) (total, boys, girls)
25c) Graduation rates (primary
leaving exam) (total, boys,
girls) Or:
25a) Number of consultations
per employee (or per qualified
employee)
Measure of population
outcomes
-Student achievement (based
on test scores)
Student or patient satisfaction
Note: --- : Data non available in the Chad PETS/QSDS survey
205
85.4 cons/empl
(297 cons/empl
qualified)
---
This indicator seeks to provide an
objective measure of outcomes
---
This indicator seeks to provide a
subjective measure of outcomes
APPENDIX B: GOVERNANCE AND SERVICE DELIVERY 27
1. Introduction
Inadequate service delivery is reflected in the poor showing of social outcomes in many
developing regions as improvements in education and health outcomes remain very slow. In
particular, many Sub-Saharan African countries will not meet the 2015 Millennium
Development Goals (MDG). For instance, with an average under-five mortality rate at 158 per
1000, the region accounts for more than half of the 10 million children around the world who
die each year before their fifth birthday. Furthermore, almost 1/3 of the primary school-age
population in the region does not attend school. Mathematics and reading scores for those
attending school are also low. Close to 40 percent of the adult population remains illiterate, of
which 62 percent are women. Also, about 44 percent of the population currently does not have
access to safe drinking water, and about 37 percent has no access to sanitation facilities 28
To accelerate progress, governments, donors and NGOs have committed increasing financial
resources in recent years to improve basic service delivery.29 However, in countries with weak
institutions, it has been shown that budget allocations barely explain the quality and quantity of
services (Filmer et al, 1999; World Bank, 2003). Two major explanations for this situation are
generally put forth. First, public funds committed officially may not necessarily reach frontline
providers for reasons related to capture, corruption and leakage (Reinikka and Svensson, 2004).
Second, even if public resources reach frontline providers their effective use to produce
services of acceptable quality is not guaranteed (World Bank, 2003).
This chapter reviews institutional and governance deficiencies that impact service delivery
performance and explores avenues of solutions brought about by PETS/QSDS.
This chapter is inspired by Gauthier and Reinikka (2007) and Gauthier and Wane (2011)
Also, in the MENA region, there remain important gaps in social outcomes as well as equity problems across
income groups and between rural and urban areas. In particular, health outcomes are also a concern in several
countries. In Djibouti, infant mortality remains at 84 per 1000 live births, much higher than the regional average of
32. Life expectancy is only 56 years. Maternal mortality in the country reaches 650 per 100,000 live births and is
also high in Yemen, Iraq, Morocco, and Algeria with rates of 430, 300, 240 and 180, respectively. Youth and adult
illiteracy also remains high in the region. On average, 28 percent of the adult population (15 years and older) in the
region is illiterate; the problems is most pronounced among women. In Yemen and Djibouti, low income countries,
close to half of the population is illiterate; including three quarters of women (UNESCO, 2009). Furthermore,
access to water and sanitation services, an important vector of sickness, is still limited in many countries. In
Morocco, and Tunisia, for example, about 20 percent of the population does not have access to clean water and
one in six does not have access to improved sanitation.
29
Low-income countries saw funding from external sources rise on average from 16.5 percent of their total health
expenditures in 2000 to 24.8 percent in 2007. Government commitments for health reported by bilateral donors
jumped from about US$4 billion in 1995 to US$17 billion in 2007 and US$20 billion in 2008. (WHO 2011, p.31
World Health Report)
27
28
206
2. Institutional Arrangements of Service Delivery
In developing countries, services such as education, health, water and sanitation are provided
by a mixture of public, private for-profit and non-for-profit (non-government organizations
(NGOs) and faith-based) providers. The relative importance of these various ownership types
varies according to sectors and countries. In the last two decades, domestic and international
NGOs have grown to occupy a greater role especially in areas neglected by public providers and
not served by private providers. The importance of the private sector varies considerably
across countries. Its role is especially prevalent in the health sector in urban areas. However,
the public sector is still the major player in service delivery in most countries. 30 In addition to
formal sector providers, many informal providers also offer traditional services, especially in the
health and water sectors.
3. How do institutional arrangements affect service delivery?
The institutional arrangements within which services are provided give rise to various
governance and accountability problems and condition the quality and efficiency of service
delivery provision.
Service delivery outcomes are the result of a complex web of principal-agent relationships
involving citizens, providers and governments. The structural relationship between the various
groups involved in service delivery and the difficulty of control within the system, makes good
governance in service delivery a challenging task.
Citizens (clients) delegate responsibilities to elected officials (state) to provide public services
and pay taxes to fund them. Politicians in turn delegate service delivery to provider
organizations by creating incentives and appropriating budgets. This network of principal-agent
relationships presents numerous incentive problems. There are two main layers of agency
problems: (i) between the citizens and elected officials, and; (ii) between elected officials and
service providers (See Figure A1). The role of intermediary agent played by the state in the
principal-agent relationship creates a situation where it is difficult for the principal (citizens) to
evaluate and control the actions of the decentralized agent (service provider) (Besley and
Ghatak, 2003; World Bank, 2003).
30
For instance in Chad, the formal private for-profit sector employs less than 1 percent of the health personnel and
the vast majority of formal health sector personnel are active in the public and nonprofit sectors (World Bank,
2004).
207
Figure A1: Clients-Government-Service Providers Relationship
State/Policymakers
Clients/Citizens
Service Providers
Source: World Bank (2003).
In an ideal world, perfect information is shared among parties and there is a good feedback
mechanism between principal and agent that would ensure accountability. However, in
practice, citizens have a weak capacity to exercise control through this process because
information is imperfect and asymmetric, agents’ objectives do not coincide with those of the
principal (citizens), and enforcement is inadequate. Furthermore, the state has a weak capacity
to supervise service providers. This leads to situations where agents put forth less effort than
citizens would wish (shirking) or divert some resources to their own ends (rent extraction).
The difficulty of control that exists in any principal-agent relationship is reinforced in public
service delivery due especially to three aspects in which public services differ from private
provision: the difficulty of measuring performance, multi-tasking considerations, and the
presence of multiple principals (Dixit, 2002; Besley and Ghatak, 2003; Burgess and Rotto, 2003).
Measurability and attributability: Measurability problems are associated with the complexity
of service provision. Agents' activities (marginal productivities) in public organizations are
generally unobservable by the principal (citizens). Typically, citizens only get to observe the
aggregate output of the production process. Citizens cannot easily determine who is
responsible for the situation they observe: the frontline service providers, the bureaucrats, or
the politicians. They cannot observe the specific contribution that a politician makes to a
program and it is also difficult for them to link this potential contribution to the program
outcome and their own welfare. Further, the objectives of service providers are often
imprecise. For instance, the overall objective of a school is to provide good education, which is
very difficult to define. In such contexts, it is hard to identify good performance measures.
Multitasking: Service providers perform a multiplicity of tasks which make evaluation of results
even more difficult. For instance, health workers perform vaccination or other preventive
208
activities as well as curative activities, which generally compete with each other in terms of
limited time and other resources. If, for instance, health workers are rewarded based on the
performance of vaccinations exclusively, this might lead to an excessive focus on these activities
at the expense of other components of good health. 31
Multi-principals: Service delivery is also characterized by the presence of multiple principals.
There are several actors who are directly affected by the actions of an agent in the provision of
public services. For instance, in the education sector, the parents, employers, school boards,
ministry of education officials and politicians could be seen as the principals, while the agents
are the teachers. These different principals might have different preferences concerning the
outcome of the various tasks carried out by the agents (Besley and Ghatak, 2003). Because each
principal would like to induce the agent to put more effort into activities that he cares more
about, if the incentive schemes are not chosen to maximize the joint payoffs of the principals,
there will be inefficiencies (Dixit, 2002).
Functional interdependence: In addition to these three factors, the difficulty of controlling
public service delivery is compounded by the presence of multiple agents engaged in a joint
production process. While delivery of quality health care and education is contingent
predominantly on what happens in consultation rooms and in classrooms, a combination of
various basic elements must be present for quality services to be accessible and produced by
health personnel and teachers at the frontline, which all depend on the overall service delivery
system and supply chain. Adequate financing, infrastructure, human resources, material and
equipment have to be available for service delivery at the frontline, as well as proper
institutions and governance structures to provide adequate incentives to the service providers.
The availability of these essential elements and institutions are a function of the efficiency of
the entire service delivery system.
Moral hazard, which arises in situations where individuals are shielded from bearing all the
costs of their actions because of the difficulty of observing their private actions, is then
reinforced in teams, especially in large public organizations. There is a crucial lack of
information at various levels in the public organizational structure, particularly at the central
level in most African countries regarding resource use and transfers through the service
delivery supply chain. Furthermore, information problems are also acute at lower levels of the
hierarchy, because decentralized administrative units are often unaware of the budgetary
resources to which they are entitled.32 The information gap and retention of information at
31
As Holmström and Milgrom (1991) emphasize, if an employee has several tasks to carry out and some have good
performance measures and others do not, then making employees’ pay sensitive to good performance measures
will cause a substitution of efforts away from the other tasks and could result in loss of efficiency. This makes
provision of incentives hard when workers have to perform multiple tasks.
32
Typically, the central administrative level organizes procedures governing resource flow in the supply chain, but
there is little or no consultation with lower levels and scarce transmission of information. Those who receive the
inputs (textbooks, medical supplies, etc) at lower levels typically do not have information on what they were entitled
to receive, in terms of the type, quantity and value of supplies, or when and from whom they should receive the
resources.
209
various levels within public organizations reinforces moral hazard problems and underlies the
problem of inefficiencies and rent seeking.
4. What is the service delivery supply chain?
Figure 5 illustrates the sequential service delivery process involving the service delivery supply
chain activities at the upstream administrative level and the activities of frontline service
providers at the downstream level. Outcomes are realized at the household level.
Government at the upstream level sets overall policies, allocates resources and designs rules
and service providers’ incentive systems; service providers’ behavior downstream is
conditioned by these sets of constraints and incentives as well as rules determined locally;
while citizens’ decisions are influenced by the choices offered and the characteristics of the
services. Together, the choices and behavior of these various actors determine outcomes
(Figure A2).
Indeed, services offered at the frontline level are determined to a large extent by the rules
determined by the higher administrative levels and by the efficiency of the entire supply chain.
Several factors within the downstream frontline service provider environment are exogenous
being determined by the upstream level decisions. In most public health clinics in Africa, for
instance, the health personnel is selected and remunerated by the higher administrative levels,
generally the national government. Also, the geographical distribution of health centers and
clinics are determined at the central level. While personnel in frontline facilities (schools, clinics,
hospitals) provide services to the population, their behavior depend to a large extent on the
upstream actions taken by the higher administrative levels which provide key inputs (financing,
infrastructure, personnel, material and equipment).
Given the sequential nature of the service delivery process, the outputs of the first stage
(upstream) service delivery supply chain are the inputs of the second stage (downstream) of the
frontline service providers. The behavior and efficiency of upstream activities will determine
the resources available that will reach the downstream level of the frontline providers. Services
provided to the population will be a function of the overall efficiency of these joint activities.
Diagnosing service provision performance requires accounting for the sequential nature of the
production process and various administrative layers in the system.
210
Figure A2: Service Delivery: Sequential Delivery Process
On the demand side, the quality and accessibility of basic services in social sectors are
important factors as part of a complex socio economic process explaining population outcomes.
The access to quality public service delivery has been recognized as fundamental for population
wellbeing and economic development in developing countries, in particular in health,
education, water and sanitation and other social sectors (World Bank, 2003).
5. What are the potential deficiencies in service delivery?
While adequate financing, infrastructure, human resources, material and equipment have to be
available for service delivery at the frontline, together with proper institutions and governance
structures to provide adequate incentives to the service providers, the availability of these
essential elements and institutions are often not present.
Table 4 presents official public spending on education and health among African countries in
proportion to GDP, as well as in terms of GDP per capita.
211
Public spending in health and education in Africa varies greatly among countries, representing
between 2 percent to 22 percent of GDP, much below the levels observed in OECD countries
which are about 13 percent. Spending per capita varies even more widely being as low as US$6
in Burundi to US$750 in Botswana.
Official public spending figures could be deceitful for assessing countries’ actual commitment to
social sectors in weak institutional environments as funding may not translate into increased
available resources and improved services given leakage and inadequate incentives.
Leakage, which is broadly defined as the share of resources earmarked to specific beneficiaries
that fail to reach them, was found to be important in developing countries. Greater leakage has
been associated with greater discretionary power granted to particular administrative units,
combined with weak supervision and poor incentives, and could lead to large fund capture
(Reinikka and Svensson, 2004). In the health sector in Chad for instance, where no fixed
allocation rules apply for most resource allocation, only 1 percent of non-wage recurrent
resources allocated to regional health administration arrived at the facility level. This
represented less than US$0.02 per capita of (non-wages) annual budget available for primary
health care in 2003 (Gauthier and Wane, 2009).
With inadequate levels of public resources at the frontline, health and school facilities often
lack the basic infrastructure (electricity, water and sanitation), and also often face important
constraints of basic material and equipment. For instance, in Africa, SACMEQ II survey found
that over half of grade six students in several SSA countries did not have a single book (e.g.,
Malawi, Mozambique, Uganda, and Zambia).
Given that frontline providers are often left to their own devices to finance their activities, they
rely more heavily on user fees which could thus be very substantial. In Chad, user fees
represent 90 percent of health center revenues (excluding salaries) compared to 2 percent for
government transfers. They represented on average a burden 10 times heavier for the poorer
quintile of revenue than for the richer one (21 percent of the monthly household incomes
versus 2.1 percent).
Leakage has also been linked to significantly higher medication cost. In Chad in particular,
primary health facilities that receive government transfers charge lower mark-ups on
medication than facilities that do not. In addition to official fees, users of services in SSA also
face important informal payments that increase accessibility problems, especially for the poor.
In Kenya for instance, the probability of being asked to make an informal payment in the health
sector was found to be 61 percent compared with 54 percent in the education sector
(Transparency International, 2008)
Other weaknesses in the broader institutional architecture are associated with inadequate staff
incentives, which translate into shortages of skilled staff and high absenteeism, especially in
poorer and rural areas. In Namibia for instance, there is an unequal allocation of teachers in the
212
education sector (number and qualifications) across regions and types of schools. Qualified
teachers tend to concentrate in urban areas where 60 percent of teachers have obtained a
bachelor degree or more, while 80 percent of teachers in rural areas do not have secondary
education.
Table A1: Official Public Spending on Education and Health in Africa (2005)
Education
Health
Sum
(% of GDP)
(% of GDP)
($US per capita)
Angola
2.39
1.47
67.82
Benin*
4.38
2.51
34.11
Botswana
9.74
4.45
751.77
Burkina Faso
4.25
3.99
32.20
Burundi
5.10
0.97
6.43
Cameroon
3.07
1.46
47.10
Cape Verde
6.77
4.58
229.37
Chad
1.91
1.47
19.00
Congo, Rep.
1.87
0.89
35.22
Equatorial Guinea**
0.60
1.37
119.14
Eritrea
5.29
1.66
15.57
Ghana
5.44
2.11
36.52
Guinea
1.63
0.67
6.57
Kenya
7.31
2.10
49.39
Lesotho
13.84
8.47
180.64
Madagascar
3.19
2.00
14.07
Malawi**
5.81
9.54
21.95
Mali
4.15
2.93
26.70
Mauritania
2.34
1.71
24.86
Mauritius
4.44
2.21
343.77
Mozambique
5.02
2.73
25.96
Namibia**
6.92
5.14
271.71
Niger**
2.40
1.97
9.15
Rwanda
3.76
4.10
18.53
Senegal
5.43
1.71
50.95
Seychelles*
5.42
4.74
834.38
Sierra Leone
3.79
1.91
12.30
Swaziland
6.98
4.04
266.09
Uganda*
5.24
2.23
18.30
Zambia
1.99
2.74
29.44
SSA average
4.68
2.93
119.97
OECD Average
5.44
7.07
4 582.19
Source: World Bank WDI on line. Data are for year 2005 except (*) 2004 (**) 2003
Improving outcomes would require more than an adequate level of qualified personnel; it
would also require them to be present to deliver services. However, large multi-country studies
show that high levels of absenteeism are prevalent in developing countries due to poor
accountability and weak staff incentives (Chaudhury et al, 2006)
213
6. What are the long and short routes of accountability?
The main solutions examined to improve service delivery performance and for which
PETS/QSDS are instrumental have focused on reducing information asymmetry through
monitoring and evaluation and strengthening the accountability relationships which are at the
root of institutional deficiencies within service delivery systems.
Besley and Ghatak (2003) underline that inefficiency in public production arises because service
providers face limited incentives to improve quality. This is because service users (citizens) are
not viewed as customers as in the private sector, where customer satisfaction is key to the
provider’s survival. Also, public service providers have no incentives to reduce costs because of
the use of soft budget constraints in the public sector. The authors argue that decentralized
organizational arrangement is in many contexts an efficient system of public service delivery
that enhances consumer empowerment. Empowering consumers means that the nature of the
principal-agent relationship changes and that customer satisfaction becomes a priority of
frontline providers.
The World Development Report 2004 “Making Services Work for Poor People” develops a
conceptual framework that focuses on the accountability relationships between principals and
agents in the service delivery system and has identified two main routes within this framework
to reduce information asymmetry and improve accountability: (i) the improvement of internal
systems and (ii) external accountability mechanisms, also called respectively the “long” and
“short” route of accountability.
The internal sequential process of accountability has been referred to as the “long route” of
accountability, as opposed to the “short route,” which involves direct accountability of
providers to clients, a situation typically encountered in the private competitive sector.
Inadequate service delivery could hence be associated with failures in one or both of the links
along the “long route” of accountability, as well as with failures in the “short route” of
accountability.
An important way to strengthen the accountability relationships in service delivery is by
reducing information asymmetry among parties in the service provision system. Collecting
information on activities of agents increases the principal’s (citizens) bargaining power and
control through improved monitoring and disciplining of policy-makers and providers.
(i) The “long route” of accountability
The accountability relationship between citizens and policymakers could be broken for several
reasons. One of the accountability problems in the service delivery system relates to the
relationship between policymakers and frontline providers. As previously emphasized,
resources and information flows in public administrations are complex. Organizational
arrangements include a large number of agents and administrative layers.
214
In many developing countries, there is a crucial lack of information at various levels in the
public organizational structure, particularly at the central level, regarding resource use and
transfers through the service delivery supply chain. Furthermore, information problems are also
acute at lower levels of the hierarchy, in that decentralized administrative units are often
unaware of the budgetary resources to which they are entitled. Typically, the central
administrative level organizes procedures governing resource flow in the supply chain, but
there is little or no consultation with lower levels and scarce transmission of information. Those
who receive the inputs (textbooks, medical supplies, etc) at lower levels typically do not have
information on what they were entitled to receive, in terms of the type, quantity and value of
supplies, or when and from whom they should receive the resources. The information gap and
retention of information at various levels within public organizations reinforces moral hazard
problems and underlies the problem of inefficiencies and rent seeking.
(ii) The “short route” of accountability
Given these shortcomings in the accountability relationships between citizens and policymakers
and between policymakers and providers, public service provision is adversely affected unless
there is a mechanism that enables clients to monitor and directly discipline service providers-that is, through the “short route” of accountability (World Bank, 2003).
Indeed, there is a growing belief that local participation by citizens in service delivery and better
information can help achieve better outcomes, especially by applying mechanisms that enable
clients to monitor and directly discipline service providers (World Bank, 2003; Banerjee et al.,
2006).
Increasing client power over service providers can increase efficiency and control in the system.
Barr et al., (2006) describe an experiment in Ethiopia that studied the impact of various
institutional arrangements on rent capture in service delivery. They find that closer service
recipient control increases efficiency. 33
The information and analysis provided by PETS/QSDS could have a positive effect on both
routes of accountability. For both internal and external approaches to accountability,
PETS/QSDS could provide crucial information and recommendations for improved services.
33
In a democratic political setting, evidence suggests that accountability could be improved when voters share
information about political responsibility for certain key outcomes (Ferejohn and Kuklinsky, 1990). Khemani (2004)
finds evidence of the impact of information on service provision in India through election cycles. She observes that
when elections are approaching, state governments in India tend to increase expenditures on public investments
that are easily observable by voters and reduce them on more broad-based public services.
215
References
Ablo, Emmanuel and Ritva Reinikka (1998) “Do Budgets Really Matter? Evidence from Public
Spending on Education and Health in Uganda” Policy Research Working Paper 1926, The
World Bank, Washington, D.C.
Amin, Samia, Das Jishnu and Markus Goldstein (editors) (2008) Are you Being Served? New Tools
for Measuring Service Delivery, The World Bank, Washington, D.C. December.
Anderson, James, Daniel Kaufmann and Francesca Recanatini (2003) “Service Delivery, Poverty and
Corruption: Common Treads from Diagnostic Surveys” Background paper for the World
Development Report 2004, The World Bank, Washington, D.C., processed.
Ahmad, Junaid, Shantayanan Devarajan, Stuti Khemani and Shekhar Shah (2005) “Decentralization
and Service Delivery” Policy Research Working Paper 3603, The World Bank, Washington,
D.C.
Baliga, S. and T. Sjöström (1998) “Decentralization and Collusion“, Journal of Economic Theory, 83:
196-232.
Banerjee, Abhijit, Angus Deaton and Esther Duflo (2004) “Wealth, Health and Health Services in
Rural Rajasthan” AER Papers and Proceedings, 94 (2): 326-330.
Banerjee, Abhijit and Esther Duflo (2006) “Addressing Absence” Journal of Economic Perspectives,
20 (1): 117-132.
Banerjee, Abhijit, Esther Duflo, Rachel Glennerster, Daniel Kenniston, Stuti Khemani and Marc
Shotland (2006) “Can Information Campaigns Raise Awareness and Local Participation in
Primary Education? A study of Jaunpur district in Uttar Pradesh”, MIT, July, mimeo.
Barr, Abigail, Magnus Lindelöw and Pieter Serneels (2005) “Choosing Between Self and Service: An
Experimental Analysis of Corruption and Monitoring”, University of Oxford, mimeo.
Behrman, Jere R. and Anil B. Deolalikar (1988) “Health and Nutrition”, in Chenery, H. and T.N.
Srinivasan (Editors) Handbook of Development Economics, Vol. 1: 631-711, Elsevier Science.
Besley, Timothy and John McLaren (1993) “Taxes and Bribery: The Role of Wage Incentives”
Economic Journal, 103: 119-141.
Besley, Timothy and Robin Burgess (2003) “The Political Economy of Government Responsiveness:
Theory and Evidence from India”, Quarterly Journal of Economics, 117 (4): 1415-51.
Besley, Timothy and Maitreesh Ghatak (2003) “Incentives, Choice and Accountability in the
Provision of Public Services” Oxford Review of Economic Policy, 19 (2): 235-249.
216
Besley, Timothy and Maitreesh Ghatac (2006) “Reforming Service Delivery”, Journal of African
Economies (16): 127-156.
Besley, Timothy, Rohini Pande and Vijayendra Rao (2005) “Participatory Democracy in Action:
Survey Evidence from Rural India”, Journal of the European Economic Association, 3 (2-3):
648-657.
Björkman, Martina, and Jakob Svensson (2009), “Power to the People: Evidence from a
Randomized Field Experiment on Community-based Monitoring in Uganda”, Quarterly
Journal of Economics 124 (2).
Björkman, Martina, Ritva Reinikka and Jakob Svensson (2006) “Local Accountability” Scandinavian
Working Papers in Economics No. 749, Institute of International Economic Studies,
Stockholm University.
Bold, T., B. Gauthier, O. Maestad, J. Svensson, andW.Wane (2010) "Service Delivery Indicators:
Pilot in Education and Health Care in Africa", Report prepared for the African Economic
Research Consortium (AERC), Nairobi, Kenya.
Burgess, Simon and Marisa Ratto (2003) “The Role of Incentives in the Public Sector: Issues and
Evidence” Oxford Review of Economic Policy, 19 (2): 285-300.
Burke, Kathleen and Kathleen Beegle (2004) “Why Children Aren’t Attending School: The Case of
Northwestern Tanzania”, Journal of African Economies, 13 (2): 333-355.
Burki, Shaid Javed, Guillermo Perry and William Dellinger (1999) “Beyond the Center:
Decentralizing the State”, World Bank, Washington, DC.
Castro-Leal, Florencia, Julie Dayton, Lionel Demery and Kalpana Mehra (2000) “Public Spending on
Health Care in Africa: Do the Poor Benefit?” Bulletin of the World Health Organization 78
(1): 66-74.
Chattopadhyay, Raghabendra and Esther Duflo (2004) “Women as Policymakers: Evidence from an
India-Wide Randomized Policy Experiment”, Econometrica, 72 (5): 1409-1443.
Chaudhury, Nazmul and Jeffrey Hammer (2004) “Ghost Doctors: Absenteeism in Rural Bangladeshi
Health Facilities”, World Bank Economic Review, 18 (3): 423-441.
Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muralidharan and Halsey Rogers
(2006) “Missing in Action: Teacher and Health Worker Absence in Developing Countries”,
Journal of Economic Perspectives, 20 (1): 91-116.
Collier, Paul, Stefan Dercon and John Mackinnon-(2002) “Density versus Quality in Health Care
Provision: Using Household Data to Make Budgetary Choices in Ethiopia”, World Bank
Economic Review, 16 (30): 425-448.
Das, Jishnu (2004) “Equity in Educational Expenditures: Can Government Subsidies Help?” Policy
Research Working Paper 3249, The World Bank, Washington, D.C.
217
Das, Jishnu, Stefan Dercon, James Habyarimana and Pramila Krishnan (2004a) “Public and Private
Funding Basic Education in Zambia: Implications of Budgetary Allocations for Service
Delivery” Africa Region Human Development Working Paper Series No. 62, The World
Bank, Washington, D.C.
-------- (2004b) “When Can School Inputs Improve Test Scores”, Policy Research Working Paper
3217, The World Bank, Washington, D.C.
--------- (2005) “Teacher Shocks and Student Learning: Evidence from Zambia” Policy Research
Working Paper 3602, The World Bank, Washington, D.C.
Das, Jishnu, Bernard Gauthier and Tomas Philipson (2004) “Black Market for Medication in
Developing Countries”, University of Chicago, mimeo.
Das, Jishnu and Jeffrey Hammer (2005) “Which Doctor? Combining Vignettes and Item Response
to Measure Clinical Competence”, Journal of Development Economics, 78: 348-383.
Das Gupta, Monika, Varun Gauri and Stuti Khemani (2003) “Primary Health Care in Nigeria:
Decentralized Service Delivery in the States of Lagos and Kogi.” Africa Region Human
Development Working Paper Series No. 70, The World Bank, Washington, D.C.
Davis, Jennifer (2003) “Corruption in Public Service Delivery: Experience from South Asia’s Water
and Sanitation Sector”, World Development, 32 (1): 53-71.
DFID (Department for International Development) “Public Expenditure Tracking Surveys.”
http://www.dfid.gov.uk/aboutdfid/organisation/pfma/pfma-pets.pdf.
Dehn, Jan, Ritva Reinikka and Jakob Svensson (2003) “Survey Tools for Assessing Performance in
Service Delivery.” In Francois Bourguignon-and Luiz Pereira da Silva (eds.). The Impact of
Economic Policies on Poverty and Income Distribution: Evaluation Techniques and Tools. A
copublication of the World Bank and Oxford University Press.
Deininger, Klaus and Paul Mpuga (2004) “Does Greater Accountability improve the Quality of
Public Service Delivery? Evidence from Uganda”, World Development, 33 (1): 171-191.
---------- (2005) “Economic and Welfare Impacts of the Abolition of Health User Fees: Evidence from
Uganda” Journal of African Economies, 14 (1): 55-91.
Dercon, Stefan and Pramila Krishnan (2000) “In Sickness and in Health: Risk Sharing within
Households”, Journal of Political Economy 103 (4): 688-727.
Devarajan, Shantayanan and Ritva Reinikka (2004) “Making Services Work for Poor People.”
Journal of African Economies, vol. 13 (suppl. 1): i142-i166.
Dixit, Avinash (2002) “Incentives and Organization in the Public Sector: An Interpretative Review”,
Journal of Human Resources, 37 (4): 696-727.
218
Dranove, David, Daniel Kessler, Mark McClellan and Mark Satterthwaite (2003) “Is More
Information Better? The Effects of “Report Cards” on Health Care Providers”, Journal of
Political Economy, 111 (31): 555Duflo, Esther, Hanna, Rema and Stephen Ryan (2007) “Monitoring Works: Getting Teachers to
Come to School, MIT, May, mimeo.
Engberg-Pedersen, Paul, Kai Kaiser, Gregory Kisunko, Inna Kushnarova and Magnus Lindelöw
(2005) Public Expenditure Tracking Surveys: A Stocktaking, The World Bank, Washington,
D.C., processed.
Feyziogly, Tarhan, Vinaya Swaroop, and Min Zhu (1998) "A Panel Data Analysis of the Fungibility
of Foreign Aid" World Bank Economic Review 12(1):29–58.
Ferejohn, John and James H. Kuklinsky (eds) (1990) Information and Democratic Processes,
Urbana: University of Illinois Press.
Filmer, Deon and Lant Pritchett (1999) “The Impact of Public Spending on Health: Does Money
Matter?” Social Science and Medicine, 58: 247-258.
Filmer, Deon (2003) “The Incidence of Public Expenditures on Health and Education,” Background
Note for the World Development Report 2004, Washington, D.C., processed.
Filmer, Deon; Jeffrey Hammer and Lant Pritchett (2000) “Weak Links in the Chain: A Diagnosis of
Health Policy in Poor Countries”, World Bank Research Observer, 15 (2): 199-224.
--------- (2002) “Weak Links in the Chain II: A Prescription for Health Policy in Poor Countries”,
World Bank Research Observer, 17 (1): 47-66.
Francken, Nathalie (2003) “Service Delivery in Public Primary Schools in Madagascar: Results of
a Budget Tracking Survey” The World Bank, Madagascar Country Office, Antananarivo,
September, processed.
Gertler, P. and J. Van der Gaag (1990) The Willingness to Pay for Medical Care: Evidence from Two
Developing Countries, Baltimore MD: Johns Hopkins University Press for the World Bank.
Gauthier, Bernard (2002) “Hierarchies and Delegation: Sequential Production Process in an
Organizational Setting”, HEC Montréal, processed.
-------- (2006) “PETS and QSDS in Sub-Saharan Africa: A Stocktaking Study”, Working paper #45091,
HEC
Montréal
and
World
Bank,
September
http://wwwwds.worldbank.org/external/default/main?pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&
searchMenuPK=64187283&theSitePK=523679&entityID=000333037_20080822011716&searchMenuPK=64187283&theSi
tePK=523679
-------- (2008) “Harmonizing and Improving the Efficiency of PETS and QSDS”, HEC Montréal and
The World Bank, August, processed.
Gauthier, Bernard and Ritva Reinikka (2007) “Methodological Approaches to the Study of
Institutions and Service Delivery: PETS, QSDS and CRCS”, African Economic Research
219
Consortium
Framework
paper,
AERC
and
World
Bank,
September.
http://aercafrica.org/documents/isd_workingpapers/GauthierReinikkaMethodologicalApproachestotheStudyofISD.pd
f
Gauthier, Bernard and Waly Wane (2005) “Suivi des dépenses publiques à destination dans le
secteur santé au Tchad: Analyse des résultats d’enquête," Development Research Group, The
World Bank, Washington, D.C., processed.
--------- (2008) “Leakage of Public Resources in the Health Sector: An Empirical Investigation of
Chad”, Journal of African Economies (18): 52-83
--------- (2011)“Bypassing of Health Providers: Competition, Price and Quality of Health Services in
Chad,” Social Science and Medicine (73): 540-549, 2011
Government of Rwanda (2004) “Rwanda: Public Expenditure Tracking Survey (PETS) Education”,
September, processed.
Government of Uganda (2004) “Access to Education and Health Care in Uganda,” mimeo.
Gurkan, Asli, Kaiser, Kai and Doris Voorbraak (2009) “Implementing Public Expenditure Tracking
Surveys for Results: Lessons from a Decade of Global Experience”, PREM Note number
145, The World Bank, November
Hellwig, Christian (2002) “Public Information, Private Information and the Multiplicity of
Equilibria in Coordination Games”, Journal of Economic Theory, 107 (2): 191-222.
Hirschman, Albert O (1970) Exit, Voice, and Loyalty: Responses to Decline in Firms,
Organizations, and States. Cambridge, Mass.: Harvard University.
Holmström, Bengt (1979) “Moral Hazard and Observability”, Bell Journal of Economics, 10: 7491.
-------- (1982) “Moral Hazard in Teams”, Bell Journal of Economics, 13: 324-340.
Hutchinson, Paul (2001) “Combating Illness.” In Ritva Reinikka and Paul Collier (Eds.) Uganda
Recovery: The Role of Farms, Firms and Government. Regional and Sectoral Studies.
Washington D.C.: World Bank.
Jeppson Andreas. (2001) “Financial Priorities under Decentralization in Uganda”, Health Policy and
Planning 16 (2): 187-192.
Keefer, Philip and Stuti Khemani (2005) “Democracy, Public Expenditures and the Poor:
Understanding Political Incentives for Providing Public Services", World Bank Research
Observer, 20 (1): 1-27.
Khemani, Stuti (2004) “Political Cycles in a Developing Economy: Effect of Elections in the Indian
States”, Journal of Development Economics 73 (1): 125-154.
220
--------- (2006) “Local Government Accountability for Health Service Delivery in Nigeria”, Journal of
African Economies, 15: 285-312.
Koziol, Margaret and Courtney Tolmie (2010) Using Public Expenditure Tracking Surves to Monitor
Projects and Small-Scale Programs: A Guidebook, The World Bank, Washington DC.
Kremer, Michael, Sylvie Moulin and Robert Namunyu (2003) “Decentralization: A Cautionary Tale”,
Harvard University, March, mimeo.
Laffont, Jean-Jacques and David Martimort (1998) “Collusion and Delegation” Rand Journal of
Economics, 29 (2): 280-305.
Le Grand, J. (1991) “Quasi-Markets and Social Policy”, Economic Journal, 101: 1256-67.
Leonard, Kenneth L. and Melkiory C. Masatu (2005) “The Use of Direct Clinician Observation and
Vignettes for Health Service Quality Evaluation in Developing Countries” Social Science &
Medicine, 61: 1944-1951.
Leonard, K.L., (2007) "Learning in Health Care: Evidence of Learning about Clinician Quality in
Tanzania," Economic Development and Cultural Change, 55 (3): 531-555
Lewis, Maureen (2006) “Governance and Corruption in Public Health Systems” Working Paper 78,
Center for Global Development, Washington D.C., January.
Lindelöw, Magnus (2004), “Health Care Decisions as a Family Matter: Intrahousehold Education
Externalities and the Utilization of Health Services.” Policy Research Working Paper 3324.
Development Research Group, The World Bank,Washington, D.C.
-------- (2006) “Tracking Public Money in the Health Sector in Mozambique: Conceptual and
Practical Challenges”, East Asia Human Development Unit, The World Bank, Washington,
D.C., April, processed.
Lindelöw, Magnus and Adam Wagstaff (2003) “Health Facility Surveys: An Introduction” Policy
Research Working Paper 2953, The World Bank, Washington D.C.
Lindelöw, Magnus, Ritva Reinikka and Jakob Svensson (2003) “Health Care on the Frontlines:
Survey Evidence on Public and Private Providers in Uganda” Africa Region Human
Development Working Paper Series, The World Bank, Washington D.C.
Lindelöw, Magnus, Patrick Ward and Nathalie Zorzi (2004), “Primary Health Care in Mozambique:
Service Delivery in a Complex Hierarchy,” Africa Region Human Development Working
Paper 69, Development Research Group, The World Bank, Washington, D.C.
Lindelöw, Magnus, Inna Kushnarova and Kai Kaiser (2006) “Measuring corruption in the health
sector: what we can learn from public expenditure tracking and service delivery surveys in
developing countries”, In J. Kotalik and D. Rodriguez (Eds.) Global Corruption Report 2006:
Corruption and Health, London: Pluto Press.
221
Lindelöw, Magnus and Pieter Serneels (2006) “The Performance of Healh Workers in Ethiopia:
Results from Qualitative Research”, Social Science and Medicine, 62: 2225-2235
Manshande, Jean-Pierre and Iona Zamfir (2005), « L’Efficience et l’Equité des Formations Sanitaires
Malgaches: Résultats d’une enquête” Africa Region Human Development Working Paper
No. 76. The World Bank, Washington D.C.
Marschak, Thomas and Stefan Reichelstein (1998) “Network Mechanisms, Informational Efficiency,
and Hierarchies”, Journal of Economic Theory, 79 (1): 106-141.
MCA-BONGA (2007) Engaging Communities and Civil Society Organizations in Public Expenditure
Tracking: A Training Manual, Millennium Challenge Account and PACT, USAID Tanzania.
Melumad, N., D. Mookherjee and S. Reichelstein (1992) “A Theory of Responsibility Centers,”
Journal of Accounting and Economics, 15: 445-484.
Ministère de la Santé Publique (2001) “Revue des Dépenses Publique du Secteur Santé” Direction
Générale, Division des Ressources Financières, République du Tchad, processed.
Minten, Bart, Mead Over, Mamisoa Razakamanantsoa and Waly Wane (2004). “The Potential of
Community-Based Insurance towards Health Finance in Madagascar”, June, processed.
Mookherjee, Dilip and I.P.L. Png (1995) “Corruptible Law Enforcers: How Should They Be
Compensated?” Economic Journal, 105: 149-159.
Mookherjee, Dilip and M. Tsumagari (2004) “The Organization of Supplier Networks: Effects of
Delegation and Intermediation” Econometrica, 72 (4): 1179-1219.
Mookherjee, Dilip (2006) “Decentralization, Hierarchies and Incentives: A Mechanism Design
Perspective”, Journal of Economic Literature, 44: 367-390.
Morris, Stephen and H. S. Shin (2002)”The Social Value of Public Information”, American Economic
Review, 92 (5): 1521-1534.
Msuya, Joyce (2004) “Horizontal and Vertical Delivery of Health Services: What Are the Trade
Offs?” Background paper for the World Development Report 2004, The World Bank,
Washington D.C., processed.
Mukwena, R.M.; Z.O Bolo and J.E. Odada (2003) A Public Expenditure Tracking Survey : Ministries
of Health and Social Services and Basic Education, Sport and Culture, The Namibian
Economic Policy Research Unit and Office of the Auditor General, May, processed.
Musgrove, Philip (1996) “Public and Private Roles in Health: Theory and financing patterns,” World
Bank Discussion Paper 339, Washington D.C.
Mwabu, G., M., Ainsworth, and A., Nyamete. (1993) “Quality of Medical Care and Medical
Treatment in Kenya: An Empirical Analysis”, The Journal of Human Resources, 38 (4): 838862.
222
Over, Mead, Mattias Lundberg, Waly Wane and Sergio Bautista (2006) “Estimating and explaining
the efficiency of health service delivery in developing countries: with applications to HIV
prevention in five countries and primary care in Chad”, Presentation at the workshop on
measuring service delivery, World Bank, Washington, D.C., April 28, processed.
Picazo, Oscar F. and Feng Zhao (2009) “Zambia Health Sector Public Expenditure Review:
Accounting for Resources to Improve Effective Service Coverage”, A World Bank Country
Study, Report # 47110, The World Bank, Washington, DC.
Paul, Samuel (2002) Holding the State to Account: Citizen Monitoring in Action. Bangalore: Books
for Change.
Radner, Roy and Jacob Marschak (1972) Economic Theory of Teams, New Haven: Yale University
Press.
Radner, Roy (1992) “Hierarchy: The Economics of Management”, Journal of Economic Literature,
30 (3): 1382-1415.
Rauch, James E. and Peter B. Evans (2000) “Bureaucratic Structure and Bureaucratic
Performance in Less Developed Countries”, Journal of Public Economics, 75: 49-71
Rajkumar, Andrew Sunil and Vinaya Swaroop (2007) “Public Spending and Outcomes: Does
Governance Matter?” forthcoming, Journal of Development Economics.
Reinikka, Ritva (2001) “Recovery in Service Delivery: Evidence from Schools and Health Centers” In
R. Reinikka and P. Collier (Eds), Uganda’s recovery: The role of farms, firms and
government. World Bank Regional and Sectoral Studies, Washington DC: World Bank.
Reinikka, Ritva and Nathanael Smith (2004) Public Expenditure Tracking Surveys in Education,
International Institute for Educational Planning, UNESCO, Paris.
Reinikka, Ritva and Jakob Svensson (2001) “Explaining Leakage of Public Funds” Policy Research
Working Paper 2709. The World Bank, Washington, DC.
------------ (2003) “Measuring and Understanding Corruption at the Micro Level.” In Donatella Della
Porta and Susan Rose-Ackerman (eds.), Corrupt Exchanges: Empirical Themes in the Politics
and the Political Economy of Corruption. Baden-Baden: Nomos Verlasgesellschaft.,
------------ (2004a) Efficiency of Public Spending: New Microeconomic Tools to Assess Service
Delivery.” In Tony Addison and Alan Roe (eds.). Fiscal Policy for Development: Poverty,
Reconstruction and Growth. Palgrave Macmillan. May
------------ (2004b) “Local Capture: Evidence from a Central Government Transfer Program in
Uganda”, Quarterly Journal of Economics, 119 (2): 1-28.
------------ (2004c) “Citizen Report Card at the Community Level: An Impact Evaluation”
Development Research Group, The World Bank, processed.
223
----------- (2005a), “Fighting Corruption to Improve Schooling: Evidence from a Newspaper
Campaign in Uganda.” Journal of the European Economic Association, 3 (2-3): 259-267.
----------- (2005b) The Power of Information: Evidence from a Newspaper Campaign to Reduce
Capture of Public Funds, Policy Research Working Paper Series 3239, The World Bank,
Washington, DC.
----------- (2006a) “Using Micro Surveys to Measure and Explain Corruption.” World Development,
33 (12) (February).
‐‐‐‐‐‐‐‐‐‐ (2006b) “How corruption affects service delivery and what can be done about it” in Susan
Rose Ackerman (ed) International Handbook on the Economics of Corruption, 441‐446,
Edward Elgar Publishing.
‐‐‐‐‐‐‐‐‐‐ (2010) “Working for God? Evidence from a Change in Financing of Nonprofit Health Care
Providers in Uganda.” Journal of the European Economic Association, 8 (6).
REPOA/ESRF (2001) “Pro Poor Expenditure Tracking” Draft report submitted to the PER working
group, Research on Poverty Alleviation and Economic and Social Research Foundation,
Tanzania, processed.
Republic of Kenya (2004) “Public Expenditure Tracking Survey (PETS) 2004”, Preliminary Report,
Ministry of Planning and National Development, Ministry of Finance, Ministry of Health,
Ministry of Education Science and Technology, Nairobi, Kenya, processed.
République du Cameroun (2004) “Enquête sur le Suivi des Dépenses Publiques et la Satisfaction
des Bénéficiaires dans les Secteurs de l’Éducation et de la Santé, Phase I: Volet Santé,”
Rapport Principal des Résultats, Institut National de la Statistique, Version provisoire, Mars,
processed.
Roberts, Mark and James Tybout (1995) “Producer Turnover and Productivity Growth in
Developing Countries” The World Bank Research Observer, 12 (1): 1-18.
Rogers, Halsey, Jose Roberto Lopez-Calix, N. Cordoba, Nazmul Chaudhury, Jeffrey Hammer,
Michael Kremer and Karthik Muralidharan (2004) “Teacher Absence and Incentives in
Primary Education: Results from a New National Teacher Tracking Survey in Ecuador,” in
Ecuador: Creating Fiscal Space for Poverty Reduction, Washington DC.: World Bank,
chapter 6.
Rodgers, Halsey and Margaret Koziol (2011) Provider Absence Surveys in Education and Health:
a guidance note, Office of the Chief Economist, Human Development Network, The
World Bank, Washington DC.
Schade, Klaus and John Ashipala (2004) Public Expenditure Tracking Survey (PETS) and
Quantitative Delivery Survey (QSDS): The Health Sector, The Namibian Economic Policy
Research Unit, August, processed.
224
Schade, Klaus and Erwin Naimhwaka (2004) Public Expenditure Tracking Survey (PETS) and
Quantitative Delivery Survey (QSDS): The Education Sector, The Namibian Economic
Policy Research Unit, August, processed.
Serneels, Pieter, Magnus Lindelöw, Jose Garcia-Montalvo and Abigail Barr (2005) “For Public
Service or Money: Understanding Geographical Imbalances in the Health Workforce”,
World Bank Policy Research Working Paper 3686, Washington, D.C., August.
Stasavage, David (2003) “On the Role of Democracy in Uganda’s move to Universal Primary
Education”, Background Paper for the 2004 World Development Report, World Bank,
Washington, D.C., processed.
Tirole, Jean (1986) "Hierarchies and Bureaucracies: On the Role of Collusion in Organizations",
Journal of Law, Economics, and Organizations, 2: 181-214.
Thomas, Duncan, Victor Lavy and John Strauss (1996) “Public Policy and Anthropometric
Outcomes in the Côte d’Ivoire” Journal of Public Economics, 61: 155-192.
Thomas, Melissa (1998) “Corruption and Political Finance in Africa”. PREM Seminar Series. World
Bank, Washington, D.C., processed.
-------- (1999) “The Incentive Structure of Systemic Corruption” Africa Region, World Bank,
Washington, D.C., processed.
TI (2008) The Kenya Bribery Index 2008, Transparency International Kenya.
Turner, Anthony G., Gustavo Angeles, Amy O. Tsui, Marilyn Wilkinson and Robert Magnani (2001)
“Sampling Manual for Facility Surveys: For Population, Maternal Health, Child Health and
STD Programs in Developing Countries”, MEASURE Evaluation Manual Series, No. 3,
MEASURE Evaluation, Carolina Population Center, University of North Carolina at Chapel
Hill, July.
UNESCO (2009) Education For All Global Monitoring Report 2009: Overcoming inequality: why
governance matters, UNESCO publishing and Oxford University Press.
van der Geest, S. (1987) “Self-Care and the Informal Sale of Drugs in South Cameroon” Social
Science and Medicine, 25: 293-305.
Wallace, Lindsay; Jim Edgerton, Richard Harris and Erisa Ocheing (2003) Sierra Leone: Public
Expenditure Tracking Surveys (PETS) Review, Final Report, DFID and World Bank,
processed.
Wane, Waly (2006) “Linking Leaking Public Funds and Outcomes: The Performance of Health
Districts in Chad”, Presentation at the Workshop on Measuring Service Delivery, World
Bank, Washington, D.C., April.
---------- (2008) “Public Expenditure Tracking Survey in a Difficult Environment: The Case of Chad” in
Amin, Das and Goldstein (editors) Are you Being Served? New Tools for Measuring Service
Delivery, The World Bank, Washington, D.C. Chap 8, pp. 191-220.
225
Wilson, James Q. (1989) Bureaucracies: What Government Agencies Do and Why They Do It, New
York, Basic Books.
World Bank (1996) “Access to Education and Health Care in Uganda”, Country Operations
Division, Eastern Africa Department and Poverty and Social Policy Department,
Washington, D.C. September, processed.
--------- (2003) World Development Report 2004: Making Services Work for Poor People, The World
Bank and Oxford University Press, Washington, D.C.
--------- (2004a) “Papua New Guinea: Public Expenditure and Service Delivery”, East Asian and the
Pacific Region, Washington, D.C., processed.
--------- (2004b) “Le Secteur de la Santé au Tchad: Analyse et Perspectives dans le Cadre de la
Stratégie de Réduction de la Pauvreté” Groupe Thématique Santé et Pauvreté, February,
processed.
---------- (2006) Expenditure Tracking Survey in Senegal: The Health Sector, April, processed.
---------- (2007) “Madagascar Service Delivery in the Education and Health Sector: Results of the
2006/2007 Public Expenditure Tracking Survey,” Washington D.C., December.
--------- (2008a) Public Sector Reform: What Works and Why? An IEG Evaluation of World Bank
Support, Independent Evaluation Group, The World Bank, Washington DC.
----------(2008b) Niger Public Expenditure Tracking Survey: Education and Health, Volume I: Main
report, PREM Africa Region, World Bank Document, Washington, DC. December.
--------- (2008c) Tajikistan: Second Programmatic Public Expenditure Review (In Four Volumes)
Volume IV: Public Expenditure Tracking Survey (PETS) Health Sector, Poverty Reduction and
Economic Management Unit Europe and Central Asia Region, Report No. 43280-TJ, The
World Bank, Washington, DC. June
--------- (2009) “Afghanistan General Education Public Expenditure Tracking”, Draft Concept Note,
December.
World Bank/INSTAT (Institut National de la Statistique Madagascar) (2005) “L’efficience et l’équité
auprès des formations sanitaires Malgaches : Résultats d’un enquête,” Africa Region,
Human Development Department, May.
Ye, Xiao and Sudharshan Canagarajah (2002) “Efficiency of Public Expenditure Distribution and
Beyond: A Report on Ghana’s 2000 Public Expenditure Tracking Survey in the Sectors of
Primary Health and Education”. Africa Region Working Paper Series No. 31, The World
Bank, Washington, D.C.
226