March 2017 update: Curbing Corruption in Aid-funded Procurement
Olli Hellmann (Sussex); Mihály Fazekas (GTI/Cambridge); Liz Dávid-Barrett (Sussex)
Olli Hellmann and team held a workshop in Lima to discuss their findings regarding the case of Peru (and its neighbours) with government officials, NGOs, and academics. Some photographs from the workshop can be found below:
This project is one of eight British Academy-funded anti-corruption research projects, and you can see the main project page with full details of all awards and award holders here.
January 2017 update: Curbing Corruption in Aid-funded Procurement
The team has spent much of year one preparing the data and variables for the quantitative analysis. This involved three main stages. First, we engaged in an extensive process of mapping the available data, collecting contracts data from the World Bank, EuropeAid, and the IDB (the Inter-American Development Bank), cleaning, parsing, and testing the data to check whether it is of sufficiently high quality for analysis. Second, we constructed composite indicators of regime type, to underpin our analysis of how regime type affects patterns of corruption. Third, we analysed guidelines and rules on aid spending through national procurement systems for our three selected donors, and classified them into four types: oversight-enhancing, discretion-limiting, integrity-building, and capacity-building. This will enable us to analyse which types of controls might be most effective. After building the variables, data analysis began in November 2016. Our preliminary results inform our selection of cases for further study. We will be able to present some results of the quantitative analysis to the World Bank, IDB and DFID in early 2017.
The use of our data and indicators by donors, recipient governments, civil society watchdogs, journalists, and bidding companies will be the most direct measure for project success. Use can be measured in terms of data downloads and website hits but also more broadly through the number of references made to our data and indicators in media reports, policy papers, and other relevant documents. Downloads and visitors can be measured at a low cost on a daily basis. Broader use of the data and indicators will be reflected in the number of hyperlinks pointing at the dissemination website, but it will also be necessary to manually screen media and policy documents to adequately measure impact.
Measuring the impact of research findings is much more challenging given the complex nature of policy change, and the difficulty of establishing causal links between research findings and changes in policy. To ascertain whether donors implement regulatory tools and practical solutions identified as effective by our research project, we will rely on personal connections with individuals in donor organisations. To establish whether newly implemented regulations are effective, we will make use of our continuously updated data to measure whether policy changes are followed by a predicted reduction in procurement corruption risks. While we will not be able to firmly establish the causes of regulatory reform, this approach promises to establish whether and to what degree the proposed tools work in line with research findings.
Finally, our project also aims to develop in-country capacity for the use of our data analysis tools. Here, success can be measured by identifying whether local stakeholders continue to use our tools past the project and whether they establish local platforms to promote wider usage in their countries. We will rely on direct indicators (e.g. the view counter for our instructional video) and networks established during workshops as our main sources of information.
September 2016 update: Curbing Corruption in Aid-funded Procurement
Data collection and cleaning
The team has undertaken an extensive mapping of the extent and quality of data available from a range of donors. This revealed that data from USAID, one of the organisations that we proposed to study (along with the World Bank and EuropeAid), would be inadequate for our purposes. We therefore replaced this donor in the research design with the Inter-American Development Bank (IDB). This allows us to retain the comparative focus of our study and produce findings that are generalizable to the developing world as a whole.
The downloading of data (e.g. web scraping) is ongoing for all three sources. The parsing of data is completed for the World Bank and nearly complete for EuropeAid, and IDB data will be ready by end- October. We are undertaking a testing phase, to check whether the data is of sufficiently high quality for analysis. The next phase is to build the variables. Data analysis will begin in early November. A pilot World Bank database was used to test out some red flags of corruption and test their reliability with promising first results.
To explore the impact of the socio-political context on corruption in aid, we need to construct relevant composite indicators of regime type. These will be based on two dimensions, in keeping with the latest research: regime time horizon and state capacity. Based on the review of the state-of-the-art academic literature, we have identified and collected data on key indicators to feed into these two dimensions and on a number of control variables commonly used in large-n analyses of corruption. We will also construct our own indicator of state capacity based on the quality of the public procurement data that we collect.
Donors’ institutional controls
Through analysis of donor guidelines and rules on aid spending through national procurement systems, we are coding changes in institutional controls. Drawing on theory about anti-corruption controls, we have constructed four categories of controls – oversight-enhancing, discretion-limiting, integrity-building and capacity-building - so as to enable us to draw conclusions about which types of controls might be most effective. We are also coding interventions according to the stage of the procurement procedure which they aim to control. We will also try, as far as possible, to match reforms to specific red flags, to maximise the precision with which we can draw conclusions about causality.
We will present preliminary results of the quantitative analysis to donors and to DFID in early 2017. As the quantitative data becomes available in November, we will finalise our selection of case studies and design the fieldwork.
Collaboration with other ACE projects
- We will share our analysis of certain contracting authorities with Professor Jan-Hinrik Meyer-Sahling, to provide outcome data on the quality of procurement and corruption risk indicators that could be related to his project’s survey of bureaucrats working in these authorities.
- We will also provide our results on aid-financed procurement in the construction sector in Zambia to the team led by Professor Mundia Muya.
Elizabeth Dávid-Barrett and Mihály Fazekas are part of a small consortium (with the University of Oxford and University of Reading) that has been successful in winning a grant from the EPSRC Global Challenges Research Fund for a related short-term project based at the University of Oxford. The project will implement the Fazekas method for analyzing corruption risks in public procurement on the software tool R-Instat, an Africa-focused statistics package developed by African mathematicians, and test it on national procurement data from Tanzania.
This project should have many useful synergies with our BA/DFID ACE project. Intellectually, it will allow us to analyse national procurement data from one of our key countries, facilitating comparisons with the aid data. In terms of knowledge exchange and impact, the opportunity to work with African statisticians will help us to ensure that our methods are widely disseminated and tailored to local needs, as well as building research capacity in Africa.
Development aid – the practice of wealthy countries giving money to support the long-term development of poorer countries – is always controversial. Governments hesitate to look too generous with distant foreign populations while their own people are feeling the pinch. And the tension mounts with every scandal suggesting that corrupt governments siphon off aid, so that it never reaches the intended recipients.
The donor community has responded in part by bypassing governments, channeling aid directly to civil society organisations and communities. But that does not solve the problem. It also puts donors in an awkward position, appearing to challenge the sovereignty of recipient governments on whom they rely for their own license to operate.
Far better would be for aid agencies to learn to better control the way in which governments spend their funds. Yet until now, aid agencies have had only blunt tools available to monitor whether recipient governments use aid for agreed purposes or rather hand it out to cronies and clients.
Our research project addresses this problem. We develop an innovative methodology using detailed procurement data made available since 1998 by major donors including EuropeAid and the World Bank. More than 50% of development aid is delivered through procurement. From this ‘big data’, we calculate targeted proxy indicators of corruption. These are based on analyzing co-variation in ‘red flags’ in the process of awarding contracts (e.g., extremely short tender periods) and outcomes on procurement markets (e.g., only one bid received). The methodology has been widely endorsed by both academic and development communities.
We then use these indicators to systematically study two questions relating to the causal determinants of corruption in foreign aid, with a strong focus on the context in recipient countries.
First, we explore how the risks of corruption in aid spending are affected by the political context in recipient-countries. We expect that the degree to which power is centralized and the level of political competition will affect the techniques used to steal money, as well as the market outcomes.
Second, we test how different institutional control mechanisms work – in and of themselves, and in these different political contexts. Some tools might work better in centralized regimes, others where power is more dispersed.
Our findings will help donor agencies to develop more efficient delivery and monitoring mechanisms for their aid, tailored to the risks in a specific context. We will also make our data analysis tools available to donors so that they can incorporate them into their evaluation frameworks beyond the life of the project. We hope to deliver not only a better understanding of how corruption occurs and can be controlled, but also concrete tools to help donors ensure that aid reaches its targets.