Originally Published on GI-ACE
Different methods for studying anti-corruption and other governance priorities have different strengths and weaknesses. The promise of using multiple methods is that in combination they are likely to be more effective at producing insights about successful anti-corruption and governance strategies, particularly when context matters.
The major strength of randomized field experiments is well recognized — they allow us to make unbiased causal estimates of the effects of interventions in a specific time and place. But they suffer from two main limitations.
- The results of field experiments might be highly sensitive to decisions about the design of interventions.Perhaps the most stinging critique of field experiments and randomized evaluations in recent years, put forward by Lant Pritchett, is the idea that they offer limited opportunities to accumulate knowledge because the effects of classes of interventions are both highly dependent on the context and on decisions about the design of specific interventions. According to this view, the pursuit of knowledge that generalizes across contexts is mostly futile and instead evaluations need to “evaluate programs in context.”
- It can be very difficult to learn about the mechanisms that lead to the effects (or lack thereof) in randomized evaluations. While it is possible to study intermediate outcomes and attempt to collect data on all parts of a theory of change to allow for “mechanism mapping”, it is never easy to figure out whether the changes that you see are part of the operative causal pathway or not. If a randomized evaluation does not produce the results that are hoped for, then it can be difficult to pinpoint the reason that the intervention failed.
Recognizing these limitations, our team has pursued a fully paired ethnography and randomized field experiment to study anti-corruption strategies in the context of revenue-sharing for Bwindi National Park. While the background to this setting is covered in more detail in a previous post, our project seeks to identify and study a relatively untested approach to anti-corruption that involves the positive recognition of local leaders who manage public funds appropriately. In particular, we ask elected village leaders who must oversee the selection, procurement, and implementation of community-driven development projects to complete a checklist of tasks based on national guidelines for the appropriate management of public funds. We are interested in the idea that offering positive, social recognition to these leaders has the potential to decrease mismanagement and corruption involving revenue-sharing funds.
The two limitations outlined above are major challenges for our project:
- Decisions about the intervention design — specifically how to recognize local officials: Should leaders who do a good job be offered a medal or should their names be posted on a public sign? Should an award ceremony be held in their village or at the park headquarters? Should the standards for the award be set very high so that only a limited number of people are recognized or should the standards be set within the reach of all local leaders? These are difficult questions that might fundamentally change the results of the study. We are not able to answer all these questions as part of a randomized experiment and, instead, need a better way to make judgments about how social recognition can best be designed in the context where we are working.
- In terms of understanding mechanisms, we face another set of challenges. Does recognition work by boostin private willingness to engage in certain practices or does it work primarily by creating a social expectation for certain types of actions? Are community expectations or leader feelings about acting in a certain way more important Does the specific type of recognition influence how leaders will internalize the value of managing public funds according to the national guidelines?
To address these challenges, Prof. Paul Bukuluki is leading a paired ethnography that seeks to generate a deep understanding about the type of recognition intervention that would work best given local customs, norms, and beliefs in our study context. For instance, through a series of focus group discussions and in-depth interviews with key participants, his team is exploring how people gain social status and what different kinds of interventions are best able to generate the positive status for leaders that could overcome the material incentives to engage in corruption. By studying how leaders react to eligibility for recognition while the intervention is being received, we are better able to identify the pathways that allow or block a positive effect from emerging. This will allow practitioners to better design future interventions, according to whether the same channels to affect change exist in other contexts.
In essence, pairing an ethnography with a field experiment provides an opportunity to make better decisions about the design of a broad approach with local knowledge. The resulting intervention thus represents the “locally designed” realization of a broad class of similar interventions that may be of interest in other places. With the opportunity to study how the intervention is received in the field, we are able to better map out and substantiate each link in a theory of change, which in our case includes both that leaders will internalize new values when receiving positive social recognition and that recognition will change the public’s expectations for leaders and the willingness to hold them accountable. By studying why and how these kinds of mechanisms work (or don’t), we will be better able to advise practitioners in other contexts on how best to evaluate the applicability of the approach that we are testing at
Bwindi National Park.
