2006 Global Integrity Country Assessments and Global Integrity Index:
Methodology White Paper

The Global Integrity Approach

The Integrity Indicators are based on a simple yet revolutionary concept. Rather than trying to measure actual corruption, considered virtually impossible by experts, Global Integrity quantitatively measures the opposite of corruption, that is, the access that citizens and businesses have to a country's government, their ability to monitor its behavior, and their ability to seek redress and advocate for improved governance. The Integrity Indicators break down that "access" into a number of categories and questions, ranging from inquiries into electoral practices and media freedom to budget transparency and conflicts of interests regulations. We unpack these concepts by looking not only at what laws or institutions are "on the books" but by assessing their staffing levels, budget independence, political independence, and citizen access to the most important anti-corruption mechanisms. Combined with vivid narration prepared by our team of leading journalists, the extensive data provided by the Integrity Indicators informs and empowers citizens, activists, donors, businesses, and governments in each country.

Global Integrity provides detailed data and comparative country assessments of the mechanisms in place to prevent abuses of power and promote public integrity. Using a blend of social science and journalism, in-country teams of independent social scientists and investigative journalists report on the de jure as well as de facto reality of corruption and anti-corruption mechanisms. While the Reporter's Notebook on the culture of corruption often paints a depressing picture (how corruption looks, tastes, feels and smells to the average citizen), the Integrity Indicators identify strengths and weaknesses in the national anti-corruption architecture and serve as a road map for possible reforms.

Transparency, both in terms of our methodology and findings, is what characterizes the Global Integrity approach and enhances the robustness and credibility of our findings. As we continue to improve our methodology and learn from our field work experiences, we welcome your critical feedback

The Washington Team

In 2006, Global Integrity benefited greatly from the feedback of an informal methodology advisory committee set up to provide technical assistance and feedback on methodological and scoring issues. The group included the following experts acting in their personal capacity:

• Omar Azfar (IRIS)
• Monica Dorhoi (World Bank)
• Gregory Kisunko (World Bank)
• Andrew Greene (USAID)
• Jennifer Gauck (ABA-CEELI)
• Leslie Harper (Inter-American Development Bank)
• Gregory Kisunko (World Bank)
• Steve Knack (World Bank)
• Gerardo Munck (University of Southern California)
• Brad Parks (Millennium Challenge Corporation)
• Gary Reid (World Bank)
• Joe Siegle (DAI)
• Fernando Straface (Inter-American Development Bank)

We are grateful for these individuals' feedback and encouragement. Global Integrity is responsible for all interpretation and analysis of the data and reporting as well as for any mistakes made.

Fieldwork and Country Selection

Argentina, Armenia, Azerbaijan, Benin, Brazil, Bulgaria, Cambodia, Democratic Republic of Congo, Egypt, Ethiopia, Georgia, Ghana, Guatemala, India, Indonesia, Israel, Kenya, Kyrgyz Republic, Lebanon, Liberia, Mexico, Montenegro, Mozambique, Nepal, Nicaragua, Nigeria, Pakistan, Philippines, Romania, Russia, Senegal, Serbia, Sierra Leone, South Africa, Sudan, Tajikistan, Tanzania, Uganda, the United States of America, Vietnam, the West Bank, Yemen, and Zimbabwe.

Our 2006 sample is by no means representative but provides interesting country coverage across several key variables.

While our ambition remains to cover all countries and regions of the world, the final set of countries for 2006 was chosen according to the following criteria:

1. Balance: Global Integrity seeks to ensure a basic geographic balance in its annual rounds of country assessments. For 2006, the breakdown is as follows:

   Sub-Saharan Africa: 15 (Benin, Democratic Republic of Congo, Ethiopia, Ghana, Kenya, Liberia, Mexico, Mozambique, Nigeria, Senegal, Sierra Leone, South Africa, Sudan, Tanzania, Uganda, and Zimbabwe)

   Southeast Asia: 4 (Cambodia, Indonesia, Philippines, Vietnam)

   Europe: 5 (Bulgaria, Montenegro, Romania, Russia, Serbia)

   Latin America: 5 (Argentina, Brazil, Guatemala, Mexico, Nicaragua)

   Middle East and North Africa: 5 (Egypt, Israel, Lebanon, the West Bank, Yemen)

   South and Central Asia: 8 (Armenia, Azerbaijan, Georgia, India, Kyrgyz Republic, Nepal, Pakistan, Tajikistan)

   North America: 1 (United States of America)

   In selecting countries we referred to Freedom House's annual Freedom in the World to ensure balanced coverage. Of this year's countries, 15 were ranked by Freedom House in 2006 as "Free," 17 were "Partly Free," and 11 were "Not Free."

2. Budget: The total number of countries covered in 2006, 43, was the maximum number of countries we could afford to assess thanks to the generous support of our current funders. Our goal remains to expand our coverage to true global coverage, at least on an alternating year basis.

3. Availability of experts: Perhaps the most important criteria that affects whether a country is selected is whether Global Integrity is able to recruit a sufficiently qualified research team in the country. We cannot, and do not, carry out country assessments where we lack confidence in our team. See additional details below on how country teams are recruited and selected.

4. Emphasis on large aid recipient countries: For 2006, our financial support from the World Bank included assessing a number of International Development Association countries (the classification for the Bank's very poorest client countries). This explains the larger selection of sub-Saharan African countries compared to other regions.

5. Emphasis on emerging markets: For 2006, Global Integrity also sought to increase its coverage of emerging markets in order to appeal to the private investment community. As such, our 2006 list of countries covered includes eleven of the 25 countries defined as emerging markets according to Morgan Stanley Capital International's (MSCI) July 2006 Emerging Markets Index.

6. Appeal to the international policy community: For 2006, we included 14 countries that were either "eligible" or "threshold" countries under the Millennium Challenge Corporation's selection criteria. Thirty eight of our 2006 countries are signatories to the U.N. Convention Against Corruption.

Country Teams

• A lead reporter who writes the Reporter's Notebook.
• A lead social scientist who compiles the initial scores for the Integrity Indicators.
• Two to six additional country readers (a mix of other in-country experts and out-of-country experts) who blindly review the raw data and reporting to provide additional perspectives and serve as part of the quality control mechanism.

The teams are coordinated from Washington via the Internet and phone. Until the public release of the country assessments, the social scientists, journalists, and peer reviewers are unaware of the identities of other members of the country team. This is done to maintain the independence of the individual responses and avoid a peer-influenced consensus.

Global Integrity's country assessments are produced through a coordinated effort of a large number of researchers and journalists, the vast majority of them located in-country. To identify appropriate social scientists, journalists, and peer reviewers in each country, Global Integrity actively recruits potential team members through informal partnerships with several well-placed international NGOs that work in the anti-corruption and good governance field. We participate in international conferences and seek referrals from colleagues with whom we already work in various countries. Global Integrity then independently verifies the expertise and independence of prospective team members. For 2006, we received around 550 CVs from individuals who expressed interest in serving on a country team in one of our three distinct capacities (journalist, lead researcher, or country reader). After a competitive internal selection process, the most qualified and available social scientists and journalists in each country were identified, invited, and sent a contract with specific instructions on the scope of the work. The total number of country team members for 2006 was more than 200 individuals.

The Country Reports

• Country Facts: Prepared by Global Integrity staff, these are drawn from a variety of publicly-available sources and provide basic political and economic background information on every country assessed.

• Corruption Timelines: Compiled by Global Integrity researchers in Washington, these are a unique political/historical timeline of significant corruption-related events at the national level. Designed as a quick reference resource, each timeline summarizes the main corruption-related events in the country from the early 1990s through present-day. Based largely on reliable English-language international and national media sources, the Corruption Timeline pulls together in an easy-to-read, chronological fashion what is essentially in the public domain about corruption in a particular country.

• The Reporter's Notebooks: Prepared by the lead in-country journalist, these are 1,250-word original essays by leading journalists on the culture of corruption and state of governance in a particular country. This hard-hitting, impressionistic essay provides a snapshot of corruption in day-to-day life as well as the recent history and context regarding the most high-profile corruption cases dominating that country's media.

• The Integrity Scorecard: Prepared by the lead in-country social scientist/researchers. Each country's Integrity Indicators are aggregated to generate the Global Integrity Index (see additional details below).

Details on the Integrity Scorecard

1. The existence of public integrity mechanisms, including laws and institutions, which promote public accountability and limit corruption.

2. The effectiveness of these mechanisms.

3. The access that citizens have to these mechanisms.

The Integrity Indicators are a unique instrument designed to provide a quantitative assessment of anti-corruption safeguards in a particular country. Carefully selected from a comprehensive review of the anti-corruption literature and other democratic governance sources, the Integrity Indicators are used to "score" the institutional framework that exists at the national level to promote public integrity and accountability and prevent abuses of power. For 2006, the core Integrity Indicators (290 including sub-indicators) are organized into six main governance categories and 23 sub-categories. They are:

1	Civil Society, Public Information and Media
1.1	Civil Society Organizations
1.2	Media
1.3	Public Access to Information
2	Elections
2.1	Voting & Citizen Participation
2.2	Election Integrity
2.3	Political Financing
3	Government Accountability
3.1	Executive Accountability
3.2	Legislative Accountability
3.3	Judicial Accountability
3.4	Budget Processes
4	Administration and Civil Service
4.1	Civil Service Regulations
4.2	Whistle-blowing Measures
4.3	Procurement
4.4	Privatization
5	Oversight and Regulatory Mechanisms
5.1	National Ombudsman
5.2	Supreme Audit Institution
5.3	Taxes and Customs
5.4	Financial Sector Regulation
5.5	Business Licensing and Regulation
6	Anti-Corruption Mechanisms and Rule of Law
6.1	Anti-Corruption Law
6.2	Anti-Corruption Agency
6.3	Rule of Law
6.4	Law Enforcement

Generating an Integrity Scorecard

There are two general types of indicators: "in law" and "in practice." All indicators, regardless of type, are scored on the same ordinal scale of 0 to 100 with zero being the worst possible score and 100 perfect.

"In law" indicators provide an objective assessment of whether certain legal codes, fundamental rights, government institutions, and regulations exist. These "de jure" indicators are scored with a simple "yes" or "no" with "yes" receiving a 100 score and "no" receiving a zero.

"In practice" indicators address "de facto" issues such as implementation, effectiveness enforcement, and citizen access. As these usually require a more informed and subjective assessment, these "in practice" indicators are scored along an ordinal scale of zero to 100 with possible scores at 0, 25, 50, 75 and 100.

Lead researchers are required to provide a reference to substantiate each of their scores. This may be an interview conducted with a knowledgeable individual, a website link to a relevant report, or the name of a specific law or institution, depending on the particular indicator. As an incredibly rich research tool we plan to expand and harmonize the types of references for each of the indicators in future iterations of the index. Lead researchers are also offered the opportunity to include additional comments to support their score and reference for a particular indicator. These are particularly useful in capturing the nuances of a particular situation, namely the "Yes, but…" phenomenon which is often the reality in undertaking this type of research.

Personality, language, and culture can all affect the interpretation of a particular indicator and the score assigned to it. To minimize this effect and maximize inter-coder reliability, Global Integrity, in consultation with the informal methodology advisory committee, developed a master codebook for scoring the 2006 Integrity Indicators. The Codebook anchors every single indicator and sub-indicator to a predefined set of criteria. In essence, the scoring criteria guide the lead researcher by suggesting, "If you see X on the ground, score this indicator in the following way." For binary yes/no "in law" indicators, scoring criteria are provided for both "yes (100)" and "no (0)" responses. For "in practice" indicators, scoring criteria are defined for each of the 100, 50, and 0 scores with 25 and 75 deliberately left undefined to serve as in between scoring options.

In summary, a given indicator or sub-indicator has the following elements:

• Indicator question, provided by Global Integrity
• Indicator scoring criteria/coding, provided by Global Integrity
• Indicator score (either yes (100)/no (0) or ordinal scale of 0 - 100 with steps at 25, 50, and 75), assigned by the lead researcher based on
  • References, provided by the lead researcher
  • Comments (optional), provided by the lead researcher
  • Reader review comments (optional), as provided through a blind peer review process (See more on the peer review process below.)

The Integrity Scorecard: A powerful tool for analysis

Generating the Global Integrity Index: A Tool for Cross-Country Analysis

The Global Integrity Index groups countries into five performance "tiers" generated from the scores assigned to the individual integrity indicators:

• Very strong (90+)
• Strong (80+)
• Moderate (70+)
• Weak (60+)
• Very Weak (&#lt; 60)

For the purpose of producing the Global Integrity Index, a simple aggregation method is used that parallels the process for generating indicator, sub-category, and category scores for each country. Original indicator and sub-indicator values are assigned by the lead social scientist for the country (those scores are adjusted following the peer review process; see more below). Each indicator score is then averaged within its parent subcategory, which produces a subcategory score. The subcategory score is in turn averaged with the other subcategory scores in a parent category. Category scores are averaged to produce a country score which then falls into one of the above five groupings.

Because some aspects of governance integrity are harder to measure definitively, some categories require a more complex matrix of questions than others. Thus, the categories are equally valued, even if some categories are derived from a more lengthy series of questions than others. Similarly, the subcategories are equally valued within their parent category. The same comparison cannot be made between subcategories in different parent categories.

IIn other words, each score (sub-indicators, indicator, and so on) is equally weighted with its peers addressing the same topic. However, indicators from different categories are not equally weighted. Our approach of using equally valued concepts and adding subordinate elements as needed has produced score weightings that reflect the six main conceptual categories.

Peer Review Process

Individually contracted and carefully vetted readers, selected for their independence and expertise in particular countries, are asked to blindly review both the raw Reporter's Notebook and the raw Integrity Indicators using the same password protected website used to file the original data and reporting from the field. The blind review process ensures that readers are unrestrained in their commentary, which most likely adds to frankness when commenting on the draft country report. Reader comments are used to interpret – and in some cases adjust – scores that reviewers identify as containing errors, bias, or out-of-date information. Score adjustments follow certain rules and generally require repetition (i.e. similar comments from several reviewers) or solid referencing of a factual dispute. See further details on this below (Final Index Scores).

In reviewing the Reporter's Notebook for the country, readers are asked to consider the following:

• Is the Reporter's Notebook factually accurate?
• Is the Reporter's Notebook fair? Is anything misrepresented or unclear?
• Are there any significant events or developments that were not addressed?

Reader comments on the Reporter's Notebook are captured in narrative, paragraph style and are published anonymously alongside the final Reporter's Notebook.

For the Integrity Indicators, readers are asked to consider the following:

• Is the particular Indicator or Sub-indicator scored by the lead researcher factually accurate?
• Are there any significant events or developments that were not addressed?
• Does the Indicator or Sub-indicator offer a fair and balanced view of the anti-corruption environment?
• Is the scoring consistent within the entire set or sub-set of Integrity Indicators?
• Is the scoring controversial or widely accepted? Is controversial scoring sufficiently sourced?
• Are the sources used reliable and reputable?

The reader process for the data scorecard, like for that of the Reporter's Notebook, does not assign direct attribution of reader comments. This ensures that readers are unrestrained in their commentary. Reader comments on the country's data scorecard are included alongside the final scorecard and play an important role in final scoring adjustments prior to publication.

For 2006, Global Integrity retained the services of approximately 130 readers for the 43 country assessments, with some readers reviewing a number of countries.

Final Index Scores

While Global Integrity makes every attempt to produce credible information we welcome all feedback on the veracity and accuracy of our data. Please email Global Integrity with specific comments on indicator scores that you may not agree with, particularly with regard to factual accuracy.