Whether the global snapshot provided in Freedom House’s annual Freedom in the World assessment characterizes the 2010 results as a “declining trend” or a “stagnation” in global democracy, the report’s outlook is certainly bleak.
Good news was hard to find in Freedom of the World 2010. Arch Puddington, Freedom House’s director of research, spoke Tuesday in great detail on his organization’s fears that in the last four years, global democracy has downgraded from a point of stagnation to an outright decline. For example, the overview essay disparagingly reveals that “overall, the Middle East and North Africa region suffered a number of significant setbacks, and these were often centered in countries that had produced some evidence of reformist intentions in the recent past.” At the January 12 release event, Puddington added to this regional analysis, characterizing the Caucus Region as having “nearly as poor of a record” as found in the Arab world and stating that Latin America is a mild “success story,” but with “serious problems” remaining.
Freedom of the World: 2010 points to incidents on every continent where human rights defenders, civil society organizations and journalists have been under attack: China, Russia, Iran, the UK, Nicaragua, and Ethiopia are just a few examples. Their analysis finds violent repression especially pronounced in countries where civil society has taken on an increasingly political mission, filling the robes typically worn by opposition parties.
Synthesizing the findings of an assessment of this great breadth into key findings and regional analyses is a difficult task. One must balance the country-specific contexts with international standards, and decide whether specific events (like the CSO oppression discussed above) are isolated incidents or part of a greater “trend”.
Thomas Carothers of the Carnegie Institution for International Peace warned readers of Freedom in the World to think critically about the index’s uses of labels like “decline”, “stagnation”, “setbacks” and “reversals.” Carothers believes that the report has at times “overstated” both specific events and preexisting conditions by labeling them as “trends.” In his critique, he pointed to multiple examples of what he found to be break-downs in the standards of the assessment where the label of “trend” was too liberally applied. For instance, the “setbacks” identified in the Middle East may be more of a stagnation of the previous impetus towards reform than an actual retrenchment in democracy. True, the political violence surrounding the Iranian elections (as highlighted in the report) can in no way be counted as progress towards democracy, but can this incident be applied regionally, attesting to a greater “trend”?
Carothers’ points get to the heart of one of Global Integrity’s greatest concerns about the difficulty of measuring fuzzy concepts like “democracy”. Labels can be misleading—for instance, what does “rule of law” or “corruption” really mean? The definition can widely differ based on the assessment and on the context in which it is being applied.
Getting back to the Freedom in the World assessment, the methodology for the final classification of a country as “free”, “partially free” or “not free” (big, broad labels!) has never been made explicitly clear. Without this criterion for assigning of scores, readers are left to wonder (as Tom Carothers does) why the political violence in Iraq has prevented it from reaching the elusive “free” score, while Mexico is labeled “free” despite its high levels of violence and political influence from drug cartels.
To briefly contrast with Global Integrity’s approach, while we do use labels like “strong” and “weak” to describe complex systems, this is always based on hundreds of discrete indicators which can clarify what we think is happening. These transparent and dense data are at times less conducive to easy explanations and can produce some counter-intuitive results. But we’re not in the business of confirming the conventional wisdom. Sometimes complex systems require complex explanations.
Also: if we title a dataset “2010”, you can be sure it won’t be published in January of that year.
But maybe it is not Freedom House’s intention to be detail-driven but rather to provide a user-friendly, easily digested, top-level snapshot? The report is visually appealing with a colorful map of the world. Country-level analysis taking the back seat to an overarching global summary. Why use four colors, when three will do? As such, the assessment maintains a high-profile and importance in providing an overview of democracy’s growing global reach.
However, to echo Carother’s comments, Freedom House users, especially policy-makers, should probe deeper than the Freedom of the World’s summary and maps, demanding that the “trends” identified by the report are supported by additional current and historical research.
For more on metrics and the difficult task of measuring fuzzy concepts like “democracy”, read Global Integrity’s A Users’ Guide to Measuring Corruption.
— Norah Mallaney
