Measuring Human Rights Kate Raworth* Claims of unjust treatment and demands for policy reform are increasingly being couched in the language of human rights. In order to define human rights more precisely, various efforts are under way to create measurements, or human rights indicators. Creating these instruments for measuring human rights is not just a technical exercise: indicators have long played important roles in social policy evaluation and are central to turning human rights concepts into useful policy tools. To assess human rights effectively, however, indicators must hold the state accountable for its policies, help to guide and improve policy, and be sensitive to local contexts without sacrificing the commitment to the universality of rights. Can indicators be developed to do this? The answer will largely determine the success of human rights claims in moving beyond rhetoric and changing real social and political conditions. Current proposals for indicators lack clarity both about what is to be assessed and how it will be measured. I propose that a far richer conceptual framework be set out that focuses on assessing state obligations, is sensitive to context-specific issues, and does not ignore the inherent tensions in the policy-making process. Whether or not such indicators can be put into practice will only become clear through case studies—which currently are sorely lacking. Why Human Rights Indicators Are Needed Indicators are variables that convey information—either quantitative or qualitative— and are consistently measurable.1 Quantitative indicators are created by counting events or objects; for example, the percentage of school-aged children enrolled in school or the amount of government spending per pupil per year. Qualitative indicators are created through assessments made by trained observers of characteristics (such as the standard of teaching in a school) that can be rated on a scale (such as “excellent/good/poor” or “from 1 to 10”). * This paper has benefited tremendously from many discussions with Christian Barry and also from valuable comments by Paul Hunt and Maria Green. 1 United Nations Office of the High Commissioner for Human Rights, The Dignity Measure: Selected Human Rights Indicators (Geneva: United Nations, 1999). 112 Kate Raworth Indicators, particularly quantitative ones, have long played important roles in the analysis of development policy. It often takes a shocking statistic in the news to draw public attention to a neglected issue such as gender discrimination in the workplace or homelessness. Indicators can motivate policy change by revealing the ill effects of current practices; the introduction of school fees in a poor country, for instance, can be countered by data that show a rapid decline in the number of children attending primary school. They can also persuade the unconvinced where opinion alone cannot. For example, many claim that African Americans are treated unjustly by the legal system—a claim that is hard to dismiss when backed up by statistics from Amnesty International showing that murderers of white Americans are eight times more likely to receive the death penalty than murderers of African Americans. One of the most widely cited indicators of development is the United Nations Development Programme’s Human Development Index (HDI).2 Calculated annually for 174 countries, currently ranked from Canada at the top to Sierra Leone at the bottom, the HDI combines four simple indicators—life expectancy at birth, the adult literacy rate, school enrollment rates, and the average annual income per person—to create a composite index that gives a basic measure of the state of human development in a country on a scale of 0 to 1. In Botswana in 1998, for example, life expectancy at birth was 46 years, 76 percent of adults were literate, 71 percent of school-aged children were enrolled in school, and average annual income per person was $6,103 (measured in international dollars for purchasing power parity). The combination of these four indicators gave Botswana a value of 0.593 on the Human Development Index and ranked the country 122nd out of 174. In contrast, Greece ranked 25th with an HDI value of 0.875 because life expectancy was 78 years, 95 percent of adults were literate, 81 percent of school-aged children were enrolled, and average annual income per person was $13,943.3 When the HDI is launched worldwide each year, it gains the attention of policymakers and development advocates alike because it provides the simplest and most widely recognized measure of the economic and social condition of people’s lives around the world. Some countries have even disaggregated the index at the national level: instead of comparing country against country, they compare district against district. In Brazil the Human Development Index has been disaggregated for all 4,500 municipalities in the country’s twenty-seven states, providing detailed data on education, health, and incomes throughout the country. The results caught the attention of local and national media, igniting debate when neighboring communities had dis- United Nations Development Programme, Human Development Report 2000 (London: Oxford University Press, 2000), p. 157. 3 Ibid., pp. 157–59. 2 MEASURING HUMAN RIGHTS parate outcomes. The data also helped to shape policy. Nationally, the government used the data as the basis for improving targeted assistance for poverty alleviation throughout the country. At the state level, some local governments used the data as the basis for redistributing government spending toward the municipalities with the worst outcomes.4 These valuable uses of indicators for development analysis indicate the potential role they could play in assessing the realization of human rights. Tools of this kind are very much in demand because human rights principles are increasingly being advocated as a framework for public policy dialogue and assessments, especially under international human rights law. Since the 1948 Universal Declaration of Human Rights, the United Nations has been promoting human rights with at least nominal success. Between 1990 and 2000, the proportion of UN member countries that had ratified all of the six major treaties of international human rights law rose from 10 percent to just over 50 percent. This increase in the nominal commitment to international human rights law has led to greater focus on developing indicators that would help monitor the implementation of these commitments. Such instruments could show the status of a particular human rights situation, reveal whether a situation is getting better or worse owing to a policy change, and guide the formation of better policy. If we were to claim, for example, that the right to education in a certain country was not being realized, then we would need to provide evidence to substantiate that claim. Indicators would be essential to the task. Indicators may be needed for assessing the extent to which human rights have been realized, but is it conceptually, politically, and practically possible to create them? It is not obvious that the rich normative concepts of human rights can be turned into policy evaluation tools that are adequately responsive to the complexities of policy-making situations. The question of whether or not adequate indicators can be created is, however, central to determining whether or not the language and commitments of human rights can ever become a workable framework for making policy assessments. The Shortcomings of Qualitative Indicators The attempt to use indicators to assess human rights is not new. Indices have been constructed since the 1950s, but over the past ten years the approach to their construction and the kinds of indicators desired have shifted significantly. Early human 4 Ibid., p. 96. 113 114 Kate Raworth rights indicators consisted of qualitative ratings, scoring a particular country as 5 out of 10 for freedom of the press, for instance, or 7 out of 10 for freedom of association.5 These ratings were made by human rights experts on the basis of all the information to which they had access, such as newspaper reports, on-the-ground sources, U.S. State Department reports, and the Amnesty International Annual Report. In the politicized realm of human rights assessments, however, this approach to creating qualitative indicators suffers from what can be termed the “who, why, so what?” problem. First: who is the rater? The identity of the individuals giving the ratings is inevitably open to questioning, both of their subjectivity and of the incompleteness and partiality of the sources of information to which they had access.6 As a result, even in cases in which the rating does give a somewhat accurate assessment of a country’s situation, the judge’s subjectivity could be used by the aggrieved government to divert attention from the judgment made and instead to attack the alleged political bias of the assessor. Second: why the rating? A score of, say, 7 out of 10 for freedom of the press in a particular country is opaque in that it reveals nothing of the set of criteria used for creating that score and nothing of the particular national circumstances that merited it. This prompts the final question: so what? If indicators are intended to be a policy tool, such ratings give no indication of how to improve the situation. Without revealing the underlying criteria and the reasons for the rating, the indicator is of little use for policy planning. Statistically, there are also difficulties. Qualitative ratings are often presented in numerical form, but they do not possess the essential characteristics of statistical data: reliability (different people will come up with consistent results) and validity (being based on identifiable criteria that measure what they are intended to measure).7 As a result, such ratings cannot validly be used for many of the operations that are in fact routinely performed on them, such as collating the ratings of more than one individual to create a composite result or using the ratings as variables in regression analysis. Sophisticated techniques are beginning to be applied more commonly to qualitative information to tackle these statistical limitations, but this manipulation is the exception rather than the norm and still will not make qualitative information as politically acceptable as quantitative. Political sensitivity to the need to provide less subjective and opaque measures has shifted the focus of current efforts toward the creation of quantitative, statistical 5 Kenneth A. Bollen, “Political Rights and Political Liberties in Nations: An Evaluation of Human Rights Measures, 1950 to 1984,” Human Rights Quarterly 8 (Special Issue 1986), p. 577. 6 Ibid., p. 583. 7 UNDP, Human Development Report 2000, p. 90. MEASURING HUMAN RIGHTS indicators because the data can be verified and, in the case of a good indicator, consistently measured across time. Of course this approach, too, has its limitations: quantitative data can measure only countable events and objects, yet there are many uncountable dimensions to human rights. This is an important caveat, a reminder that, as Einstein noted, “not everything that counts can be counted, and not everything that can be counted counts.” With the increasing involvement of professional statisticians in gathering data on human rights, however, it is fortunately the case that much more can be measured quantitatively than was previously assumed. The Conceptual Commitments of Human Rights What are the main elements in human rights concepts that have particular implications for developing indicators? There are many competing and widely divergent conceptions of human rights, yet they share several core features. First, almost all of them endorse the idea that social rules should be evaluated in terms of the extent to which they enhance or diminish the personal freedom and dignity of participants within a social system. Human rights are realized within a social system when people within it enjoy certain freedoms and when there are social arrangements in place that are sufficient to protect them against standard threats to their enjoyment of those freedoms. Establishing what the scope of those freedoms should be calls for specifying the object of each right—specifying what the right is a right to—be it freedom of association, an adequate wage, primary education, or freedom from arbitrary arrest. A second important feature shared by many conceptions of rights is that rights are correlated with duties in that they entail claims on individuals or institutions (collectivities of individuals) to secure these rights for people. Moreover, even in situations in which these rights cannot immediately be secured for all people—for example, owing to resource constraints—individuals or institutions are still duty bound to shape their conduct so as best to promote the realization of these rights. Human rights can thus be understood as claims on the behavior of individuals and on the design of social arrangements. In this way, rights discourse emphasizes the idea of accountability and responsibility for the negative consequences of social structures and requires identifying the primary duty bearers and articulating what duties they are held under. Arising from these two features of human rights discourse are two concurrent but very different interests that we may have in assessing rights. We care about the condition of people’s lives—whether or not they enjoy the objects of their 115 116 Kate Raworth rights—but we also care about whether persons who are held under duties are meeting them. The former interest can be seen as an enjoyment approach to assessment and the latter as an obligations approach. These two concurrent interests are compatible, but obviously different. It is quite conceivable in a particular social system that, for example, not all people have enough to eat due to natural resource constraints, yet all duty-bound persons have met their obligations to provide the maximum possible amount of food fairly to all. In this case, by the enjoyment approach to assessment the right to food has not been realized, yet by the obligations approach all duties have been met. Given this potential divergence between an assessment of the enjoyment of rights and an assessment of the obligations of rights, it will clearly be important to distinguish between the two approaches when developing indicators. A third important feature shared by many conceptions of human rights is universality. Although we recognize that peoples of diverse cultures can, to some extent, aim to realize divergent ideals of justice in different social contexts, we also recognize that all individual persons have interests of sufficient importance to constrain the social order that can be imposed upon them regardless of its legitimacy within their culture. Though it may not be possible to agree precisely on what makes up the human good, we may still be able to agree that that it will include certain vital elements—for example, basic political freedoms, nutrition, shelter, education, and social interaction—which just social orders should secure for all. This commitment to universality expresses itself through two desires, corresponding to the enjoyment and obligation approaches to rights. First is the desire to specify universally those freedoms in which all human beings have a vital interest, such as a primary education for all children or the freedom of association for all people. Second is the desire to specify universally those obligations of conduct that persons are held under, such as the duty not to prohibit any child from attending school, or the obligation of the education authorities to provide adequate buildings and teaching staff. Again, the distinction between these two ways in which universality is reflected in human rights will be important when developing indicators. Current Attempts to Develop Human Rights Indicators Current efforts are aimed at creating statistical indicators that could be used as policy tools to monitor the commitments made under international human right law. The particular conception of human rights under international law shares the cen- MEASURING HUMAN RIGHTS tral features of rights set out above, but with a narrower emphasis on the conduct of official institutions, that is, the state. Although the framework of international law is only one particular conception of human rights, and its focus on state conduct alone is considered by many to be problematic, the issues that arise in the attempt to develop indicators under this framework are also relevant to alternative conceptions of rights. There are several important ongoing efforts, including projects within the United Nations High Commission of Human Rights, the Organization for Economic Cooperation and Development (OECD), and several national think tanks such as the Danish Center for Human Rights.8 The various tentative proposals emerging from these efforts, however, often lack clarity as to what specifically should be measured and how core characteristics of human rights should be reflected in the way indicators are constructed and used. Among economic and social rights, the right to education is a common starting point for developing indicators. One broadly representative example of the approach being taken is a proposed set of indicators for the right to education arising out of a workshop held by the World University Service–International.9 The workshop participants agreed that the desirable result was to propose “a set of common indicators, agreed upon by all treaty bodies and agreed upon by the specialized UN agencies, in order to have a common language currency in which to discuss the right to education.” Articles 13 and 14 of the International Covenant on Economic, Social, and Cultural Rights were taken as the primary source from which to derive indicators. The workshop aimed to produce ten universally applicable indicators that were quantitative, simple (as opposed to composite, like the Human Development Index), comparable across countries and time, and reliable in that different people would expect to get the same results if they collected data. For practical reasons, indicators should also be affordable and draw upon good data already available. The resulting proposal—intended as a provisional, not conclusive, list— included commonly cited indicators such as the adult literacy rate; the net school enrollment rate; the required level of teacher training, percentage of teachers who have reached it, and composition of teaching staff; the percentage of government expenditure spent on education and expenditure per pupil; and the cost of education for a family, including direct (fees) and indirect costs per pupil. Each of these proposed 8 The Danish Centre for Human Rights, Human Rights Indicators 2000: Country Data and Methodology (Copenhagen: The Danish Centre for Human Rights, 2000). 9 World University Service–International, “Report of the Workshop on Indicators to Monitor the Progressive Realisation of the Right to Education” (workshop organized by World University Service–International, Geneva, May 1999). 117 118 Kate Raworth indicators is potentially interesting as part of a human rights assessment, but what the proposal does not explain is how these indicators are to be used in an assessment. As a result, it is unclear what conclusions or implications about the right to education could be drawn from the data. Yet if indicators are to be used as tools for guiding and evaluating policy, clarity about their use in assessment is critical. Three deeper conceptual issues need to be addressed. First, what should human rights indicators for policy evaluation be measuring? There is a lack of conceptual clarity as to what indicators should be assessing— is it the extent to which people have the objects of their rights, or the extent to which the conduct of officials meets the obligations they are held under? Both are valuable interests, and indicators could be created for each, but for the purpose of creating tools for guiding and evaluating policy, the obligations approach provides the more relevant focus. Second, in what way should the principle of the universality of human rights be reflected in indicators? The claim that human rights are universal does indeed require that this principle be reflected in their measurement, but how? Current approaches to developing indicators are embedding the principle at the wrong conceptual level by assuming that indicators should be universally applicable across countries. Universality should, in fact, be reflected in the principles underlying the obligations of state conduct. Instead of being universally applicable, indicators should be context-specific so they can reflect the ways that those universal principles of conduct are manifested in culturally, economically, and politically different contexts. Third, what further work is needed to move forward? Given the importance of the universal obligations of conduct, there is a lack of clarity about how they should be weighed against each other. Particularly lacking is an appreciation of the repeated conflicts of interest occurring between resource needs, social groups, and competing freedoms that characterize the dilemmas of the public policy decision-making process. This is where additional conceptual work needs to be done. What Indicators Should Be Measuring Attempts to develop indicators are understandably pulled in two directions because of the two concurrent but very different interests that people commonly have with respect to human rights. As discussed above, we care about the condition of people’s lives and their enjoyment of freedoms, but we also care about establishing whether MEASURING HUMAN RIGHTS official agents, either individually or collectively, are doing enough to meet their obligations in securing those freedoms. Both of these approaches to developing indicators are compatible with the interests of international human rights law, which aims to promote the realization of rights and also to monitor the conduct of governments in that process. Current proposals to develop indicators, however, are neither implicitly nor explicitly clear about which assessment approach they are using. Most proposals appear to be focused on creating indicators for assessing the enjoyment of rights; few are specifically designed to assess obligations. Which approach is in fact appropriate for which goals? If the aim were simply to know the condition of people’s lives in each country, then the enjoyment approach would be ideal. The data from each country could be compared directly to the ideal outcome—for example, comparing a primary-school enrollment rate of 72 percent to the 100 percent ideal that would mean all children had secured their right to a primary education. Such an assessment would in many cases use indicators in a similar way to assessments of human development (such as the Human Development Index), looking at average life expectancy at birth, adult literacy rates, maternal mortality rates, and school enrollment rates. The outcomes of such assessments would inevitably be closely related to the country’s physical and human capital resources and its historic economic and political situation; hence wealthy countries would tend to be assessed as doing well and poor countries as doing badly. A politically feasible approach to developing indicators, however, must allow for the possibility that a government in a historically poor and underdeveloped country could introduce a set of legislative, budgetary, and institutional policies and that its conduct would be assessed as a fulfillment of all its obligations under human rights law—whatever the prevailing level of development of the country; whatever the prevailing condition of people’s lives. Assessing human rights for the purpose of guiding and evaluating policy must, therefore, be something other than simply assessing people’s enjoyment of the objects of their rights. The assessment needs to reveal the conduct of the government, give an indication of what action is required (avoiding the why and so what questions discussed above), and be responsive to changes in that conduct. The obligations approach is therefore the more appropriate: while focusing on the extent to which states meet their obligations of conduct, it allows for the very different contexts within which they face these challenges. Much emphasis inevitably falls on developing a sufficiently rich conception of what those obligations are. Within international human rights law, the most basic elements of the framework of principles underpinning state obliga- 119 120 Kate Raworth tions have been broadly set out. First are the obligations of action for the state to respect, protect, and fulfill rights.10 Respecting rights requires the state to refrain from interfering directly with people’s pursuit of their rights, whether the interference is through committing torture or arbitrary arrest, enacting illegal forced housing evictions, or imposing food blockades. Protecting rights requires the state to prevent violations by third parties, whether by ensuring that private employers comply with basic labor standards, preventing monopolistic ownership of the media, preventing parents from withholding their children from school, or regulating and forcing abatement of harmful pollution from private industry. Fulfilling rights requires the state to take action—be it legislative, budgetary, judicial, or other—to provide the best possible set of policies that pursue the realization of the right. In addition to these obligations of action are four commonly recognized obligations of process that must be met in official conduct: nondiscrimination, adequate progress, participation, and effective remedy. The principle of nondiscrimination obliges the state to act toward all individuals with respect to their rights without distinction of any kind such as ethnicity, gender, language, religion, or political opinion. The principle of adequate progress obliges states to take steps toward meeting their obligations within a reasonably short time after entering legal commitments and with steps that are deliberate, concrete, and targeted as clearly as possible. The principle of participation obliges states to encourage popular participation in all spheres of development and in achieving the realization of rights. Finally, the principle of effective remedy obliges states to ensure that all persons within its jurisdiction are provided an effective remedy for conduct that violates their rights. These obligations of action and obligations of process together form the foundation under international law of a framework of principles for assessing state conduct.11 What are the implications of these principles for the kinds of indicators that are required? In fact, an obligations approach would use many of the same data as an enjoyment approach; the difference lies in how they would be used. Instead of comparing a country’s performance against a specified human rights goal (say 72 percent school enrollment against the goal of 100 percent), the data would be used to assess the adequacy of the state’s conduct with respect to its various obligations to ensure the realization of the right. 10 This framework of respect, protect, and fulfill was first set out by Ashbjorn Eide in the context of the right to food, but has become the common basis of state obligations as applied to all rights. Asbjorn Eide, “The Human Right to Adequate Food and Freedom from Hunger,” in The Right to Food: In Theory and Practice (Rome: Food and Agricultural Organisation, 1998). 11 See UNDP, Human Development Report 2000, chap. 5, pp. 89–111. MEASURING HUMAN RIGHTS A simple example illustrates this distinction. Imagine a country in which 87 percent of children are enrolled in secondary school. What does this statistic reveal about the right of a child to education? Certainly, from the perspective of the enjoyment approach, it reveals that the right is not being realized because not all children are attending secondary school. But from the perspective of the obligations approach, what implications can be drawn regarding the conduct and accountability of policymakers and state officials? From this information alone, none. We might assess the state’s conduct very differently on the basis of this enrollment rate depending on whether the statistic came from, say, a low-income country in sub-Saharan Africa or from a wealthy industrial country in Europe. To better substantiate this requires more detailed data that reveal the reasons for this enrollment rate, and, by implication, whether or not policymakers and officials are accountable for the shortfall. If additional data show that only 77 percent of girls are enrolled compared to 97 percent of boys, then much of the problem is likely due to discrimination. Surveys of parental attitudes could further reveal whether or not it is parents who are discounting the importance of girls’ education (hence the state would be failing to protect the right of girls from this prejudice), and surveys of school facilities could reveal whether the discrepancy was due to something as simple as a lack of toilet facilities for girls, placing the accountability with the school authorities for failing to provide the facilities needed to fulfill girls’ right of access to an education without discrimination. Perhaps there is no discrimination but rather, schools lack resources and hence the ability to attract and retain students. Then the question is whether the government is giving enough priority to fulfilling the right to education. Analysis of the budgetary allocations to education as against alternative spending priorities would reveal whether it was a lack of priority or a lack of overall resources that resulted in low spending on education. Likewise, the direction of change over time would reveal whether the obligation to make adequate progress was being met. A fallback in enrollments from 95 percent to 87 percent over the previous five years would be assessed very differently from an increase in enrollments from 55 percent to 87 percent over the same period. This example shows that instead of using the data simply to assess the level achieved—here 87 percent—against the enjoyment approach goal of 100 percent, the focus should be on using more detailed data to reveal whether enrollments are 87 percent because some obligations have not been met or because, despite all obligations having been met, a shortage of resources and capacity make it currently impossible for this right to be realized. 121 122 Kate Raworth Is this distinction between the two approaches really important? Yes. It makes a tremendous political difference, particularly to developing countries. When a new government comes in to a historically underdeveloped situation and implements an impressive set of policies to improve the human rights outlook, an obligations approach can recognize that achievement. It means that even a government in a country with a secondary-school enrollment rate of 50 percent could possibly be given a positive assessment of its conduct with respect to the right to education. Likewise, a wealthy country with more than 95 percent enrollment could possibly be given a negative assessment if it were shown that the state was clearly failing in its obligations toward the final 5 percent of children. Since the language of human rights is being advocated as a framework for policy dialogue particularly in developing countries, the acceptance of this framework within those countries will be critical in deciding whether or not the framework is politically feasible. The importance of making assessments that are sensitive to context, as discussed above, prompts a second conceptual exploration of the approach being taken to indicators: a re-evaluation of the assumption that indicators should be universally applicable. Acknowledging Local Contexts without Sacrificing Universal Principles It is widely assumed at the international level that human rights indicators will be internationally agreed upon and universally applicable across countries. At the International Association for Official Statistics Annual Conference on Human Rights and Statistics in September 2000, for example, Thomas Hammarberg appealed to the international community’s “capacity to agree on relevant human rights indicators” as the necessary will needed to move forward.12 The intuitive appeal of creating such universally applicable indicators is both conceptually and politically understandable. Conceptually, it is assumed that the universality of human rights entails the universal applicability of indicators for policy evaluation. This assumption is too quick, however, as will be discussed below. Politically, there would be several advantages to having universally applicable indicators. First, universally applicable indicators would be a high-profile Thomas Hammarberg, “Searching for the Truth: The Need to Monitor Human Rights with Relevant and Reliable Means” (paper presented at the Statistics, Development, and Human Rights Conference of the International Association for Official Statistics, Montreux, Switzerland, September 4–8, 2000). 12 MEASURING HUMAN RIGHTS reinforcement of the claim that those human rights set out in international law are indeed universal, whatever the cultural context—helping to counter claims that cultural relativity prevents universal legal obligations from being imposed on all countries. Second, the reporting system of international human rights treaties requires countries to submit reports documenting the status of rights and the progress they have made in realizing them, and quantitative data play an important role in specifying what information is required. The quality and extent of information given in state party reports is currently highly variable, and as a part of strengthening and standardizing the reporting system, universally applicable indicators would be a desirable way of increasing the quality of information presented in every report. In addition, setting fixed data categories could pressure recalcitrant governments to provide data that they would not otherwise collect or make available. The more that providing a particular piece of data were seen as an international norm, the more political pressure there would be to conform—and such pressure is extremely useful in ensuring greater provision of and access to information for citizens about the conduct of their own government. Finally, universal indicators would enable crosscountry comparisons, which are useful for evaluating the effects of different policy stances in otherwise similar countries. Given this intuitive appeal of creating universal indicators, the reasons for the bias toward creating indicators for assessing the enjoyment approach to rights—instead of the obligations approach—become more evident. In practical terms, it is easier to specify universally desirable goals—objects of rights—for the human condition that are relevant across country contexts, such as a long life expectancy or compulsory primary education for all children, than it is to specify universally the obligations of official conduct and institutional arrangements that would lead to these outcomes. Furthermore, the enjoyment of many objects of rights can be assigned specific desirable outcomes, such as 100 percent for primary-school enrollments, or zero cases of torture, and for many it is clear that either a higher or lower score is better or worse—such as higher school enrollments and fewer incidences of torture both being better outcomes in all contexts. Statistics showing the extent to which people are enjoying the objects of their rights are simple to grasp (although this focus may be the reason why many cynics dismiss economic and social rights as “aspirations”): it enables international comparisons of the condition of people’s lives in different countries and creates easy-to-communicate advocacy messages. Despite the appeal of creating universally applicable indicators, however, such assumptions are misguided. The claim that human rights are universal does indeed 123 124 Kate Raworth entail that the principle of universality should be reflected in their measurement but, for the purposes of evaluating policy, it should be reflected not in the specific indicators chosen but in the principles underlying state obligations—the universal obligations to respect, protect, and fulfill rights, and to do so with nondiscrimination, adequate progress, participation, and effective remedy. If these principles form the foundation of the obligations of state conduct then indicators to measure the extent to which they are being met cannot be universal, because it is possible to specify universally neither which indicators should be used nor what the value of any particular indicator should be. First, realizing rights requires different policy mixes in each country, hence different conduct by states to meet their obligations. The transformation of policy measures such as government spending, institutional arrangements, and legislation into desired human conditions such as low infant mortality rates and high school enrollment rates is not deterministic. Not only does the economic, social, and political environment differ from one country to the next, requiring a different policy mix in each case, but even in a given context in which all these environmental variables are known, there is no universal policy equation that can be applied with the certainty of producing the optimal outcome. More specifically, the conduct required by the state to meet its obligations will differ across contexts—for example, the state obligation to protect the right to health will demand conduct from the state that is contingent on the kinds of threats presented by third parties. In one country the threat may be from a private industry polluting the water sources of communities surrounding its factories. In another country the threat may be from a systemic social culture of domestic violence toward women. In yet another the threat may be from social norms and interactions that cause a particularly rapid spread of HIV/AIDS. Different indicators would be needed in each case to identify the source of the threat to health and to assess the adequacy of the state’s conduct to protect individuals from that threat. Second, prespecified indicators may be inappropriate to reflect particular national circumstances, resulting in errors of omission, such as the failure to identify a violation of an obligation where a violation has actually occurred, and errors of commission, such as the false identification of certain conduct to be a violation of an obligation. For example, assessing the obligation to respect the right of freedom of speech might include an assessment of whether or not the state has interfered with people’s choice to speak in opposition to the state: one indicator proposed might be the number of political demonstrations held by groups other than the ruling political party. A low number of such demonstra- MEASURING HUMAN RIGHTS tions could, however, be either because free speech is respected but general political satisfaction causes few protest rallies, or because a politically repressive regime makes such protest politically impossible. It is easy to imagine the number of additional indicators that would be required as supplements to this one to ensure that such an error of omission was not being committed—but such detailed data requirements would make such prespecification conceptually impossible. Third, even for those indicators that are relevant across contexts and for which it appears to hold that more is better, such as government spending on secondary education as a percentage of national income, the assumption does not hold across countries or within countries over time. In a global context in which the average annual national income per person in a country ranges from $500 in the poorest countries to $30,000 in the richest (measured in international dollars for purchasing power parity), the range of possible policy measures is obviously tremendously different across countries. Different social arrangements and levels of national income require different proportions of government spending on education: in relatively wealthy countries where much education is provided through the private sector and can adequately be paid for directly by households, government spending may be appropriately lower than in other countries that are less able to rely on the private provision of secondary education. Despite spending proportionately less public money on secondary education, the former countries could feasibly have a higher secondary-school enrollment rate than the latter. An illustration of this type is given by the Third International Mathematics and Science Study, which conducted standardized tests on secondary-school students in many OECD and other high-income countries. In 1993, although state spending per pupil on secondary education in South Korea was less than one third of that in Germany, Denmark, and the United States, the performance of students from South Korea was ranked second overall, compared to students in the other three countries who ranked twenty-third, twenty-seventh, and twenty-eighth respectively.13 Of course there are many reasons why students in one country do well and in another badly, but the example illustrates the lack of even the most basic certainty of the transformation of financial policy into educational achievements across countries. Fourth, there are significant practical limitations to data collection. If a complete set of indicators was prespecified to cover all possible contexts, the data requirements would be logistically and financially impossible to fulfill. Much data would be irrelevant in a particular country, and although every additional piece of data would bring additional information—and more information is seen to be a good 13 “Who’s Top?” The Economist, March 29,1997. 125 126 Kate Raworth thing—this perspective must be placed in the context of the situations in a significant number of countries in which even the most basic data such as school enrollments are missing or unreliable and data collection is a competing area of resource allocation. For practical as well as conceptual reasons, the data collected should be those that are the most relevant to the context and best highlight the success or failure of the state to meet its obligations. Of course it probably would be the case that across many or even all country situations, some particular indicators were, in every case, relevant to the assessment of official conduct—they were in fact universally applicable. But this conclusion would be reached a posteriori, as a result of examining the relevance of measures in different contexts, not because those indicators were assumed a priori to be relevant measures across contexts. Though the result of having a universally applicable indicator would be the same under each approach, the rationale would be significantly different. Given that most ongoing attempts to develop indicators are based on the assumption that the indicators selected should a priori be universally applicable, the alternative approach of deriving context-specific indicators has received relatively little attention. This may also be because most attempts to develop indicators are taking place within international discussions of human rights law. Attempts undertaken within national contexts would not be so wedded to universal applicability and so would be more likely to explore alternative options. One example is a case study of the right to health in Ecuador in the 1990s by the Center for Economic and Social Rights.14 The CESR assessed this right by setting out the principles of state obligation with respect to the right to health and then presenting indicators that would contribute toward a relevant assessment of each obligation within Ecuador’s current economic, political, and social context. Although the lack of data available limited the scope of the case study, the CESR’s research clearly demonstrated the rich potential of the obligations approach to human rights assessments. With respect to the obligation of the state to protect rights, the CESR asked whether people suffered systematic, harmful effects to their health from actions by private actors and whether the state was taking adequate measures to protect them. The physical and sexual abuse of women by partners and family members is one of several private sources of threats to health identified: data showed in 1998 that 88 percent of women in Guayaquil, the largest city, said they had suffered some form of intrafamilial violence. Was the state doing enough to 14 Center for Economic and Social Rights, From Needs to Rights: Realizing the Right to Health in Ecuador (Quito: Genesis Ediciones, 1999). MEASURING HUMAN RIGHTS protect women from this threat? Previously, between 1989 and 1992, of 1,920 complaints relating to sex crimes against women and girls in Guayaquil, only 2 percent had resulted in convictions—implying that the state did not adequately protect victims through the judicial system. Later in the 1990s the introduction of the Law against Violence against Women and the Family brought additional legal protection. New data are now needed to show whether the law actually is resulting in better protection for women from this particular threat and whether this protection has had any effect on the number of cases of abuse occurring. With respect to nondiscrimination, the CESR asked whether the state conducted its policy with equity. Data show that despite high inequality and extreme deprivation of rural, poor, and indigenous populations, the state disproportionately allocated scarce resources to urban and better-off social groups. In 1997, 84 percent of urban dwellers had access to health services, compared with only 10 percent of rural dwellers; 80 percent of health personnel were in urban areas, and the state spent six times as much on social security for urban dwellers as for rural ones. These data call for a prompt reallocation of the health budget and institutional capacity toward providing services to rural populations: more extensive data would reveal whether a similar policy shift was required for low-income and indigenous populations. As a final example, with respect to making adequate progress, the CESR asked whether the state could be judged to be moving fast enough toward realizing the right to health. Taking just one example, in 1970 the state set targets to provide adequate sanitation facilities for 70 percent of the urban and 50 percent of the rural population. Yet more than a decade later, the situation was moving in reverse of those targets. Between 1982 and 1990 the share of households with access to sanitation facilities fell from 46 percent to 38 percent in urban areas, and from 15 percent to 10 percent in rural ones. Even by its own targets, the state was not making adequate progress. The Ecuadorian example demonstrates the potential of this approach to creating indicators, but its complexity cannot be ignored. More than this partial picture must be provided. Data disaggregated by gender, ethnicity, income level, and region are needed to enable analyses of discrimination. Data relevant to the duration of a specific government are needed to provide an analysis of the effects of its policies. When interpreting these data, analysts face the inevitable complications of defining which failures are the fault of the government and which are due to external conditions. The example from Ecuador illustrates this well. By 1998 only 3 percent of the national budget was allocated to health, in contrast to 30 percent for debt servicing. The influence of the international environment, and international institutions, on this 127 128 Kate Raworth outcome must be considered as part of assessing whether the government of Ecuador could have done more to meet its obligations. These and other complications raise the final question of what needs to be done to make the obligations approach to indicators a feasible tool in practice. Moving Forward with Indicators: Challenges Ahead If it is accepted that, for the purposes of evaluating policy, indicators should be focused on the obligations approach to assessing state conduct, and if it is also accepted that the universality of human rights should be reflected in the principles underlying those obligations of conduct, then the focus of future work should not be on continuing to propose lists of indicators but rather on addressing the remaining conceptual and practical complications. What conceptual complications need to be tackled to make an operational policy tool? Certainly a better elaboration of the principles of conduct is needed, one that does not implicitly assume that all state obligations for all rights can be met and maintained simultaneously. This assumption implicitly underlies the present structure of principles, yet it has not been proved theoretically and has certainly not been demonstrated empirically by any country. The potential conflict between obligations is often waived in dialogue or is dogmatically rejected because, it is claimed, a policy solution must be found that enables all obligations with respect to all rights to be met simultaneously, since countries have ratified the covenants of international law and thus are accountable for meeting the obligations within them. This deep and unjustified assumption that such a policy stance is always possible flies in the face of reality. There are of course genuine conflicts of interest— indeed, the overriding characteristic of the policy-making process is one of weighing conflicting interests among potential resource allocations, social groups, and competing freedoms. It is interesting to note that this world of tradeoffs is the familiar domain of economists, and so it should not be surprising that they often have little patience with the language of international human rights law, which largely fails to address the existence of such conflicts. As an example, an elaboration is needed of the priorities to be allocated among the obligations to respect, protect, and fulfill a right, as such conflicts do arise. Is it ever permissible for a state to fail to respect a right of some people in order to promote or fulfill a right of others, or indeed another right of those same people? Curtailing freedoms from unreasonable search and seizure may contribute to signifi- MEASURING HUMAN RIGHTS cant decreases in rates of violent crime, thereby better fulfilling the rights to integrity of the person.15 Carlos Bosambrio has noted that “a growing number of people in Latin America believe that civil and political rights can, if necessary, be sacrificed to guarantee one’s right to live in peace and in a secure environment.”16 How should human rights advocates evaluate such cases? Likewise, the obligations of process need further elaboration. For example, what criteria turn differences in the way two people are treated or affected by official institutions into an instance of discrimination? What rate of progress toward realizing rights is adequate and who is the legitimate judge of that? What extent and type of participation on the part of those affected by policy is required in the formulation of that policy? Furthermore, current approaches to creating indicators focus on making assessments of one right at a time in isolation from other rights, creating “silo rights” assessments with no explicit or even implicit mechanism for interaction or tradeoffs between them. Yet in reality there are always competing claims to be weighed between rights, such as conflicting claims on resource use or conflicts of interest between and impacts on different social groups. A policymaker weighing a decision on one right would inevitably be taking into consideration the consequences of this policy stance on the state’s ability to meet its obligations on other rights. Such a tension lies at the heart of state choices regarding policy impacts on human rights, yet the current framework of obligations is structured in such a way that it does not reflect this dilemma. The complexity of policymaking situations needs to be reflected in the ways that indicators are used, and this will require substantial work on elaborating those principles that guide choices between competing interests. If, in addition, the influence of external factors is taken into account, specifying the scope of obligations becomes increasingly difficult. When the health spending per person is sharply reduced because of an externally triggered financial crisis in a country, what is the obligation of the government to act, and to what extent can it be held accountable? I have so far largely avoided discussing the complexities introduced by the influences of the external environment, but they would inevitably enter into a full specification of state obligations. Although it is clear that further conceptual elaboration of the principles of obligations is needed for all the reasons given above, it is not clear how that elabora15 This example is drawn from Christian Barry, “The Challenges of Conceiving and Measuring Human Rights” (paper presented at UNDP Global Forum, Rio De Janeiro, Brazil, October 10–11, 2000); available at www.undp.org/hdro/rioforum/rioagenda.html. 16 Carlos Bosambrio, “Crime: A Latin American Challenge for Human Rights,” Human Rights Dialogue (Winter 2000); available at www.carnegiecouncil.org/hrdwinter2000.html. 129 130 Kate Raworth tion should be accomplished. Should, for example, the prioritization of obligations be universally applicable, such as “in any conflict between obligations to respect and to protect, the obligation to respect should prevail,” or should it be specific to each right, to the particular context, or to other moral criteria? Should the interaction of these principles of obligation be decided a priori or should it be contingent upon the particular instance of conflict? And should the decision be made externally to any particular conflict of interest by human rights legal experts through jurisprudence or internally to the particular conflict by, say, the popular choice of those people affected by the decision? For any of these options, could an additional category of indicators be developed to help show whether or not the obligations were being prioritized according to the principles? For example, if popular choice were believed to be the correct way to decide the weighting between particular conflicting obligations, could the results of a referendum or a popular survey be used as one element in the assessment of whether the state is meeting its obligations? These are evidently complex issues, and perhaps they make the creation of human rights indicators even more daunting. But unless the principles underlying the obligations of conduct are better elaborated, any attempt to use indicators as monitoring and policy tools will clearly be inadequate because they will fail to capture the complex reality of conflicts that decision-makers constantly face. Making assessments of obligations is inevitably complicated—there are rarely simple answers—but perhaps it is appropriate that this is so evident within the obligations approach: the complexity of making assessments reflects the fact that human rights often involve the most basic dilemmas of human society. What practical complications need to be addressed? Given the contextual specificity of assessing obligations and the detailed data required to do it, the assessments clearly need to be rooted in national contexts. With the rise of national human rights commissions and other nongovernmental organizations focused on promoting human rights over the past fifteen years, such national rootedness is increasingly possible—and a far cry from the externally conducted qualitative ratings of the 1950s. Of course, in some countries national assessments still are not possible because of strictures on freedom of expression, either in policy or in practice, or because of the lack of national expertise, and so the need for international or overseas organizations to undertake assessments will still be there. This point of arrival may be disappointing in its inconclusiveness and frustrating in its complexity: the desire to revert to assessing human rights by universally applicable indicators of the enjoyment of rights would be understandable—but it would be a mistake. Instead, progress must be made simultaneously on several fronts. Philosophers and human rights lawyers need to elaborate further the interre- MEASURING HUMAN RIGHTS lations of the principles of obligation. Statisticians should collect more detailed and disaggregated data that can be used to assess various obligations. National human rights commissions and other NGOs then can use these improved tools to undertake more case-study assessments, along the lines of the study undertaken in Ecuador. The results will be the judge of whether the concepts of human rights can be turned into a framework for assessing policy, and whether indicators can be developed to accomplish the task. 131