Social Science, Policy and Democracy

Can social science provide policy-guidance without undermining some basic democratic values? It would clearly be devastating if the answer was “no”: Most people are deeply committed both to democracy, as well as to the idea that policy decisions should be informed by the best available science, including the best available social science. Accordingly, the many philosophers who have worried about potential tensions between science and democracy have come out arguing that, if done right, good science and democracy mutually support rather than undermine each other, John Dewey and Philip Kitcher being paradigmatic examples.1 This article argues that there is an under-appreciated democratic challenge for policy-relevant science, which I will articulate specifically in the context of value-laden social scientific indicators. Value-ladenness has long been acknowledged to pose an obstacle for reconciling science and democracy: It creates the potential for the value judgments made by a small subset of the population to have a significant impact on policy decisions, in a way that bypasses normal processes of democratic legitimization. Consequently, solutions to this challenge have either defended the value-free ideal,2 or stressed the need to, in one way or another, democratically align the values entering science, in a way that is parallel to how democratic legitimacy is given to public decision-making more generally. The nature of many social scientific indicators makes the value-free ideal wholly unworkable, lest we give up the entire project of aiming to measure poverty, inequality, or wellbeing. And so only the second common type of response seems to be available in their case. But, I will argue, this response misses a significant part of the challenge value-ladenness poses to democracy. The solution, I will argue, is greater value pluralism rather than democratic alignment. As I write, the United Kingdom, like many countries around the world, is facing a devastating cost of living crisis. The incomes of large numbers of households cannot keep up with the rising prices of the goods and services they consume, pushing increasing numbers into poverty. To tackle this crisis, it is important that policy-makers have a clear picture of inflation—of how much the cost of living has increased, how it is projected to further increase, and how different policy options will affect the rate of change. In the United Kingdom, the Office of National Statistics (ONS) supplies a number of measures of inflation that are treated as key indicators by policy-makers, by the public and media holding them to account, and by social scientists studying the causal relationships between inflation and other social scientific variables or policy interventions. The most widely used and reported indicator is the Consumer Price Index (CPI). But what is the cost of living? The first thing to note is that it is clearly a value-laden concept and treated as such by those using it. Saying that the cost of living has gone up involves making a comparative evaluation of the kinds of goods and lifestyles that can be afforded by households before and after changes in prices. In the most general terms, it is good for people to be able to afford to live well, and measures of changes in the cost of living aim to capture the extent to which households can do that. Clearly, however, there are different ways of making this idea precise. Two salient ways of making it at least somewhat more precise would be to either think of it as the cost of meeting one's basic needs, or to conceive of it as the cost of consuming the kinds of things one is actually consuming—which in many cases will go beyond meeting one's basic needs. In either case, the cost of living in fact differs between different people. Clearly, different people actually consume different things, the prices of which may change to different extents. But even the cost of meeting basic needs may differ between different people; I need corrective lenses to function in daily life, while you may not. A measure of inflation useful for large scale policy needs to capture changes in the aggregate cost of living despite this heterogeneity.3 Both the choice of concept of “cost of living” as well as choices about how to aggregate over a diverse population involve important value judgments, about what makes life good, about who matters how much, about what we are entitled to expect compensation for from each other and the state, and about our policy priorities. Actual measures of inflation embed these value judgments. In practice, the headline CPI published by the ONS aspires to capture changes in the prices of the things households are actually consuming.4 It uses a representative basket of 700 goods, regularly collecting price quotations from different kinds of outlets all over the country. In deciding on the quantities of each good in the basket, the headline CPI defers to the consumption decisions of households. More specifically, it gives weight to different households in proportion to their share of the total expenditure of all households, making the CPI more sensitive to the consumption decisions of richer households. The ONS has been under increased pressure from anti-poverty campaigners in recent years arguing that the headline inflation figures do not capture well the extent of the cost of living crisis experienced by the poorest households.5 The criticism is both that the standard basket used by the ONS does not include enough of the items the poorest households are consuming to meet basic needs, such as supermarket own-brand products, and that, as an effect of aggregation, higher price increases on those products are washed out by lower price increases in the goods disproportionately consumed by richer households. Periodic disaggregation carried out by the ONS can address the second concern to some extent, but not the first. The central controversy here is at heart about values and policy priorities: Relative to the other purposes inflation measures may help to serve, campaigners want greater policy priority given to compensating the worst-off households for increases in the cost of meeting basic needs, or to preventing such increases from happening in the first place. Consequently, they want inflation measures to be designed for serving those purposes to the greatest extent possible, which they traditionally have not. The ONS is currently making some changes to address these challenges better, to which we will return below.6 To get to the heart of my concern in this article, note that there are two levels to this controversy. On the one hand, there is disagreement about the values that, in virtue of being captured by policy-guiding social scientific indicators, will end up shaping public policy. In that respect, the controversy is much like any other disagreement in the political sphere. But on the other hand, there is also the complaint that one side to this debate does not have access to the measurements (and scientific studies employing them) that would help them make their case in public debate, while the other side does. One side can appeal to official statistics capturing what they care about, while the other needs to rely on evidence that remains anecdotal. They are not entering public deliberation as equals. This is a distinct issue from the first, and, I argue, one that is neglected by the literature aiming to reconcile democracy and the value-ladenness of science. It is also an issue that, for reasons explored below, is unlikely to sufficiently self-correct in democratic societies. Measures of inflation are just one type of social scientific indicator that is at once value-laden, and highly relevant for guiding policy. Were it the case that the value judgments in question were made entirely by unelected social scientists, based exclusively on personal values unrepresentative of the population at large, this would be problematic on any plausible normative theory of democracy. To the extent that value judgments have an impact on policy, we may want to ensure that the values embedded in them are picked in a democratically legitimate way, just like we aim to ensure democratic legitimacy for any significant policy decision. But there is another important aspect of the relationship between policy-relevant science and democracy. The outputs of policy-relevant science, such as social scientific indicators, are not only used to inform the choices of policy-makers. They also feed back into public deliberation, and are used to hold policy-makers to account. Focusing only on the democratic selection of the values that enter the research that is then used to guide policy choice ignores this second purpose of policy-relevant science. This is problematic specifically in contexts of significant value disagreement. Modern democracies often find themselves confronted with irresolvable disagreement about the values at stake in a policy decision, including the types of values that are embedded in policy-relevant social scientific indicators. Still, most accounts of democracy allow for such decisions to be made without requiring that consensus is in fact reached. Likewise, we may, in a democratically legitimate way, settle on the values to be embedded in the social scientific indicators that are then used to guide specific types of policy, without in fact reaching consensus on those being the right values. However, if no other social scientific indicators than the democratically selected ones are widely available to the public and to social scientists, this raises, I argue, a problem of epistemic inequality: Only those who share the democratically selected value judgments embedded in the chosen indicators can make full and reliable use of them in public deliberation and when holding policy-makers to account. Others are at an epistemic disadvantage, since the available indicators only partially capture what they take to be important, and at worst, there may be no available measurements of the things most crucial to them given their values and interests. In an important respect, then, those others do not enter further public deliberation as equals, and are not to the same extent able to hold the government to account. While the values were, by hypothesis, originally chosen in a democratically legitimate way, there is a danger they become entrenched, undermining the good functioning of democracy going forward. What is needed to address this problem, I will argue, is for social science to aim for an output that is value pluralist, in order to help ensure a distribution of the epistemic goods needed for good public deliberation that is sufficiently equal to achieve important democratic values. In the case of the measurement of value-laden social scientific indicators, two feasible strategies to ensure greater value pluralism are the availability of publicly accessible dashboards of multiple indicators on the one hand, and tools that allow policy-makers and the public to adjust the weightings of different components of aggregate indicators, on the other. Unfortunately, this conclusion runs counter to trends in several important fields of research in policy-relevant social science, where there are ambitions of and political pressures toward a single metric to guide policy. An important upshot of my argument is that these ambitions are highly problematic from a democratic point of view. And insofar as pluralism is already practiced or advocated for in the social sciences, I provide an additional and neglected democratic justification for it. The next section will outline what I take to be some uncontroversial ideas regarding social scientific indicators, values and science to serve as a backdrop to my argument. It will draw some implications for the common ambition of a single, dominant policy-guiding indicator. Section III will spell out a first democratic challenge for such indicators, and show how it motivates existing philosophical accounts of how this democratic challenge can be addressed. I go on, in Section IV, to characterize the notion of epistemic equality at the core of my new democratic challenge, and explain how existing philosophical attempts to reconcile social science and democracy fail to address it. Section V contains my positive proposal along with examples of existing good practice. My argument in this article concerns scientific outputs that serve both an epistemic as well as a policy-guiding function. Social scientific indicators are paradigmatic examples of such scientific outputs. Let me start by outlining five uncontroversial ideas concerning social scientific indicators. Many of the most policy-relevant social scientific indicators measure “thick concepts,” which are concepts that are partly descriptive, and partly evaluative. As discussed in the introduction, “cost of living” is one such concept. Other important examples include wellbeing (more on which in what follows), economic welfare, economic inequality, poverty, and health, all of which are measured and studied by the social sciences.7 Disagreement may be about how a particular concept should be understood in a particular context. For instance, in our opening example, part of the controversy over the ONS's inflation measurements are about whether the notion of the cost of living to be measured should be the cost of consuming our actual consumption baskets, or the cost of meeting basic needs. In other domains, it may be about what it means to be poor, or healthy, or, as we'll see, about what wellbeing consists in. This is disagreement about concepts, that is, disagreement about what we should count. There is, in addition, also often disagreement about how we should count it: How do we compare the cost of living, wellbeing, health, or economic welfare of different people, and how do we aggregate them in order to study population-level effects? There are two aspects to this: First, we often need to determine how to compare and meaningfully add up two kinds of effects in the first place, how to compare apples and oranges, as it were (quite literally, in the case of inflation measurement!). Call disagreement about this disagreement about commensuration. Second, there is the question of how much weight to assign to different effects once we have in principle decided how we can compare them. The crucial question here is often how much weight to give to effects on different people, e.g., different people's wellbeing or cost of living. Call disagreement about this disagreement about aggregation. Finally, there is also often disagreement about the relative importance of the values captured by a social scientific indicator. We may, for instance, agree on what poverty is, but may disagree on how important it is relative to other policy priorities. Call this disagreement about priorities. For the purposes of this article, when people give priority to their own interests relative to others, and as a consequence favor a way of aggregating, for instance, wellbeing in a way that gives more weight to the wellbeing of people like them, I will treat this as a kind of value disagreement (where interests are not explicitly specified alongside values, this should be read to be included). And so, value disagreement in the relevant sense can be expected to be especially common where the interests of different segments of society are in conflict with one another. To some extent, the process of sharing reasons with each other may resolve some of these disagreements, especially where people's views may have been based on insufficient reflection or experience. We may come to see each other's point of view and adjust our own, regarding, for instance the nature and relative importance of poverty. However, professional philosophy provides good evidence that complete consensus on fundamental questions of value, such as, for instance, the correct theory of wellbeing, is not a realistic prospect even in an environment where the constant exchange of arguments is routine. Here, I have in mind mainly two senses in which an indicator can be aggregate: It can aggregate effects on many different people, and it can aggregate different kinds of effect. For instance, the CPI is aggregate in both respects: It aggregates the changes in the cost of living of many different people. And it also aggregates price changes of many different types of goods. Compared to disaggregated price indices for individual people or individual types of goods, this involves making additional value judgments: First, there are judgments about commensuration, about how to compare different types of effects on different people in the first place. And second, there are judgments of aggregation proper, of how much weight to give to different people and different kinds of effect. For instance, do different households count equally or proportionately to share in overall spending? These additional value judgments mean that there is more room for value disagreement, namely for what we above called disagreement about commensuration and disagreement about aggregation. This makes it less likely that consensus can be reached even in an extensive process of public deliberation. Policy-makers can only effectively serve society, and society can only engage in fruitful public deliberation, if we have reliable and usable information on what the likely consequences of the different available policy options are. Many of the most important consequences of public policy concern phenomena studied by the social sciences, and these sciences offer the best means of reliable knowledge about the effects of public policy. And so, it is commonplace, among both social scientists and philosophers of science, to treat it as a desideratum that social science should provide useful policy-guidance. While the general idea is very intuitive, on closer inspection, there are at least two important ambiguities to this desideratum. The first relates to whether what is desirable is that social science (a) studies some of the effects of policy that are choice-relevant, (b) studies all of the effects of policy that are choice-relevant, or (c) supplies a measure of how choice-worthy different policy options are all-things-considered. These differ in the extent of guidance demanded from social science. The second ambiguity arises from the fact that which effects are choice-relevant depends on the goals and values to be pursued. Some salient possibilities of what these could be are either (i) those of the actual policy-maker, (ii) those that would be decided on in a democratically legitimate process, or (iii) all policy-relevant values reasonably held by members of the population. The point of applying (i) or (ii) would be that social science should assist (democratic) public decision-making, and demands for policy-relevance have usually focused on these notions of policy-relevance. Sense (iii) is salient once we take seriously an idea explored below: that social science should also inform the public deliberation preceding public choice, and the holding to account of public decision-makers. Once we clear up these ambiguities, some of the weakest formulations of the desideratum are clearly uncontroversial: Social science should produce reliable and usable information on some of the policy effects that are choice-relevant relative to some values reasonably held by members of the population. Plausibly, they should also, in particular, produce reliable and usable information that is choice-relevant relative to the values a policy-maker is pursuing in a democratically legitimate way. As we will see, in some debates, however, the desideratum is understood in stronger terms, namely as requiring social science to produce all-things-considered measures of policy-evaluation. Call this strong policy-guidance, an idea admittedly (and fortunately) far from policy-making practice. In these discussions, the relevant values to be pursued are typically thought of as the values policy-makers are either assumed to actually have or ought to have. Such measures would provide action-guidance to policy-makers in the most direct terms. We will see below that the desideratum of strong policy-guidance is highly problematic. Ultimately, policy-makers do need to take into account all relevant effects of the policy options open to them, weigh up their relative importance and arrive at an all-things-considered judgment as to what the best policy option is. This will typically require them to perform some kind of aggregation of effects on different people and of different types, either explicitly or implicitly. Now suppose you subscribe to the desideratum of strong policy-guidance, and so take it to be desirable for the social sciences to produce measures that tell policy-makers how well different policy options serve policy-makers' goals all-things-considered. Given the need for aggregation, this implies that social science should be producing the most aggregate social scientific indicator, one single metric to integrate all positive and negative effects of different policy options on different people and of different types, to be used to directly guide public policy. This, in a nutshell, is the motivation for a number of projects in the social sciences to provide policy-makers with, in the words of Anna Alexandrova, a “master number” to guide public decision-making.8 Sometimes these projects have included efforts to push for the preferred metric of policy analysis to be integrated in the standard procedures of the administrative machinery of government. A recent phenomenon of this type is the advocacy of behavioral scientists and happiness economists subscribing to the subjective wellbeing approach to treat an aggregate subjective wellbeing metric as just such a “master number” to provide an all-things-considered evaluation of policy.9 In the United Kingdom, these efforts have had some success, with subjective wellbeing now being included in the government's Green Book.10 Note, however, that while the academic ambition in the relevant social scientific communities is a truly all-things-considered metric for policy evaluation, in practice, of course, even sympathetic policy-makers will inevitably take other things into account, in a way that is sensitive to context and the purpose at hand. Past projects that don't aim at a measure that is strictly speaking all-things-considered, but still aim at a fairly comprehensive assessment of how well a country is serving its people's interests, have been Gross Domestic Product (GDP), and, building on that measure's weaknesses, the UN's Human Development Index (HDI). Going back to our list of uncontroversial claims, one consequence of any such “master number” approach, if it insists on a single all-things-considered metric, is that there is likely going to be widespread disagreement about the values embedded in this single highly aggregate metric. The same is true, to a slightly lesser extent, of metrics that are less “all-things-considered” but yet highly aggregate. Specific measures of the cost of living, poverty, or inequality aim to capture only one type of policy-relevant effect. Still, they embed value judgments about which there is widespread disagreement. Within these domains, too, however, debate between social scientists often focuses on finding the one best metric, to then present to public decision-makers and the public as policy-guiding. This tendency toward monism within the social sciences, be it regarding a single all-things-considered “master number” or within individual sub-domains of policy interest is reinforced by political and pragmatic pressures11: political pressures to provide unambiguous policy-guidance, enabling policy-makers to say they were merely “guided by the science”; and pragmatic pressure based on the fact that the cost of the measurement of social scientific indicators, such as the measurement of inflation, is very high. The result is that in the case of many of the social and economic effects of policy, a single or a small number of indicators at once serve to guide policy and inform public debate, while alternative measures are either lacking or much less known and accessible—just think of the dominance of GDP in reporting and public deliberation about the economy. And notably, these measures by and large remain the same over time even in the face of shifts in public opinion and changes in government. The next section spells out one type of threat this phenomenon can pose to democracy, familiar from more general discussions on values in science. It will then outline philosophical accounts of the democratization of science that have aimed to address this threat, all of which aim at some form of what I call democratic alignment. I will ultimately argue that there is another democratic threat that these accounts do not sufficiently address, and which can only be countered with greater value pluralism. Consider this hypothetical example of how value-laden social science may guide policy: A social scientist develops a measure of national wellbeing, making all required value judgments on the basis of only her own moral intuitions, theorizing, and reading of the philosophical literature on wellbeing. Her theorizing remains inaccessible to the public and policy-makers, but she and her team produce both regular measurements of national wellbeing, as well as studies estimating the effects of different potential policies on national wellbeing. Policy-makers use the results of these studies in an effort to maximize national wellbeing, explaining to the public that they are merely “following the science.” This is clearly an example of a problematic kind of technocracy that runs counter to core democratic values on any normative theory of democracy. The scientist was neither publicly elected, nor is her reasoning about which values to capture with her national wellbeing measure open for public scrutiny. Her reasoning is also not responsive to the values of either the public or democratically elected public decision-makers. For the reasons discussed in the last section, it is moreover unlikely that the values the scientist ends up capturing are uncontroversial, given the aggregate nature of a measure of national wellbeing. Yet, her choice of measure ends up having a substantial effect on public policy, in a process that bypasses ordinary processes of democratic legitimization. It is clear that our imagined example is not a case of democratic decision-making—power is not exercised by the people here, either directly or indirectly. Different normative accounts of democracy explain what is bad about this in slightly different ways. One important branch of democratic theory sees democracy as a way of treating everybody as equals. There is a variety of such egalitarian conceptions of democracy. One idea is that democracy is the best way to ensure public equality, that public institutions and decision-making procedures treat everybody equally, giving everybody an equal say over how the communal aspects of our lives ought to be organized.12 Another is that democracy ensures relational equality, that citizens stand in relationships of equality to each other, and not in relations of domination and subordination.13 Technocracy of the type I described is problematically inegalitarian on each of these types of views. The social scientist has disproportionate power over her fellow citizens, being able to impose her value judgments on everybody else. This is relationally inegalitarian, and she is being given that power by the way in which public decision-making is structured. Disproportionate power is also at the heart of other potential explanations of what is bad, from a democratic perspective, about the kind of technocracy just described. According to some accounts, the crucial value of democracy consists in enabling the public justification of the institutions and policies that rule our lives.14 Public justification requires, at a minimum, that the reasons for and against different policy options are made public and subjected to a process of public deliberation among equals. But in the technocratic scenario, not only does the scientist have a disproportionate say, but the reasons for public decisions also remain opaque. Well-informed public deliberation has been emphasized more generally by advocates of the deliberative democracy movement. Aside from the ones already mentioned, it is also crucial on other prominent views of democracy. For instance, on perfectionist views of democracy, such as that put forward in Mill,15 democracy is good because it gives citizens the opportunity to exercise their moral capacities. On instrumentalist conceptions of the value of democracy, democracy is good because it tends to lead to good decisions, for which a well-informed process of public deliberation is in turn important.16 While our scientist has important technical and scientific expertise, she likely does not have significantly more expertise on questions of value than any other citizen, so that decision-making could be improved by including others in the deliberative process.17 The example I started this section with is of course only a cautionary tale. But it is not far off the vision pursued by some of the advocates of “master number” approaches discussed in the last section.18 And it highlights a certain kind of danger for policy-relevant social science more generally. The danger comes from the fact that contentious value judgments need to be made in such research, and that such research in turn often directly influences public policy. Now it might seem that to address this problem, all that needs to be ensured is that the values that enter the social scientific research are selected in a democratically more legitimate way than in our cautionary t