Michael presented the paper at a workshop on fundamental issues in the measurement of subjective well-being in February 2021. You can watch Michael's talk below and view all the recordings from the workshop here.
It is very common for people to give numerical ratings of their subjective experiences. For instance, people often score their happiness, life satisfaction, job satisfaction, health, pain, the movies they watch, and so on, on a 0-10 scale.
A long-standing worry about these self-reports is whether the numbers represent the same thing to different people at different times. For example, if two people say they are 5/10 happy, can we assume they are as happy as each other? More technically, the basic (but not only) question is whether subjective scales are cardinally comparable: does a one-point change, on a given scale, represent the same size change?
If the scales are merely ordinal - that is, the numbers represent a ranking but contain no information on the relative magnitudes of differences - we are in trouble: it would not be possible to use subjective scales to say what would increase overall happiness.
Opinions about the Cardinality Thesis - the idea that subjective scales are cardinal - seem to divide on disciplinary lines: economists are sceptical of it, psychologists less so (Ferrer‐i‐Carbonell and Frijters, 2004). Despite this division, and the fact this is a foundational methodological issue, it seems to have attracted little attention in the literature. Researchers will tend to assume the scales are ordinal or cardinal, and so use different statistical tests, without defending their decision (Kristoffersen, 2011). This lack of scrutiny is likely explained, in part, because the topic is too applied for philosophy and too theoretical for social science.
The result is that, at present, there is insufficient conceptual clarity around the Cardinality Thesis to know whether or not there is a problem or what could be done about it. Quoting Stone and Krueger (2018, p189):
In order to have more concrete ideas about the extent to which this may be a problem, we should have a better idea of why such differences [in scale interpretation] might exist in the first place, and have some theoretical justification for a concern with systematic differences in how subjective well-being questions are interpreted and answered.
Against this background, this paper makes four main contributions: First, it proposes a novel theoretical explanation for how people interpret scales that draws on philosophy of language and game theory.
In brief: conversation is a cooperative endeavour governed by various maxims (Grice, 1989). As subjective scales are vague and individuals want to be understood, scale interpretation should be understood as a search for a ‘focal point’ (or ‘Schelling point’), a default solution chosen in the absence of communication (Schelling, 1960).
A specific focal point is proposed: when given an undefined scale with a finite number of options, individuals (unconsciously) use the end-points to refer to the realistic limits of that quantity, e.g. 10/10 is maximum happiness anyone feels. Further, they interpret the scale as linear, so each point represents the same change in quantity. If this hypothesis is correct, self-reports will be cardinally comparable (given one further assumption, phenomenal cardinality, defined below).
Second, the paper states four conditions that are individually necessary and jointly sufficient for cardinal comparability to hold on average.
Stated roughly, these are:
C1: phenomenal cardinality (the underlying subjective state, e.g. happiness, is felt in units)
C2: linearity (each reported unit change represents the same change in magnitude)
C3: intertemporality (each individual uses the scale the same way over time and the scale end-points represent the real limits)
C4: interpersonality (different individuals use the scale the same way and the scale end-points represent the real limits)
What should we conclude about the cardinal comparability of subjective data if the conditions fail? It depends which condition(s) fails.
The first condition is binary and fundamental: each subjective phenomenon is either felt in units or it is not. Of course, a non-cardinal phenomenon cannot be measured on a cardinal scale. However, all the other conditions can fail by degree and what’s important is by how much they deviate.
By analogy, C1 is about whether we can have a measuring stick at all, C2 concerns whether the measuring sticks are bent, C3 is if the length of each stick changes over time, and C4 is whether different people have the same length sticks.
It makes a difference if our measuring sticks are slightly bent or very crooked. The cardinality thesis could fail to be exactly true, but nevertheless be approximately true, such that it is unproblematic to treat it as true.
Third, the paper notes we can use evidence and reasoning to assess whether, and to what extent, the conditions hold, even though they concern subjective states. While it is always true that more than one hypothesis will fit the facts, that does not mean all hypotheses are equally likely. Here, as elsewhere, we rely on inference to the best explanation.
The paper examines each condition in turn, primarily drawing on the subjective well-being literature. In each case, there is evidence indicating the condition does hold and no strong evidence suggesting it does not. As such, the tentative conclusion of the paper is that subjective scales are best understood as cardinally comparable unless, and until, other evidence suggests otherwise.
Fourth, the paper sets out some testable predictions of the theory and explains how such tests could be used to ‘correct’ the data if people do not intuitively interpret subjective scales in the way hypothesised. The conclusion of the paper is an optimistic one. It seems there is not a problem where we feared there might be one, and we may be able to fix the problem if we later discover it does exist.
The Happier Lives Institute (“HLI”) is operating through a fiscal sponsorship with Players Philanthropy Fund (Federal Tax ID: 27-6601178), a Maryland charitable trust with federal tax-exempt status as a public charity under Section 501(c)(3) of the Internal Revenue Code. Contributions to HLI are tax-deductible to the fullest extent of the law.