The problem with health metrics (QALYs and DALYs)
Researchers in the effective altruism community have tended to use health metrics (QALYs and DALYs) as the proxy for Well-being Adjusted Life Years (WELLBYs). However, these standard health metrics are misleading proxies for well-being. For ease, we quote at length from Clarke et al. (2018, p85): In the QALY system, the impact of a given illness in reducing the quality of life is measured using the replies of patients to a questionnaire known as the EQ5D. Patients with each illness give a score of 1, 2, or 3 to each of five questions (on Mobility, Self-care, Usual Activities, Physical Pain, and Mental Pain). To get an overall aggregate score for each illness a weight has to be attached to each of the scores. For this purpose members of the public are shown 45 cards on each of which an illness is described in terms of the five EQ 5D dimensions. For each illness members of the public are then asked,“Suppose you had this illness for ten years. How many years of healthy life would you consider as of equivalent value to you?” The replies to this question provide 45N valuations, where there are N respondents. The evaluations can then be regressed on the different EQ5D dimensions. These “Time Trade-Off” valuations measure the proportional Quality of Life Lost (measured by equivalent changes in life expectancy) that results from each EQ5D dimension. 
Figure 4. How life satisfaction and daily affect (0-1) are affected by the EQ5D, compared with weights used in QALYs (Data from Dolan and Metcalfe 2012)
Imagination vs. experience
QALYs are not a very good guide to what makes people happy or satisfied because they are based on people’s preferences over how bad they imagine various health states are, rather than how bad they are when they experience them and we are not very good at imagining what makes us or others happy (see below). To highlight a particularly outstanding discrepancy, Dolan and Metcalfe (2012, from whom the above figure 4 is derived) report subjects agreed to hypothetically give up as many years of their remaining life, about 15%, to be cured of ‘some difficulty walking’ as they would to be cured of ‘moderate anxiety or depression.’ However, from SWB measures ‘moderate anxiety or depression’ is associated with 10 times a greater loss to life satisfaction, and 18 times a greater loss to daily affect, than ‘some difficulty walking’ is (note the time trade-off lines for ‘mobility 2’ and ‘anxiety 2’ in figure 4 are the same length but the two SWB lines are very different). This seems compelling evidence, if we need any, that if we rely on people’s preferences about imagined futures we will get the wrong answers about what makes individuals happy. Psychologists use the term ‘failures of affecting forecasting’ to refer to predictive mistakes we make when predicting what will make us and others happier in future. One reason for this is that, when imagining the future, we fail to anticipate that our ‘psychological immune system’ will ‘kick in’ and cause us to adapt to some circumstances but not others: what Gilbert et al. (2009) call ‘immune neglect’. Conditions such as mobility impairment are things we stop paying attention to, whereas mental illnesses are comparative ‘full-time’ and continue to affect our subjective experiences. We are unaware of any studies comparing DALYs and SWB measures directly, but given how DALYs are constructed - typically by asking experts for ratings - we would expect the same problems to occur. See Sassi (2006) for a comparison of the methodologies for QALYs and DALYs. Physical health vs. mental health interventions An implication of this analysis is that we should substantially reduce how cost-effective physical health interventions are compared to mental health interventions, assuming we’d previously judged them by QALYs and DALYs as Giving What We Can did in their reports into mental health (GWWC 2015, 2016). By itself, this doesn’t mean mental health interventions are more cost-effective than physical health interventions. We need to say more about the costs and effects than this but it is a relatively big update in our analysis. Perhaps the reason the effective altruism has largely overlooked mental illness is largely because of the movement’s early reliance on QALYs/DALYs as an approximation of well-being. Given how much QALYs underrate the badness of mental health, it’s not much of a surprise individuals using those metrics would be led to the conclusion mental health is comparatively unimportant. ⇦ Previous
Well-being Adjusted Life Years (WELLBYs) |