Mental health interventions vs other life-improving interventions
Even if there are mental health interventions that work, effective altruists shouldn’t donate to organisations that combat mental illness if they can find something even better. As the global development charities recommended by GiveWell are widely accepted as the most effective organisations to donate to at present, they are the natural point of comparison. On the present evidence, a mental health charity, StrongMinds, appears to be at least four times more cost-effective than at least one of GiveWell’s life-improving charities, GiveDirectly, which provides unconditional cash transfers to poor Kenyan farmers. We’re not able to say whether it is better than GiveWell’s other life-improving charities - which all, ultimately, aim to alleviate poverty - or GiveWell’s life-saving charities either. Below, we compare StrongMinds first to GiveWell’s life-improving, then to their life-saving, recommendations setting out the key uncertainty in each case. Evaluating GiveDirectly To evaluate StrongMinds and GiveDirectly, we need to convert their impact into a common unit. We use self-reported life satisfaction scores, as mentioned earlier, to make this comparison. In effect, the question becomes: which charity is more cost-effective at increasing life satisfaction? A longer treatment of how and why life satisfaction scores should be used to measure happiness can be found here. [LINK] A study of GiveDirectly indicates cash transfers increase life satisfaction by about 0.3 life satisfaction points - hereafter ‘LSPs’ - on a 10 point scale.[1] To explain, this effect would be equivalent to increasing someone's life satisfaction from 6 to 6.3 out of 10. This was measured after 4.3 months on average, but let's assume this effect lasts a whole year. We assume it is the same for everyone in the recipient household, and there are 5 people per household on average. Hence the annual LSP impact is 0.3 (LS/person) x 1 (year) x 5 (persons) = 1.5 LSPs. The average cash transfer is $750, implying a cost-effectiveness of 2 LSPs/$1000. Negative spillovers However, this may well overstate the effectiveness of cash transfers. It accounts only for the life satisfaction increase of recipients. Research into GiveDirectly has suggested that their cash transfers, while making some people wealthier (and so more satisfied with life) have negative spillovers: it makes non-recipients less satisfied with their lives. As Haushofer, Reisinger and Shapiro (2015, p1) state: The decrease in life satisfaction induced by transfers to neighbors more than offsets the direct positive effect of transfers, and is largest for individuals who did not receive a direct transfer themselves.[2] Relative income effects
While this result may surprise the reader, as Clark (2017) notes ‘there is considerable evidence from a variety of sources to suggest that well-being is a function of relative income’.[3] In other words, your absolute level of wealth matters much less than whether you are richer or poorer than your peers. Hence the comparison effects found in the GiveDirectly study are not unusual, but the typical result we would expect from other research on happiness. Indeed, Clark et al. (2018), using developed country data, finds doubling one person’s income causes a reduction in life-satisfaction in that individual’s peer group that is nearly identical in size to the life satisfaction gain to the individual themself [4]. What is novel about the 2015 GiveDirectly result is that it finds this relative effect at such a low level of absolute wealth. We might hope these negative spillovers would dissipate eventually and, over the long run, cash transfers would be effective in increasing life satisfaction. A 2018 study on the long-term (3 year) effects of GiveDirectly by Haushofer and Shapiro (2018, p. 22) finds recipients, compared to non-recipients in distant villages, have 40% more assets but that recipients do no better on a psychological well-being index.[5] Hence, the effect seems to be in the short-term. A surprising inconsistency Confusingly however, GiveWell claim a more recent ‘general equilbrium’ (GE) study of GiveDirectly, to which GiveWell have been given private access to a draft, shows there is: No evidence of across village spillover effects on household asset ownership or subjective well-being. They find a positive spillover effect on subjective well-being of ineligible households within treatment villages.[6] We note this (more recent) finding is inconsistent both with the 2015 paper, and the more general finding of relative income noted by Clark (et al.) above. Given this inconsistency, and without access to the draft, it’s unclear how to update our views accordingly. Let’s return to this concern in a moment.
How does StrongMinds compare? There isn’t research on StrongMinds which has directly measured its impact in terms of life satisfaction, so we estimate this using other available information, explained in this endnote.[7] We infer that the treatment effect is an increase of 0.8 LSPs per person (to 1 decimal point; modelled either as 0.2 LSPs per person per year for 4 years, or 0.2 LSP in the first year with a 75% annual retention thereafter). StrongMinds say their per-participant costs are $102 (StrongMinds Q1.2018 report). That suggests the impact is 8 LSPs/$1000 (to 1 d.p.). The cost-effectiveness evaluation between StrongMinds and GiveDirectly is not sensitive to inclusion of the negative spillovers as, even if we assume GiveDirectly has no negative spillover effects, then StrongMinds would still seem to be around four times more cost-effective (2 vs 8 LSPs/$1,000). Note that, if we were take the view that its negative spillovers were just as big as its positive impacts, then GiveDirectly would not seem to increase aggregate life satisfaction at all and anything with a positive impact would be more cost-effective. Comparisons with other life-improving charities In contrast, the analysis of whether StrongMinds is cost-effective than GiveWell’s other life-improving charities - namely SCI, Deworm the World, SightSavers and END - is sensitive to how strong we think the negative spillover effect of increasing wealth is. To see this, we must first recognise that these other charities could have negative spillovers: they all eventually increase the wealth of their beneficiaries. While those organisations provide deworming interventions, the vast majority of the benefit of deworming, according to GiveWell, comes not from reducing the physical discomfort the worms cause, but from the fact dewormed children earn more in later life.[8] If it’s generally true that making some wealthier (and so more satisfied), reduces the satisfaction by others to at least extent, then this concern will apply there too. According to GiveWell, all these other charities are at least four times more cost-effective than GiveDirectly. Deworm the World is rated as 18.3 times better (the highest), and END 5.5 (the lowest). If StrongMinds is only 4 times more cost-effective on GiveDirectly then, assuming there are no negative spillovers, all these other charities would be better (assuming GiveWell’s analysis is otherwise correct). If, on the other hand, these charities have negative spillovers large enough to entirely cancel out their positive impacts, then StrongMinds will be better. Thus, the size of negative spillovers is highly important, which is why it is frustrating the evidence is inconsistent (at least, once we account for the latest, unpublished draft of GiveDirectly seen by GiveWell). Further research required We are not able to offer a resolution to this issue here. More work is needed to:
More generally, the possibility that mental health interventions could be more cost-effective than the best poverty-alleviation interventions should be taken seriously. Endnotes
[1] All the calculations for the cost-effectiveness of both GiveDirectly and StrongMinds, including references, can be found in the following spreadsheet: Michael Plant, (2018). “Life Satisfaction Impact of Treating Mental Health vs Alleviating Poverty.” [2] Haushofer, J., Reisinger, J. and Shapiro, J. (2015). “Your Gain Is My Pain: Negative Psychological Externalities of Cash Transfers, Working Paper.” [3] Clark, A. E. (2017). “Happiness, Income and Poverty.” International Review of Economics, 64(2), 145–58. [4] Andrew Clark, Sarah Flèche, Richard Layard, Nattavudh Powdthavee and George Ward. The Origins of Happiness: The Science of Well-Being over the Life Course, 2018. [5] Haushofer, J. and Shapiro, J. (2018). “The Long-term Impact of Unconditional Cash Transfers: Experimental Evidence from Kenya.” [6] Available at: https://blog.givewell.org/2018/05/04/new-research-on-cash-transfers [7] Plant, M. (2018). “Life Satisfaction Impact of Treating Mental Health vs Alleviating Poverty.” Note the impact is nearly identical whether we assume a constant benefit of four years (i.e. the inference for Wiles et al. 2016) or if we assume a 75% annual retention of benefits, which is the method taken by Halstead and Snowden, “Cause Area Report: Mental Health”, Founders Pledge. [8] GiveWell "2018 Cost-effectiveness analysis -version 4" states that 2% of the cost-effectiveness of the deworming charities (DtW, SCI, Sightsavers, END) comes from 'short-term health effects' and 98% from 'eventual income and consumption gains'. See 'Results' tab in this spreadsheet. |