Donating money, buying happiness
November 2023: Update to our analysis
We have made a substantial update to our psychotherapy report. This 90+ page report details new methods as well as an updated cost-effectiveness analysis of StrongMinds and a novel cost-effectiveness analysis of Friendship Bench. We now estimate StrongMinds to produce 30 WELLBYs per $1,000 donated (or 3.7 times more than GiveDirectly cash transfers).
See our changelog for previous updates.
1. Background and Summary
In order to do as much good as possible, we need to compare how much good different things do in a single ‘currency’. At the Happier Lives Institute (HLI), we believe the best approach is to measure the effects of different interventions in terms of ‘units’ of subjective wellbeing (e.g. self-reports of happiness and life satisfaction).
In this post, we discuss our new research comparing the cost-effectiveness of psychotherapy to cash transfers. To see the advantage of making comparisons in terms of subjective wellbeing, it will help to flag some alternative methods. We could assess the effect they have on wealth, but this would fail to capture the benefits of psychotherapy. It’s implausible to think that treating depression is only good insofar as it helps you to earn more. We could assess their effects using standard measures of health, such as a Disability-Adjusted Life-Year (DALY), but it’s similarly mistaken to think that alleviating extreme poverty is only good insofar as it helps you to be healthier. We could make some arbitrary assumptions about how much a given change in income and DALYs each contribute to wellbeing; this would allow us to ‘trade’ between them. But this would just be a guess and could be badly wrong. If we measure the effects on subjective wellbeing, how individuals feel and think about their lives, we can provide an evidence-based comparison in units that more fully capture what we think really matters.
Efforts to work out the global priorities for improving subjective wellbeing are relatively new. Earlier work conducted by HLI’s Director, Michael Plant, has indicated that this approach might reveal different priorities, with mental health standing out as one area that is crucial for subjective wellbeing. Plant’s (2018, 2019 ch. 7) prior back-of-the-envelope calculations indicated that StrongMinds, a mental health charity which treats women with depression in Africa, could be as cost-effective as GiveWell’s top charity recommendations.
These initial findings motivated us to do a much more rigorous analysis of the same interventions in terms of subjective wellbeing, so we undertook meta-analyses in each case. These aimed to address three questions:
1. Is assessing cost-effectiveness in terms of subjective wellbeing feasible: are there enough data that we can make these sorts of comparisons without making major assumptions to fill in the blanks?
2. Is this approach worthwhile: does it indicate new or different priorities?
3. Does this specific comparison between cash transfers and psychotherapy indicate that donors and decision-makers should change the way they allocate their resources, assuming they want to do the most good?
Our research focused specifically on studies in low- and middle-income countries (LMICs). Our meta-analyses consist of 45 studies (total participants=116,999) for cash transfers and 39 studies (total participants=29,643) for psychotherapy. We assessed their cost-effectiveness using measures of subjective wellbeing (SWB) and mental health (MHa), which we combined (see section 2.2 for more details).
We estimate that the average psychotherapy intervention in our dataset would be 12 times (95% CI: 4, 27) more cost-effective than the average monthly cash transfer. We used this wider evidence base to estimate the cost-effectiveness of two charities that are highly effective at implementing each type of intervention: Give Directly (which provides $1,000 lump-sum cash transfers) and StrongMinds (which provides psychotherapy). When we repeated the analysis for these specific charities, adjusting for how they differed to the average intervention of their type, we found that StrongMinds is 12 times (95% CI: 4, 24) more cost-effective than GiveDirectly.
A few aspects of this research are worth highlighting. First, this is an empirically substantial, meta-analytic approach compared to the back-of-the-envelope calculation in Plant (2019), therefore we can be more confident in our pooled estimates. Second, meta-analyses only tend to report the initial effects of the intervention. We estimated the effects over time and the average costs which means we can calculate cost-effectiveness (which is not possible just with initial effects). Third, we used Monte Carlo simulations to generate our estimates. Rather than using a single number for each part of the cost-effectiveness analysis (e.g. a single number for cost) Monte Carlo simulations involve specifying a range of possible values for each input, then recalculating the results over and over, using a randomly drawn number for each input. This allows us to better account for uncertainty and to assign confidence intervals to our final cost-effectiveness estimate.
What are the main conclusions to draw from this new analysis? Let’s return to the three stated aims in turn.
First, such an analysis was feasible. There was more than enough data for a meta-analysis, although we did have to pool ‘classic’ subjective wellbeing measures (happiness and life satisfaction) with mental health measures (see section 2.2). Aggregating in this way has some precedent (Banerjee et al., 2020; Egger et al., 2020; Haushofer et al., 2020; Luhmann et al., 2012).
Second, performing cost-effectiveness analysis in terms of subjective wellbeing does seem worthwhile. We found that StrongMinds was 12 times more cost-effective than GiveDirectly. This 12x multiple puts StrongMinds roughly on a par with GiveWell’s top-rated life-improving charities – these mostly focus on deworming. Although GiveWell recommends GiveDirectly, it estimates it to be 10-20x less cost-effective than it’s top deworming charities, in terms of years of doubled consumption. Hence, if we take these multiples at face value, StrongMinds is in the same ballpark as the top-rated life-improving interventions.
The third aim of this research was to determine if donors should allocate their resources differently. The picture here is more complicated. As noted, taking the multiples at face value, StrongMinds is on a par with GiveWell’s top life-improving interventions, it is not more cost-effective. However, we have not yet looked deeply into deworming ourselves and we would not be surprised if doing so indicated substantially different conclusions.
We suspect that deworming will turn out to be less effective than GiveWell currently estimates (relative to cash transfers and in terms of subjective wellbeing). In GiveWell’s model of the effect of deworming, its benefits come almost entirely from improved educational attendance which, in turn, causes a small annual increase in income for many years. Hence, although it is a health intervention, the effect comes through reducing poverty. One issue is that a small, annual income increase may have less effect on wellbeing compared to a single, potentially life-changing lump sum, even if the total increase in lifetime income is the same (the $1,000 lump sum given by GiveDirectly is equivalent to a year of household income). We have raised this concern with GiveWell and you can read the relevant section of our conversation notes here. In our meta-analysis of cash transfers, we find that monthly transfers are about 2.5 times less cost-effective than large lump sums, which suggests that a non-trivial downward adjustment to deworming (relative to cash transfers) may be appropriate. However, it’s also possible we will conclude that deworming has a more substantial and positive impact on health than GiveWell currently models, which would raise its cost-effectiveness.
For any donors who need to make an immediate decision, our weakly-held view is that StrongMinds is more cost-effective than any of GiveWells’ top life-improving charities. We plan to conduct further analysis of deworming over the next few months and expect to be able to make a more confident claim then.
Given all this, we believe it is both feasible and worthwhile to investigate more interventions in terms of subjective wellbeing. As well as deworming, we have several interventions that we’re excited to look at. Based on our shallow analyses so far, we think there’s a good chance these interventions will be more cost-effective than psychotherapy. They run the gamut of ‘micro’ interventions, such as providing clean water, cement flooring, and cataract surgery, to ‘macro’ interventions, such as advocating to integrate wellbeing metrics in public policy to improve institutional decision-making.
The following sections of this post go on to summarise our three in-depth cost-effectiveness analyses: one on cash transfers (which includes our analysis of GiveDirectly); one on psychotherapy in general; and one on StrongMinds specifically. Our results are summarized below in Figure 1, where the lines with a steeper slope reflect a higher cost-effectiveness in terms of MHa and SWB improvements. Each point is a single run of a Monte Carlo simulation for the intervention, mapping the uncertainty around our estimates of the effects and the costs. Although our estimates for psychotherapy and StrongMinds are more uncertain than cash transfers, we still estimate that they are more cost-effective at improving subjective wellbeing.
Figure 1: Cost-effectiveness of psychotherapy compared to cash transfers
Note: We assume that GiveDirectly CTs are as effective as other lump-sum CTs, but we think GiveDirectly probably has lower operating costs. This is why we do not display them separately.