The Happier Lives Institute searches for the most cost-effective ways to improve well-being.
We start with broad-brush research into global problems such as mental health and pain. We then seek out the most cost-effective interventions for alleviating those problems and the best organisations that provide those interventions.
Our analysis draws on previous academic work in psychology and economics, as well as our
own research into the nature and measurement of well-being.
Which interventions have we looked at so far?
Our first two meta-analyses looked at the cost-effectiveness of cash transfers and
psychotherapy in low- and middle-income countries. We used this wider evidence base to estimate the cost-effectiveness of two charities that appear highly effective at implementing each one.
What have we found?
We estimate that StrongMinds (which provides psychotherapy) is 12 times (95% CI: 4, 24) more cost-effective than Give Directly (which provides $1,000 lump-sum cash transfers).
This page provides the background and summary of our work so far. We will continue to update this page with new information as we publish more results. You can also read and contribute to further discussion on the Effective Altruism Forum.
Authors: Joel McGuire and Michael Plant
Last updated: October 2021
1. Background and summary
2. Cost-effectiveness analyses (CEAs)
2.1 Cash transfers
2.3 Strong Minds
3. Conclusions and next steps
1. Background and summary
In order to do as much good as possible, we need to compare how much good different things do in a single ‘currency’. At the Happier Lives Institute (HLI), we believe the best approach is to measure the effects of different interventions in terms of ‘units’ of subjective well-being (e.g. self-reports of happiness and life satisfaction).
In this post, we discuss our new research comparing the cost-effectiveness of psychotherapy to cash transfers. Before we get to that comparison, let's consider the advantages of doing it in terms of subjective well-being compared to alternative methods.
Efforts to work out the global priorities for improving subjective well-being are relatively new. Nevertheless, the recent push to integrate well-being in public policy-making in countries such as Scotland and New Zealand, as well as the reach of publications such as the World Happiness Report (which started in 2012), indicates that this is a viable approach.
Earlier work conducted by HLI’s Director, Michael Plant, suggested that using subjective well-being might reveal different priorities for individuals and organisations seeking to do the most good, with mental health standing out as one area that is crucial and potentially neglected. Plant’s (2018, 2019 ch. 7) prior back-of-the-envelope calculations indicated that StrongMinds, a mental health charity that treats women with depression in Africa, could be as cost-effective as GiveWell’s top charity recommendations.
These initial findings motivated us to do a much more rigorous evaluation of the same interventions in terms of subjective well-being; we undertook meta-analyses in each case. These aimed to address three questions:
Our research focused specifically on studies in low- and middle-income countries (LMICs). Our meta-analyses consist of 45 studies (total participants=116,999) for cash transfers and 39 studies (total participants=29,643) for psychotherapy. We assessed their cost-effectiveness using measures of subjective well-being (SWB) and mental health (MHa), which we combined (see section 2.2 for more details).
We estimate that the average psychotherapy intervention in our dataset would be 12 times (95% CI: 4, 27) more cost-effective than the average monthly cash transfer. We used this wider evidence base to estimate the cost-effectiveness of two charities that are highly effective at implementing each type of intervention: Give Directly (which provides $1,000 lump-sum cash transfers) and StrongMinds (which provides psychotherapy). When we repeated the analysis for these specific charities, adjusting for how they differed to the average intervention of their type, we found that StrongMinds is 12 times (95% CI: 4, 24) more cost-effective than GiveDirectly.
A few aspects of this research are worth highlighting. First, this meta-analytic approach is much more empirically rigorous than the back-of-the-envelope calculations made in Plant (2018, 2019); we can take these results seriously, rather than as merely indicative. Second, meta-analyses only tend to report the initial effects of the intervention. We estimated the effects over time and the average costs which means we can calculate cost-effectiveness (which is not possible just with initial effects). Third, we used Monte Carlo simulations to generate our estimates. Rather than using a single number for each part of the cost-effectiveness analysis (e.g. a single number for cost) Monte Carlo simulations involve specifying a range of possible values for each input, then recalculating the results over and over, using a randomly drawn number for each input. This allows us to better account for uncertainty and to assign confidence intervals to our final cost-effectiveness estimate.
What are the main conclusions to draw from this new analysis? Let’s return to the three stated aims in turn.
First, such an analysis was feasible. There was more than enough data for a meta-analysis, although we did have to pool ‘classic’ subjective well-being measures (happiness and life satisfaction) with mental health measures (see section 2.2). Aggregating in this way has some precedent (Luhmann et al., 2012; Banerjee et al., 2020; Egger et al., 2020; Haushofer et al., 2020).
Second, performing cost-effectiveness analysis in terms of subjective well-being does seem worthwhile. We found that StrongMinds was 12 times more cost-effective than GiveDirectly; some grantmakers, including Open Philanthropy, consider GiveDirectly to be "the bar" for allocating funds. What's more, this 12x multiple puts StrongMinds roughly on a par with GiveWell’s top-rated life-improving charities - these mostly focus on deworming. Although GiveWell recommends GiveDirectly, it estimates it to be 10-20x less cost-effective than it’s top deworming charities, in terms of years of doubled consumption. Hence, if we take these multiples at face value, StrongMinds is in the same ballpark as the top-rated life-improving interventions.
The third aim of this research was to determine if donors should allocate their resources differently. The picture here is more complicated. As noted, taking the multiples at face value, StrongMinds is on a par with GiveWell’s top life-improving interventions, it is not more cost-effective. However, we have not yet looked deeply into deworming ourselves and we would not be surprised if doing so indicated substantially different conclusions.
We suspect that deworming will turn out to be less effective than GiveWell currently estimates (relative to cash transfers and in terms of subjective well-being). In GiveWell’s model of the effect of deworming, its benefits come almost entirely from improved educational attendance which, in turn, causes a small annual increase in income for many years. Hence, although it is a health intervention, the effect comes through reducing poverty. One issue is that a small, annual income increase may have less effect on well-being compared to a single, potentially life-changing lump sum, even if the total increase in lifetime income is the same (the $1,000 lump sum given by GiveDirectly is equivalent to a year of household income). We have raised this concern with GiveWell and you can read the relevant section of our conversation notes here. In our meta-analysis of cash transfers, we find that monthly transfers are about 2.5 times less cost-effective than large lump sums, which suggests that a non-trivial downward adjustment to deworming (relative to cash transfers) may be appropriate. However, it's also possible we will conclude that deworming has a more substantial and positive impact on health than GiveWell currently models, which would raise its cost-effectiveness.
For any donors who need to make an immediate decision, our weakly-held view is that StrongMinds is more cost-effective than any of GiveWells’ top life-improving charities. We plan to conduct further analysis of deworming over the next few months and expect to be able to make a more confident claim then.
Given all this, we believe it is both feasible and worthwhile to investigate more interventions in terms of subjective well-being. As well as deworming, we have several interventions that we’re excited to look at. Based on our shallow analyses so far, we think there’s a good chance these interventions will be more cost-effective than psychotherapy. They run the gamut of ‘micro’ interventions, such as providing clean water, cement flooring, and cataract surgery, to ‘macro’ interventions, such as advocating to integrate well-being metrics in public policy to improve institutional decision-making.
The following sections of this post go on to summarise our three in-depth cost-effectiveness analyses: one on cash transfers (which includes our analysis of GiveDirectly); one on psychotherapy in general; and one on StrongMinds specifically. Our results are summarized below in Figure 1, where the lines with a steeper slope reflect a higher cost-effectiveness. Each point is a single run of a Monte Carlo simulation for the intervention, mapping the uncertainty around our estimates of the effects and the costs. Although our estimates for psychotherapy and StrongMinds are more uncertain than cash transfers, we still estimate that they are more cost-effective at improving subjective well-being.
Figure 1: Cost-effectiveness of psychotherapy compared to cash transfer
2. Cost-effectiveness analyses (CEAs)
2.1 Cash transfers (see the full CEA report)
Cash transfers (CTs) are direct payments made to people living in poverty. They have been extensively studied and implemented in low- and middle-income countries (LMICs) and offer a simple and scalable way to reach people suffering from extreme financial hardship.
Our cash transfers cost-effectiveness analysis (CEA) determines the cost-effectiveness of lump-sum CTs and monthly CTs in LMICs using standard deviation (SD) changes in subjective well-being (SWB) and affective mental health (MHa) . We use ‘affective mental health’ to refer to the class of mental health disorders and measures (e.g. PHQ9, CESD20, GHQ12, and GAD7) that relate to lower levels of affect or mood (depression, distress, and anxiety). Measures of affective mental health differ from measures of SWB because they also include questions that ask about how well someone is functioning such as the quality of someone’s appetite, sleep quality, and concentration. These factors influence, but do not constitute, someone’s mood. This analysis provides us with a well-evidenced benchmark that we can use to compare a wide range of interventions.
Much of our analysis extends a meta-analysis of the effect of cash transfers on SWB and MHa in LMICs by McGuire, Kaiser, and Bach-Mortensen (2020). That paper is the result of a collaboration between HLI researchers and academics at the University of Oxford and has been conditionally accepted for publication by Nature Human Behaviour. We think this meta-analysis summarizes the entire extant literature on how cash transfers affect SWB. Our CEA uses the same data (45 studies, 114,274 individuals in LMICs) and shares some analysis on the effectiveness of cash transfers. It expands on the meta-analysis by adding cost information and estimating the total benefit we expect a recipient of a cash transfer to accrue. Please refer to the meta-analysis for more background on cash transfers and a more thorough description of the evidence base.
Our models for estimating the effectiveness of lump-sum CTs and monthly CTs are presented in Figure 2 below. The chart on the left shows the relationship between time and the effect on SWB and MHa. As we expect, the effects decay over time. The chart on the right shows the relationship between the value of a CT and the effect on SWB and MHa. Unsurprisingly, the effect is greater for larger CTs (although the data is quite noisy). We haven’t differentiated between the effects on SWB and MHa for two reasons. Our sample of studies was too underpowered to break down the analyses by both delivery mechanism and measure type (SWB or MHa). But the point estimates suggested that the differences between the measure types were small. For instance, using only SWB measures leads to a 2% and 13% increase in the total effect for monthly and lump CTs. These seemed like sufficient reasons to keep SWB and MHa pooled for our main analysis.
Figure 2: The effect of cash transfers in relation to time and value
We present the estimated cost-effectiveness of CTs in Table 1, alongside the per-person effects and per-transfer costs. To get the confidence intervals for the effect and cost we ran Monte Carlo simulations in R where each parameter was drawn from a normal distribution based on its standard error (for regression coefficients) or standard deviations for the cost.
Table 1: Estimated cost-effectiveness of lump-sum CTs and monthly CTs
Notably, the estimated total effect of a $1,000 lump-sum CT provided by GiveDirectly is about twice as large as the effect of $1,000 transferred in monthly increments by (mostly) governments. This seems like a suspiciously large difference at first, since we do not think a priori that giving people of similar means the same sum should have dramatically different effects.
Some possible explanations we have considered include:
There are three main limitations to our analysis.
Firstly, the recipient is plausibly not the only person impacted by a cash transfer. They can share it with their partner, children, and even friends or neighbours. Such sharing should benefit non-recipients' well-being. However, it’s also possible that any benefit that non-recipients receive could be offset by envy of their neighbour’s good fortune. There appears to be no evidence of significant negative within-village spillover effects, but there is some evidence for positive within-household and across-village spillover effects. We have not included these spillover effects in our main analysis because of the large uncertainty about the relative magnitude of spillovers across interventions and the slim evidence available to estimate the household spillover effects.
Secondly, we have not extensively reviewed the evidence that explores the mechanisms through which CTs improve the SWB of their recipients. This limits our ability to estimate the effects of income gains through other means, such as deworming. For instance, we have so far found little evidence regarding how much of the benefit of a CT is contingent on recipients' relative versus absolute improvement in material circumstances. Is the benefit due to recipients making beneficial comparisons to those who didn’t receive a CT or is it because of the absolute change in the recipients’ living standards as a result of higher consumption? If the channel for CTs to improve SWB and MHa primarily runs through social comparison, then that would suggest that the effects would be smaller for an intervention that increases everyone's income in an area.
Thirdly, we have sparse data on the long term effects of CTs on SWB. There is only one study that follows up with its recipients after five years. Having long-term follow-ups of CTs is important for understanding the persistence of CTs benefit, and thus the total benefit they provide to their recipients.
2.2 Psychotherapy (see the full CEA report)
Having determined the cost-effectiveness of cash transfers in terms of subjective well-being, we can now conduct a direct comparison with the cost-effectiveness of psychotherapy.
Psychotherapies vary considerably in the strategies they employ to improve mental health, but some common types of psychotherapy are cognitive behavioural therapy (CBT) and interpersonal therapy (IPT). Our analysis does not focus on a particular form of psychotherapy. Previous meta-analyses find mixed evidence supporting the superiority of any one form of psychotherapy for treating depression (Cuijpers et al., 2019).
Instead, we focus our analysis on the average intervention-level cost-effectiveness of any form of face-to-face psychotherapy delivered to groups or by non-specialists deployed in LMICs. As before, we seek to measure the effect as the benefit they provide to subjective well-being (SWB) and affective mental health (MHa).
We extracted data from 39 studies that appeared to be delivered by non-specialists and/or to groups from five meta-analytic sources, and any additional studies we found in our search for the costs of psychotherapy. These studies are not exhaustive. We stopped collecting new studies due to time constraints and our estimation of diminishing returns. We aimed to include all RCTs of psychotherapy with outcome measures of SWB or MHa but only found studies with measures of MHa.
In Table 2, we display our estimated post-treatment effects and how long they last for the average psychotherapy intervention in our sample. The post-treatment effects are estimated to be between 0.342 and 0.611 SDs of MHa. After running further regressions, we found evidence that group psychotherapy is more effective than psychotherapy delivered to individuals which is in line with other meta-analyses (Barkowski et al., 2020; Cuijpers et al., 2019). One explanation for the superiority is that the peer relationships formed in a group provide an additional source of value beyond the patient-therapist relationship.
Table 2: Post-treatment effect and decay through time
Before we compare the total effect of psychotherapy to cash transfers, we discount interventions with relatively lower-quality evidence. We estimate that the evidence base for psychotherapy overestimates its efficacy relative to cash transfers by 11% (95% CI: 0%, 40%) because psychotherapy has lower sample sizes on average and fewer unpublished studies, both of which are related to larger effect sizes in meta-analyses.
We reviewed 28 sources that estimated the cost of psychotherapy and included 11 in our summary of the costs of delivering psychotherapy (see Figure 3 below). Nearly all are from academic studies except the cost figures for StrongMinds. The cost of treating an additional person with lay-delivered psychotherapy ranges from $50 to $659.
Figure 3: Distribution of the cost of psychotherapy interventions.
Having established estimates for the cost and the effectiveness of psychotherapy, we can now show in Table 3, that psychotherapy is estimated to be 12 times (95% CI: 4, 27) more cost-effective than cash transfers.
Table 3: Comparison of unconditional cash transfers to psychotherapy in LMICs
In Figure 4, we take the results in Table 3 and show the results of simulations that compare the cost-effectiveness of each intervention. The total effect is on the y-axis and the total cost is on the x-axis for both interventions. Each point represents a run of a simulation. The higher the slope of the line, the more cost-effective the intervention. What should be clear is that the distributions of cost-effectiveness do not overlap. However, we do not include spillover effects on the household or the community in our comparison. Spillover effects are much more uncertain for reasons we explained earlier.
Figure 4: Cost-effectiveness of psychotherapy compared to cash transfers
The following considerations and limitations are discussed in detail in our full psychotherapy CEA report:
2.3 Strong Minds (see the full CEA report)
Having established that psychotherapy appears to be more cost-effective than cash transfers in terms of subjective well-being, we now need to identify the best donation opportunities for those who wish to direct more financial resources towards this intervention. We will now explain why we think StrongMinds is likely to be one of the best opportunities of this type.
In 2019, HLI began the Mental Health Programme Evaluation Project (MHPEP) to identify the most cost-effective charities delivering psychotherapy interventions in LMICs. The first round of the evaluation began with a review of 76 interventions from the Mental Health Innovation Network, from which we identified 13 priority programmes for further evaluation. Our screening criteria included: whether the intervention is targeted at depression, anxiety or stress disorders in LMICs; whether a controlled trial has been conducted on the programme; and an initial evaluation of cost-effectiveness. You can read more about the process we followed here.
In 2020, we began reaching out to organisations that deliver one of our priority programs. Unfortunately, many of the people we approached did not respond or told us they didn’t have enough time to participate. The only organisation that was on our shortlist and did provide detailed cost information was StrongMinds, a non-profit founded in 2013 that provides group interpersonal psychotherapy (g-IPT) to impoverished women in Uganda and Zambia. As a result, StrongMinds was one of the organisations we were most excited about prior to doing any detailed analysis. Friendship Bench, another promising mental health NGO, has since offered to provide us with their cost data, but we have not been able to review it yet.
Our cost-effectiveness analysis of StrongMinds builds on previous work by Founders Pledge (Halstead et al., 2019) in three ways. First, we combined broader evidence from the psychotherapy literature with direct evidence of StrongMinds’ effectiveness in order to increase the robustness of our estimates. Secondly, we updated the direct evidence of StrongMinds’ costs and effectiveness to reflect the most recent information. Lastly, we designed our analysis to include the cost-effectiveness of all StrongMinds’ programmes. This allows us to compare the impact of a donation to StrongMinds to other interventions, such as unconditional cash transfers delivered by GiveDirectly.
We estimate the total effect of StrongMinds on the individual recipient to be an expected 1.92 SD (95% CI: 1.1, 2.8) improvement in depression scores. Figure 5 visualises the trajectory of the effects of StrongMinds’ core programme compared to the average lay- or group-delivered psychotherapy through time.
Figure 5: Trajectory of StrongMinds compared to lay psychotherapy in LMICs.
StrongMinds records their average cost of providing treatment to a person and has shared the most recent figures for each programme with us. We assume that StrongMinds can continue to treat people at levels of cost comparable to previous years. It is also worth noting that StrongMinds defines treatment as attending more than six sessions (out of 12) for face-to-face modes and more than four for teletherapy. If we had used the cost per person reached (attended at least one session), then the average cost would decrease substantially.
Having incorporated all our uncertainties into a Monte Carlo simulation, we estimate that a $1,000 donation to StrongMinds will result in a 12 SD (95% CI: 7.2, 20) improvement in depression. This range of uncertainty is similar to that of the core programme (95% CI: 8.2 to 24 SDs). This may sound very large, but we expect that $1,000 will cover the treatment of around seven people. We show the distribution of this estimate in Figure 6 below.
Figure 6: Distribution of overall cost-effectiveness of a $1,000 donation to StrongMinds
The following considerations and limitations are discussed in detail in our full StrongMinds CEA report:
Figure 7 illustrates a simulation of the comparison between psychotherapy, StrongMinds, monthly cash transfers, and GiveDirectly. Each point is an estimate given by a single run of a Monte Carlo simulation. Lines with a steeper slope reflect a higher cost-effectiveness in terms of depression reduction. Bold lines reflect the cost-effectiveness gradient of interventions, while grey lines are for reference. In this comparison, a $1,000 donation to StrongMinds is around 12 times (95% CI: 4, 24) more cost-effective than a comparable donation to GiveDirectly.
Figure 7: Cost-effectiveness of StrongMinds compared to GiveDirectly
word of caution is warranted when interpreting these results. Our comparison to cash transfers is only based on affective mental health. We were unable to find any studies that used direct measures of SWB to determine the effects of psychotherapy, so it remains to be seen whether the impacts are substantially different when using those measures instead.
Additionally, this comparison only includes the effects on the individual and it is possible that the spillover effects differ considerably between interventions. That being said, even if we take the upper range of GiveDirectly’s total effect on the household of the recipient (8 SDs), psychotherapy is still around twice as cost-effective.
In Table 4 we present the exact figures shown in the comparison of Figure 7. The key takeaway from showing the exact figures we used is to show that this result is driven mostly by the difference in the cost of each intervention, not the effectiveness.
Note: 95% confidence intervals are shown below the estimates.
3. Conclusions and next steps
We think these reports show that working out how to do the most good by using subjective well-being is feasible and worthwhile. In particular, we believe that our analysis convincingly demonstrates that high-quality mental health interventions can be as good, or better than, highly-regarded and well-evidenced economic interventions.
Of course, there are many more interventions that need to be analysed in terms of subjective well-being before a fuller picture begins to emerge, but we do want to stress to prospective donors that we consider StrongMinds to be an excellent donation opportunity if your aim is to improve global health and well-being. As of October 2021, they have a funding gap of $6.6 million for 2022. They are also planning for rapid growth (treating 50,000 in 2022, 100,000 in 2023, and 150,000 in 2024) equalling 300,000 women treated over the next three years. To fuel this growth trajectory, they will need to raise $30 million in total over the next three years.
We suspect there are other highly effective donation opportunities waiting to be discovered. We have several avenues to explore and we can do this because our method allows us to compare different outcomes with a single metric. Our research pipeline includes a range of other promising interventions to analyse (conditional on us acquiring the necessary funding). These include Friendship Bench, mental health apps, cataract surgery, cement flooring, drug liberalisation (including better access to opioids and psychedelics) as well as advocacy to integrate well-being metrics in public policy (as a replacement/complement to GDP) and to make employee well-being a central indicator of environmental, social, and corporate governance (ESG) performance.
In addition, there are a number of theoretical questions about the nature and measurement of well-being that require further investigation. For instance, in order to compare StrongMinds to GiveWell’s death-averting recommendations we must first establish the location of the ‘neutral point’ on happiness scales -- a vital, but highly contested factor in assessing the cost-effectiveness of life-saving interventions in terms of subjective well-being. For more, see our analysis using subjective well-being to compare life-saving to life-improving interventions. Further research is also required to establish the best methods for converting between different subjective well-being measures. You can learn more about our future plans in our research agenda.
If you found this post valuable, and you would like to support our future research, please consider making a donation to the Happier Lives Institute. We are actively seeking new donors to fund our future research. If you would like to discuss our research further, or are thinking about making a donation, please contact firstname.lastname@example.org.