StrongMinds Comprehensive Summary

Last updated: November, 2023

Problem

Depression is strongly linked with lower subjective wellbeing (Clark et al., 2017), but mental health services are significantly underfunded in low- and middle-income countries (LMICs)¹Mental and addictive disorders form between 7% and 13% of the global disease burden (Vigo et al., 2019) and their relative share has grown in recent years (Rehm & Shield, 2019), but only receive 1% of governmental health budgets in LMICs (Vigo et al., 2019) and 0.3% of health-directed international assistance (Liese et al., 2019). . As many as 94% of people with depression in LMICs do not receive adequate treatment (Thornicroft et al., 2017).

Intervention

Group interpersonal therapy (g-IPT) treats depression by increasing social support, decreasing the stress of social interactions, and improving communication skills. When g-IPT is facilitated by community members who receive a few weeks of training – instead of psychotherapists – it is called “task-shifted”.  Task-shifted g-IPT is the intervention that the WHO recommends for cost-effectively tackling mental health in LMICs.

Organisation

StrongMinds provides g-IPT in Uganda and Zambia through several programmes. The core programme is task-shifted g-IPT delivered in person in 90-minute weekly sessions over multiple weeks. Historically, the programme was delivered over 12 weeks, but by 2024 StrongMinds plans to reduce this to 6 weeks with more focused content ² Individuals are divided into groups depending on which coping strategy appears most relevant to their case: increasing social support, decreasing the stress of social interactions, or improving communication skills (StrongMinds personal communication, 2023). The official programme is sometimes followed by a longer unofficial phase where the groups continue to meet and support one another without the presence of an official facilitator (StrongMinds, 2017). . In recent years, StrongMinds has largely shifted away from directly deploying its programmes to training non-government and government organisations to deliver g-IPT³StrongMinds primarily provides psychotherapy through partner organisations (62%). StrongMinds (and partners) mainly operate in Uganda (60%) and Zambia (37%). The partners StrongMinds works with are 66% government affiliated workers: community health workers (56%) and teachers (10%). The remaining 34% of the partnerships are through a variety of NGOs..

Evaluation

Methods

To estimate the impact of StrongMinds, we combined general evidence of the impact of psychotherapy on subjective wellbeing in low- and middle-income countries with evidence directly evaluating the impact of StrongMinds’ program. 

To do this, we first conducted a systematic review to gather general evidence on the impact of psychotherapy⁴We defined psychotherapy as an intervention with a structured, face-to-face talk format, grounded in an accepted and plausible psychological theory, and delivered by someone with some level of training. on subjective wellbeing in low- and middle-income countries. We collected 74 RCTs with a sample of 28,491 unique participants. 

We then estimated the general effect of psychotherapy using a meta-analysis⁵For each psychotherapy intervention, we extract every follow-up over time for every outcome measure that fits our inclusion criteria. This means that there is dependency (i.e., non-independence) between the effect sizes within an intervention between outcomes collected for a certain timepoint, and between timepoints for a given intervention. We select a 4-level (random effects) model. We do so because there is dependency between the multiple effect sizes in the different outcomes (level 3) and the different interventions (level 4) in the structure of our data.. We measured the effect from each study using Hedges’ g standardised mean differences, which is interpreted as the improvement in standard deviations⁶Our preferred measures of wellbeing are self-reported life satisfaction or happiness, but many of the studies we found used measures of affective mental health (MHa; i.e., depression, general anxiety, or general distress). Standard deviations of MHa were converted to standard deviations of life satisfaction using a 1:1 conversion. Across several sources of evidence, we find that SWB changes tend to be between 9% and 15% larger than MHa changes, so using MHa as a 1:1 proxy for SWB will not inflate findings (indeed, the contrary would be true). To be conservative, we do not apply an adjustment here. See McGuire et al. (2023c). . 

Some studies measured the impact of psychotherapy at different time points. We used this information to estimate the total effect of psychotherapy over time using a meta-regression model⁷Meta-regressions are like regressions, except the data points (i.e., dependent variables) are effect sizes weighted according to their precision and the explanatory variables are study characteristics. Meta-regressions allow us to explore why effects might differ between studies; in this case, we examine how effects differ depending on the length of time to the follow-up.that assumes the benefits decline at a constant rate over time.

After estimating the total recipient effect over time, we adjust our estimate down to account for possible outliers in the data and publication bias more generally. 

Although no randomised controlled trials (RCTs) have evaluated the impact of StrongMinds’ programmes directly⁸The only completed randomised controlled trials (RCTs) of StrongMinds is the long anticipated RCT by Baird and co-authors, which has been reported to have found a “small” effect. However, this study is not published, so we are unable to include its results and unsure of its exact details and findings. A second RCT is also currently underway., there is one RCT evaluating a very similar task-shifted g-IPT programme (Bolton et al., 2003; Bolton et al., 2007)⁹Given that the Barid RCT is reported to have found “small” effects, we discount the effect estimates by Bolton by 95% as a placeholder for the small effect. This is a very large discount. . We use Bayesian updating to combine this StrongMinds specific evidence with the general psychotherapy evidence¹⁰This approach allows us to weigh the general evidence and charity-specific evidence quantitatively, rather than relying on our subjective judgments. . To do this, we use the meta-analytic effect as our ‘informed prior’¹¹When calculating our informed prior, we remove StrongMinds specific evidence from the meta-analysis to make it independent. We then add moderators to our model to predict what would be the initial effect (the effect post-intervention, or the intercept) for an intervention with the same characteristics as StrongMinds., and the StrongMinds RCT as our ‘new data’¹²We use a similar model specification to the one we used for the prior, but only use the StrongMinds-specific evidence. . Combining the prior and the new data gives us our estimated total effect on the recipient. 

Finally, because improving one person’s mental health can benefit others who are close to that person, we also estimated the spillover effects on household members. Thus, the total household effect includes the estimated impact on recipients and household members over time.

Impact

We estimate StrongMinds has a total effect of 1.31 WELLBYs on the individual recipient of StrongMinds. We estimate that recipients live with four other household members on average, and that each receives 16% of this benefit, resulting in a total household effect of 2.09 WELLBYs per treatment. 

However, we think that effects reported in psychotherapy studies are inflated by 10% due to a statistical issue known as ‘range restriction’¹³‘Range restriction’ may occur in psychotherapy trials that select participants based on a threshold of mental health conditions, as this restricts the distribution of scores, which then reduces the variance in the groups, which then inflates effect size estimates. , so we add a final discount of 10%, resulting in a final estimated total household effect of 1.88 WELLBYs per treatment.

Cost

Based on StrongMinds’ expenditures in 2023, we estimate that it costs StrongMinds $63 to provide treatment to one person.

Cost-effectiveness

The total cost-effectiveness of StrongMinds is $63 / 1.88 WELLBYs = $33 per WELLBY. This means for every $1,000 donated to StrongMinds, the organisation provides 30 WELLBYs.

To quantify the statistical uncertainty in our results, we simulated StrongMinds’ cost-effectiveness thousands of times by varying key parameters¹⁴These include: the initial effect and  the annual decay rate (which allows for variation in the duration of effects), the household spillover, as well as the household size.to see how the results might differ across a range of reasonable inputs (e.g., Monte Carlo simulations)¹⁵Monte Carlo simulations allow us to treat inputs in a cost-effectiveness analysis (CEA)—often merely stated as point estimates—as distribution. Thereby, this allows us to communicate a range of probable values (i.e., uncertainty around the point estimates). See our cost-effectiveness methodology for more detail.. This is illustrated in Figure 1 below.

Figure 1. Density plot of the quantified uncertainty around StrongMinds’ cost-effectiveness

Note. The diamond represents the central estimate of cost-effectiveness (30 WELLBYs per $1,000). The shaded area is a probability density distribution and the whiskers represent the 95% confidence interval.

Quality of evidence

We think the quality of evidence supporting the effect of psychotherapy interventions is moderate. The primary reasons for this are:

• The effect of psychotherapy is based on a large number of studies (74 RCTs) and participants (n = 28,491), and this evidence is of moderate quality. However, there is a small amount of charity-specific evidence for StrongMinds (1 RCT of a similar programme, 1 RCT forthcoming, and 1 underway).
• There are only 5 RCTs for household spillover effects in LMICs (compared to the many more RCTs for the individual effects, see above). This increases our uncertainty about the overall effect.
• The overall evidence base shows signs of publication bias. While we used methods to correct for publication bias¹⁶Publication bias is “when the probability of a study getting published is affected by its results” (Harrer et al., 2021). We adjust for publication bias using an average of several state-of-the-art publication bias correction models. See McGuire, et al. (2023) Section 5 for more details. , it is not possible to know for sure if all the bias has been accounted for.

Depth of our analysis

Moderate to high. We have completed a substantial amount of work to date, but  there is still room for further analysis.

Funding need

StrongMinds is planning for rapid growth. In order to do this, it needs to raise a further $19 million over the next two years (i.e., 2023-2024). The funding will be used primarily to support ongoing work to deliver services to more women. Some funds will also be used to help the organisation scale, and to launch an RCT evaluating its effectiveness.

Conclusion

StrongMinds is one of the most cost-effective life-improving charities we have evaluated in-depth so far. We have moderate confidence in the evidence, so it is possible that future research could update our evaluation. Overall, we think StrongMinds is a cost-effective way to improve global wellbeing, and is a particularly good fit for donors who value improving lives.

FAQs

• Does psychotherapy actually work?
• Psychotherapy has been shown to be an effective treatment against depression: as good or better than the alternative of drug treatment in many meta-analyses (Cuijpers et al., 2019). Cuijpers et al. (2016) found that interpersonal therapy does not differ in efficacy from other therapy forms, and Cuijpers et al. (2018) supports the idea that psychotherapy is at least as effective in non-Western countries as Western countries. Singla et al. (2017) also found psychological treatments deployed in LMICs to be an effective way to treat depression.
• How does psychotherapy work?
• Psychotherapy works by providing people with a safe space to process their thoughts and feelings, and learn adaptive skills to correct their maladaptive thoughts, emotional processing, behaviours, and social interactions. IPT is a time-limited intervention that focuses on addressing stressful life events and interpersonal challenges, while also helping patients connect with new social supports and improve existing relationships (Weissman et al., 2007; Ravitz & Watson, 2014).
• Is poor mental health only a problem for high-income countries?
• No, the idea of the ‘happy poor’ is not accurate: there are about as many people suffering from mental health problems in LMICs as in HICs (Our World in Data, 2022). Furthermore, treatment for mental health is especially underfunded in LMICs, and even when it is funded, it can vary considerably in quality¹⁷For example, it’s still relatively common for people with severe mental illnesses in LMICs to be restrained in solitary confinement and given no other treatment. (Walker et al., 2021).
• What’s the relationship between mental health and poverty?
• Poverty and mental health have a complex bidirectional relationship, where each can worsen each other (Ridley et al., 2020). However, one can be poor without being depressed, and vice-versa. Notably, if mental health problems occur because of maladaptive thoughts, behaviours, and social interactions, then these are causes that can be independent from poverty.
• Does psychotherapy just make people accept poverty?
• No, psychotherapy helps people change maladaptive thoughts, behaviours, and relationships. These can worsen poverty and may not be addressed by alleviating financial burdens – instead, many of these symptoms need specific interventions targeting them, such as psychotherapy. Additionally, mental health interventions – and presumably psychotherapy as well – can improve economic outcomes (Lund et al., 2022)¹⁸In fact, the authors find in a quick calculation that mental health interventions are more cost-effective than cash transfers at improving economic outcomes (p. 32)..
• Shouldn’t we just give people cash, so they can decide whether to buy therapy – or something else – for themselves?
• One perspective that’s common among economists is that people are the best judges of what’s good for them. While this may be true in theory, in the real world, people have imperfect information and inadequate options, and other barriers (such as stigma around seeking mental health treatment) may get in the way. The advantage of the WELLBY approach is that, through people’s self-reports, we get evidence on what actually makes a difference to their lives as they live them – not just what they expect would matter. Using these self-reports, our analyses indicate that the effect of providing a depressed person with therapy improves happiness more than giving them, or another non-depressed person, the cash equivalent to the cost of that programme.
• What if people are just saying they feel better because they think it’ll benefit them, or someone else, materially?
• This is a concern about a type of response bias called ‘experimenter-demand effects’, where respondents shift their behaviour in response to their guess of what the study is about. The available evidence we’ve found suggests this is not a major concern, and it likely applies to all research with self-reported outcomes (i.e., it is not specific to psychotherapy). However, the research on the question has been sparse and may not address all concerns, so we’d be interested in seeing more research on this topic. See our more detailed explanation.
• What’s the relationship between HLI and StrongMinds?
• HLI and StrongMinds are completely independent organisations. We are not paid by StrongMinds, and would never accept money to recommend any organisation.

Is your question missing from this list? Contact us at hello@happierlivesinstitute.org.

Our reports so far

We first identified that mental health was a key global priority in our Mental health cause area report. We then evaluated the impact of psychotherapy interventions in the Psychotherapy cost-effectiveness analysis. We identified StrongMinds as a promising charity in the Mental health programme evaluation. We assessed the cost-effectiveness of StrongMinds in the StrongMinds cost-effectiveness analysis. We compared the cost-effectiveness of psychotherapy/StrongMinds to cash transfers/GiveDirectly in our Psychotherapy vs cash transfer comparison. We updated our analysis to include the impact of household spillovers in our household spillover effects report. 

In November 2023, we published a substantial update to our psychotherapy analysis and StrongMinds evaluation. This update involved systematically reviewing the literature, updating the evidence for the individual and household effects, correcting for publication bias, and combining the general and charity specific evidence using Bayesian methods. Note that this is a working paper and will be updated over time.