Our approach

The goal of our charity research is to find the most cost-effective ways to improve people’s lives, and to share our recommendations with donors. Our distinctive approach is to take happiness seriously: we compare charities by how good they are at improving people’s subjective wellbeing – how people feel during and about their lives.

Specifically, we evaluate charities on a single metric called wellbeing-adjusted life years (WELLBYs). The metric is simple: one WELLBY is equivalent to a 1-point increase on a 0-10 life satisfaction scale for one year. WELLBYs allow us to impartially compare the impact of different charities, even if they address different problems, like comparing the impact of a medical intervention against the impact of cash transfers. Unlike indirect measures of wellbeing, like wealth or health, WELLBYs measure people’s wellbeing directly. This method, like any other, has limitations, but we think it is the best way of capturing what really matters – at least with our current tools.

We did not invent the WELLBY; the approach is based on decades of research in social science (Brazier & Tsuchiya, 2015; Frijters et al., 2020; Layard & Oparina, 2021; Barrington-Leigh, 2022; McGuire et al., 2022), and is endorsed by the UK Treasury (HM Treasury, 2021). However, we are the first to use this tool to work out the best opportunities for donors to improve the lives of others around the world.

Note: this page describes our approach as of 2023, which may not match our older reports. We described our approach prior to 2023 in this EA Forum blog post.

Problems, interventions, and charities*

*We define these terms in an endnote.¹A problem is an issue or opportunity that impacts wellbeing, such as mental health, pain, or poverty. Problems can often be broken down into further subproblems. For example, chronic pain and palliative care are two subproblems within pain. Because our focus is on solving these problems through donations, we and other organisations in the effective altruism community sometimes refer to problems as ‘causes’.

An intervention is a specific tool or solution to address the problem, such as group psychotherapy to improve mental health, opioids to treat pain, or cash transfers to reduce poverty.

A charity is an organisation or other entity that delivers interventions.

In some cases, the distinction between these levels of analysis are not clear cut. For example, in some cases there might only be one obvious intervention to address a problem. Or perhaps the reason the problem exists is that some promising intervention isn’t being used, so a problem area report and intervention report might be merged into a single report. We might also investigate a specific intervention that seems unusually promising before or instead of conducting a problem area report (e.g., cash transfers as an intervention to address the problem of poverty).

We follow a three-stage process to identify where additional resources can do the most good:

1. Find global problems that are important, solvable, and neglected²This is consistent with the Importance, Tractability, Neglectedness (ITN) framework adopted by many organisations within the effective altruism community..
2. Identify interventions that alleviate those problems and assess their cost-effectiveness.
3. Evaluate the best charities that deliver those interventions.

Our process starts broad and gets increasingly deep as we hone in on the most effective charities. There’s a bit of mystery, and quite a lot of discussion, about ‘cause prioritisation’ methodology (Global Priorities Institute, 2020). To be clear, these stages are not logically separate levels of analysis. Ultimately, we want to find the best actions we can take. To assess a problem, even in broad terms,  you still need to implicitly or explicitly consider the solutions to that problem. What happens is that analysis gets more thorough as we go; think of it like starting with a long list of job candidates, ruling some out, moving to a shortlist, and then looking closer at those until you determine the best one.

This enables us to explore many options quickly while spending most of our time on deep analysis of the most promising solutions.

Problem areas

Our search for the most cost-effective charities starts by looking for problems around the world that have the greatest impact on happiness but are relatively overlooked and solvable. We find problems through background research, referring to the work of other organisations in global health and wellbeing, and conversations with experts.

We conduct reviews³For examples, see our reports on lead exposure, immigration reform, pain, and mental health. to get a basic understanding of the problem area so we can evaluate it against our prioritisation criteria (described below). The goal of a review is to:

• Define the problem
• Estimate its impact on wellbeing
• Explore interventions that address the problem
• Assess the quality of the evidence for the scale of the problem and the potential interventions
• Roughly estimate the cost-effectiveness of the interventions
• Identify charities that deliver the interventions
• Assess whether there are funding gaps that could be filled

The depth of these reports vary as information is uncovered: problem areas that become less promising receive shorter reviews, while we spend more time on problem areas that are more promising⁴We typically publish reports summarising our findings, but we may skip publishing reports in cases where our initial research suggests a cause area is not promising. In the future, we plan to publish brief reports explaining why a cause area that initially seemed promising did not end up meeting our criteria to prioritise..

Prioritisation criteria

In evaluating and prioritising the most pressing problems, we consider the following factors:

Scale: How widespread and severe are the impacts on wellbeing⁵In practice, we mostly only exclude problems with very small scales; most problem areas we investigate are sufficiently large to warrant our attention. This criteria will become more relevant as our organisation influences more funding.?

Neglectedness: Does the problem seem relatively overlooked by others and/or have large funding gaps? Or, if a problem is already getting lots of attention, could donor funding be better placed elsewhere (e.g., to address other problems with more cost-effective solutions)?

Solvability: Does the problem have, or could it have, cost-effective interventions?

The idea is that additional resources will do the most good when a problem is large, neglected, and solvable. We evaluate these criteria together to make a holistic judgement⁶When possible, we assess these criteria using quantitative data. In practice, we often don’t have all the data we need, so we have to rely on subjective judgements. So, these criteria are best thought of as a framework rather than formal, quantitative criteria. about what problems to prioritise.

Interventions

After identifying the most pressing problems⁷We may also skip directly to assessing interventions if existing research has already demonstrated that an intervention meets our criteria. This was the case with our evaluation of cash transfers for alleviating material poverty (McGuire & Plant, 2021)., we then evaluate the most promising solutions. At this stage, we shift our focus onto cost-effectiveness to prioritise interventions.

The goal of an intervention report is to:

• Find and evaluate the evidence on the impact of the intervention
• Estimate the total impact of the intervention on subjective wellbeing
• Estimate the cost to deliver the intervention

This allows us to estimate the cost-effectiveness of the intervention and to assess our level of confidence in the estimate.

Finding and evaluating the evidence

In our most thorough analyses, we conduct systematic reviews to ensure we capture all relevant studies on a topic. Although this is the academic ‘gold standard’, it is time consuming and not always feasible to undertake (and sometimes, it is better to do a shallow analysis first to see if a deeper analysis is useful). So we commonly rely on more efficient methods to gather the most relevant and essential research on a topic (e.g., asking experts, snowballing, and hybrid methods). We evaluate the quality of evidence using criteria adapted from the GRADE approach for systematic reviews⁸The GRADE approach is a widely adopted systematic method for evaluating the quality of evidence and making recommendations in healthcare. We plan to follow these methods as of October 2023. Prior to this, we assessed quality of evidence informally (i.e., without a formal evaluation framework)..

Estimating impact on wellbeing

Ideally, we use meta-analyses to combine all the available evidence on an intervention⁹For example, see our evaluations for psychotherapy and cash transfers (McGuire & Plant, 2021d).. In some cases where only a few or single studies exist on a topic, we may conduct original secondary analysis of the study data¹⁰For example, see our evaluation of deworming (Dupret et al., 2022)..

When possible, we estimate the impact of the intervention over time, and we also include the impact of the intervention on others (i.e., spillover effects on other members of the household). Together, these methods help us better estimate the overall effect of the intervention, rather than just the short-term impact on individuals.

Estimating costs

We try to estimate the general expected costs for organisations to deliver the intervention. These estimates might be based on publicly available financial data of existing organisations, or estimates based on generic costs (if they are known). The goal is to get a rough estimate, which will be further refined when researching specific organisations.

To learn more, see our page on cost-effectiveness analysis methodology.

Charity evaluation

After identifying the most promising interventions to the most pressing problems, we then search for the best charities implementing those interventions.

Our aim is to find the charities that are the most cost-effective, as costs per treatment and effectiveness can vary based on how a charity implements the intervention. To estimate the charity’s impact, we take our intervention-level impact estimates as our starting point: the thought is that if charity A and charity B are implementing the same type of intervention equally well, they should have the same effectiveness. We then consider making adjustments based on any notable differences in how the charity delivers the intervention. To estimate the cost per treatment, we use historical cost figures from the charity (with adjustments for any notable expected future changes). Finally, we combine these figures to estimate the total cost-effectiveness in WELLBYs.

See more details on this process on our page on our cost-effectiveness analysis methodology.

In addition to our cost-effectiveness model, we also take into account other aspects of the organisation that are important for us to be confident in recommending it to donors, but harder to quantify. These include: track record, strength of team, strength of future projects, need for funding, and transparency¹¹We plan to use these criteria as of October 2023. Prior to this, we assessed qualitative factors informally.. We evaluate these factors holistically to ensure that our recommended charities are well-suited to use additional funding effectively and adhere to high standards.

The process described above is an idealised version of our process for simplicity. In many cases, the stages in our process may overlap, or we may skip some stages. When we deviate from these methods, we describe the deviations in our reports. We see this methodology as a work in progress, and we expect we will continue to refine our process and criteria as we go. If you have feedback on our methodology, please reach out to ryan@happierlivesinstitute.org.