Method of the month: Distributional cost effectiveness analysis

Once a month we discuss a particular research method that may be of interest to people working in health economics. We’ll consider widely used key methodologies, as well as more novel approaches. Our reviews are not designed to be comprehensive but provide an introduction to the method, its underlying principles, some applied examples, and where to find out more. If you’d like to write a post for this series, get in touch. This month’s method is distributional cost effectiveness analysis.

Principles

Variation in population health outcomes, particularly when socially patterned by characteristics such as income and race, are often of concern to policymakers. For example, the fact that people born in the poorest tenth of neighbourhoods in England can expect to live 19 fewer years of healthy life than those living in the richest tenth of neighbourhoods in the country, or the fact that black Americans born today can expect to die 4 years earlier than white Americans, are often considered to be unfair and in need of policy attention. As policymakers look to implement health programmes to tackle such unfair health disparities, they need the tools to enable them to evaluate the likely impacts of alternative programmes available to them in terms of the programmes’ impact on reducing these undesirable health inequalities, as well as their impact on improving population health.

Traditional tools for prospectively evaluating health programmes – that is to say, estimating the likely impacts of health programmes prior to their implementation – are typically based on cost-effectiveness analysis (CEA). CEA selects those programmes that improve the health of the average recipient of the programme the most, taking into consideration the health opportunity costs involved in implementing the programme. When using CEA to select health programmes there is, therefore, a risk that the programmes selected will not necessarily reduce the health disparities of concern to policymakers as these disparities are not part of the evaluation process used when comparing programmes. Indeed, in some cases, the programmes chosen using CEA may even unintentionally exacerbate these health inequalities.

There has been recent methodological work to build upon the standard CEA methods explicitly incorporating concerns for reducing health disparities into them. This equity augmented form of CEA is called distributional cost effectiveness analysis (DCEA). DCEA estimates the impacts of health interventions on different groups within the population and evaluates the health distributions resulting from these interventions in term of both health inequality and population health. Where necessary, DCEA can then be used to guide the trade-off between these different dimensions to pick the most “socially beneficial” programme to implement.

Implementation

The six core steps in implementing a DCEA are outlined below – full details of how DCEA is conducted in practice and applied to evaluate alternative options in a real case study (the NHS Bowel Cancer Screening Programme in England) can be found in a published tutorial.

1. Identify policy-relevant subgroups in the population

The first step in the analysis is to decide which characteristics of the population are of policy concern when thinking about health inequalities. For example, in England, there is a lot of concern about the fact that people born in poor neighbourhoods expect to die earlier than those born in rich neighbourhoods but little concern about the fact that men have shorter life expectancies than women.

2. Construct the baseline distribution of health

The next step is to construct a baseline distribution of health for the population. This baseline distribution describes the health of the population, typically measured in quality-adjusted life expectancy at birth, to show the level of health and health inequality prior to implementing the proposed interventions. This distribution can be standardised (using methods of either direct or indirect standardisation) to remove any variation in health that is not associated with the characteristics of interest. For example, in England, we might standardise the health distribution to remove variation associated with gender but retain variation associated with neighbourhood deprivation. This then gives us a description of the population health distribution with a particular focus on the health disparities we are trying to reduce. An example of how to construct such a ‘social distribution of health’ for England is given in another published article.

3. Estimate post-intervention distributions of health

We next estimate the health impacts of the interventions we are comparing. In producing these estimates we need to take into account differences by each of the equity relevant subgroups identified in the:

  • prevalence and incidence of the diseases impacted by the intervention,
  • rates of uptake and adherence to the intervention,
  • efficacy of the intervention,
  • mortality and morbidity, and
  • health opportunity costs.

Standardising these health impacts and combining with the baseline distribution of health derived above gives us estimated post-intervention distributions of health for each intervention.

4. Compare post-intervention distributions using the health equity impact plane

Once post-intervention distributions of health have been estimated for each intervention we can compare them both in terms of their level of average health and in terms of their level of health inequality. Whilst calculating average levels of health in the distributions is straightforward, calculating levels of inequality requires some value judgements to be made. There is a wide range of alternative inequality measures that could be employed each of which captures different aspects of inequality. For example, relative inequality measures would conclude that a health distribution where half the population lives for 40 years and the other half lives for 50 years is just as unequal as a health distribution where half the population lives for 80 years and the other half lives for 100 years. An absolute inequality measure would instead conclude that the equivalence is with a population where half the population lives for 80 years and the other half lives for 90 years.

Two commonly used inequality measures are the Atkinson relative inequality measure and the Kolm absolute inequality measure. These both have the additional feature that they can be calibrated using an inequality aversion parameter to vary the level of priority given to those worst off in the distribution. We will see these inequality aversion parameters in action in the next step of the DCEA process.

Having selected a suitable inequality measure we can plot our post interventions distributions on a health equity impact plane. Let us assume we are comparing two interventions A and B, we can plot intervention A at the origin of the plane and plot intervention B relative to A on the plane.

 

 

If intervention B falls in the north-east quadrant of the health equity impact plane we know it both improves health overall and reduces health inequality relative to intervention A and so intervention B should be selected. If, however, intervention B falls in the south-west quadrant of the health equity impact plane we know it both reduces health and increases health inequality relative to intervention A and so intervention A should be selected. If intervention B falls either in the north-west or south-east quadrants of the health equity impact plane there is no obvious answer as to which intervention should be preferred as there is a trade-off to be made between health equity and total health.

5. Evaluate trade-offs between inequality and efficiency using social welfare functions

We use social welfare functions to trade-off between inequality reduction and average health improvement. These social welfare functions are constructed by combining our chosen measure of inequality with the average health in the distribution. This combination of inequality and average health is used to calculate what is known as an equally distributed equivalent (EDE) level of health. The EDE summarises the health distribution being analysed as one number representing the amount of health that each person in a hypothetically perfectly equal health distribution would need to have for us to be indifferent between the actual health distribution analysed and this perfectly equal health distribution. Where our social welfare function is built around an inequality measure with an inequality aversion parameter this EDE level of health will also be a function of the inequality aversion parameter. Where inequality aversion is set to zero there is no concern for inequality and the EDE simply reflects the average health in the distribution replicating results we would see under standard utilitarian CEA. As the inequality aversion level approaches infinity, our focus becomes increasingly on those worse off in the health distribution until at the limit we reflect the Rawlsian idea of focusing entirely on improving the lot of the worst-off in society.

 

Social welfare functions derived from the Atkinson relative inequality measure and the Kolm absolute inequality measure are given below, with the inequality aversion parameters circled. Research carried out with members of the public in England suggests that suitable values for the Atkinson and Kolm inequality aversion parameters are 10.95 and 0.15 respectively.

Atkinson Relative Social Welfare Function Kolm Absolute Social Welfare Function

When comparing interventions where one intervention does not simply dominate the others on the health equity impact plane we need to use our social welfare functions to calculate EDE levels of health associated with each of the interventions and then select the intervention that produces the highest EDE level of health.

In the example depicted in the figure above we can see that pursuing intervention A results in a health distribution which appears less unequal but has a lower average level of health than the health distribution resulting from intervention B. The choice of intervention, in this case, will be determined by the form of social welfare function selected and the level of inequality this social welfare function is parameterised to embody.

6. Conduct sensitivity analysis on forms of social welfare function and extent of inequality aversion

Given that the conclusions drawn from DCEA may be dependent on the social value judgments made around the inequality measure used and the level of inequality aversion embodied in it, we should present results for a range of alternative social welfare functions parameterised at a range of inequality aversion levels. This will allow decision makers to clearly understand how robust conclusions are to alternative social value judgements.

Applications

DCEA is of particular use when evaluating large-scale public health programmes that have an explicit goal of tackling health inequality. It has been applied to the NHS bowel cancer screening programme in England and to the rotavirus vaccination programme in Ethiopia.

Some key limitations of DCEA are that: (1) it currently only analyses programmes in terms of their health impacts whilst large public health programmes often have important impacts across a range of sectors beyond health; and (2) it requires a range of data beyond that required by standard CEA which may not be readily available in all contexts.

For low and middle-income settings an alternative augmented CEA methodology called extended cost effectiveness analysis (ECEA) has been developed to combine estimates of health impacts with estimates of impacts on financial risk protection. More information on ECEA can be found here.

There are ongoing efforts to generalise the DCEA methods to be applied to interventions having impacts across multiple sectors. Follow the latest developments on DCEA at the dedicated website based at the Centre for Health Economics, University of York.

Credit

Chris Sampson’s journal round-up for 27th August 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Ethically acceptable compensation for living donations of organs, tissues, and cells: an unexploited potential? Applied Health Economics and Health Policy [PubMed] Published 25th August 2018

Around the world, there are shortages of organs for transplantation. In economics, the debate around the need to increase organ donation can be frustratingly ignorant of ethical and distributional concerns. So it’s refreshing to see this article attempting to square concerns about efficiency and equity. The authors do so by using a ‘spheres of justice’ framework. This is the idea that different social goods should be distributed according to different principles. So, while we might be happy for brocolli and iPhones to be distributed on the basis of free exchange, we might want health to be distributed on the basis of need. The argument can be extended to state that – for a just situation to prevail – certain exchanges between these spheres of justice (e.g. health for iPhones) should never take place. This idea might explain why – as the authors demonstrate with a review of European countries – policy tends not to allow monetary compensation for organ donation.

The paper cleverly sets out to taxonomise monetary and non-monetary reimbursement and compensation with reference to individuals’ incentives and the spheres of justice principles. From this, the authors reach two key conclusions. Firstly, that (monetary) reimbursement of donors’ expenses (e.g. travel costs or lost earnings) is ethically sound as this does not constitute an incentive to donate but rather removes existing disincentives. Secondly, that non-monetary compensation could be deemed ethical.

Three possible forms of non-monetary compensation are discussed: i) prioritisation, ii) free access, and iii) non-health care-related benefits. The first could involve being given priority for receiving organs, or it could extend to the jumping of other health care waiting lists. I think this is more problematic than the authors let on because it asserts that health care should – at least in part – be distributed according to desert rather than need. The second option – free access – could mean access to health care that people would otherwise have to pay for. The third option could involve access to other social goods such as education or housing.

This is an interesting article and an enjoyable read, but I don’t think it provides a complete solution. Maybe I’m just too much of a Marxist, but I think that this – as all other proposals – fails to distribute from each according to ability. That is, we’d still expect non-monetary compensation to incentivise poorer (and on average less healthy) people to donate organs, thus exacerbating health inequality. This is because i) poorer people are more likely to need the non-monetary benefits and ii) we live in a capitalist society in which there is almost nothing that money can’t by and which is strictly non-monetary. Show me a proposal that increases donation rates from those who can most afford to donate them (i.e. the rich and healthy).

Selecting bolt-on dimensions for the EQ-5D: examining their contribution to health-related quality of life. Value in Health Published 18th August 2018

Measures such as the EQ-5D are used to describe health-related quality of life as completely and generically as possible. But there is a trade-off between completeness and the length of the questionnaire. Necessarily, there are parts of the evaluative space that measures will not capture because they are a simplification. If the bit they’re missing is important to your patient group, that’s a problem. You might fancy a bolt-on. But how do we decide which areas of the evaluative space should be more completely included in the measure? Which bolt-ons should be used? This paper seeks to provide means of answering these questions.

The article builds on an earlier piece of work that was included in an earlier journal round-up. In the previous paper, the authors used factor analysis to identify candidate bolt-ons. The goal of this paper is to outline an approach for specifying which of these candidates ought to be used. Using data from the Multi-Instrument Comparison study, the authors fit linear regressions to see how well 37 candidate bolt-on items explain differences in health-related quality of life. The 37 items correspond to six different domains: energy/vitality, satisfaction, relationships, hearing, vision, and speech. In a second test, the authors explored whether the bolt-on candidates could explain differences in health-related quality of life associated with six chronic conditions. Health-related quality of life is defined according to a visual analogue scale, which notably does not correspond to that used in the EQ-5D but rather uses a broader measure of physical, mental, and social health.

The results suggest that items related to energy/vitality, relationships, and satisfaction explained a significant part of health-related quality of life on top of the existing EQ-5D dimensions. The implication is that these could be good candidates for bolt-ons. The analysis of the different conditions was less clear.

For me, there’s a fundamental problem with this study. It moves the goals posts. Bolt-ons are about improving the extent to which a measure can more accurately represent the evaluative space that it is designed to characterise. In this study, the authors use a broader definition of health-related quality of life that – as far as I can tell – the EQ-5D is not designed to capture. We’re not dealing with bolt-ons, we’re dealing with extensions to facilitate expansions to the evaluative space. Nevertheless, the method could prove useful if combined with a more thorough consideration of the evaluative space.

Sources of health financing and health outcomes: a panel data analysis. Health Economics [PubMed] [RePEc] Published 15th August 2018

There is a growing body of research looking at the impact that health (care) spending has on health outcomes. Usually, these studies don’t explicitly look at who is doing the spending. In this study, the author distinguishes between public and private spending and attempts to identify which type of spending (if either) results in greater health improvements.

The author uses data from the World Bank’s World Development Indicators for 1995-2014. Life expectancy at birth is adopted as the primary health outcome and the key expenditure variables are health expenditure as a share of GDP and private health expenditure as a share of total health expenditure. Controlling for a variety of other variables, including some determinants of health such as income and access to an improved water source, a triple difference analysis is described. The triple difference estimator corresponds to the difference in health outcomes arising from i) differences in the private expenditure level, given ii) differences in total expenditure, over iii) time.

The key finding from the study is that, on average, private expenditure is more effective in increasing life expectancy at birth than public expenditure. The author also looks at government effectiveness, which proves crucial. The finding in favour of private expenditure entirely disappears when only countries with effective government are considered. There is some evidence that public expenditure is more effective in these countries, and this is something that future research should investigate further. For countries with ineffective governments, the implication is that policy should be directed towards increasing overall health care expenditure by increasing private expenditure.

Credits

Simon McNamara’s journal round-up for 6th August 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Euthanasia, religiosity and the valuation of health states: results from an Irish EQ5D5L valuation study and their implications for anchor values. Health and Quality of Life Outcomes [PubMed] Published 31st July 2018

Do you support euthanasia? Do you think there are health states worse than death? Are you religious? Don’t worry – I am not commandeering this week’s AHE journal round-up just to bombard you with a series of difficult questions. These three questions form the foundation of the first article selected for this week’s round-up.

The paper is based upon the hypothesis that your religiosity (“adherence to religious beliefs”) is likely to impact your support for euthanasia and, subsequently, the likelihood of you valuing severe health states as worse than death. This seems like a logical hypothesis. Religions tend to be anti-euthanasia, and so it appears likely that religious people will have lower levels of support for euthanasia than non-religious people. Equally, if you don’t support the principle of euthanasia, it stands to reason that you are likely to be less willing to choose immediate death over living in a severe health state – something you would need to do for a health state to be considered as being worse than death in a time trade-off (TTO) study.

The authors test this hypothesis using a sub-sample of data (n=160) collected as part of the Irish EQ-5D-5L TTO valuation study. Perhaps unsurprisingly, the authors find evidence in support of the above hypotheses. Those that attend a religious service weekly were more likely to oppose euthanasia than those who attend a few times a year or less, and those who oppose euthanasia were less likely to give “worse than death” responses in the TTO than those that support it.

I found this paper really interesting, as it raises a number of challenging questions. If a society is made up of people with heterogeneous beliefs regarding religion, how should we balance these in the valuation of health? If a society is primarily non-religious is it fair to apply this valuation tariff to the lives of the religious, and vice versa? These certainly aren’t easy questions to answer, but may be worth reflecting on.

E-learning and health inequality aversion: A questionnaire experiment. Health Economics [PubMed] [RePEc] Published 22nd July 2018

Moving on from the cheery topic of euthanasia, what do you think about socioeconomic inequalities in health? In my home country, England, if you are from the poorest quintile of society, you can expect to experience 62 years in full health in your lifetime, whilst if you are from the richest quintile, you can expect to experience 74 years – a gap of 12 years.

In the second paper to be featured in this round-up, Cookson et al. explore the public’s willingness to sacrifice incremental population health gains in order to reduce these inequalities in health – their level of “health inequality aversion”. This is a potentially important area of research, as the vast majority of economic evaluation in health is distributionally-naïve and effectively assumes that members of the public aren’t at all concerned with inequalities in health.

The paper builds on prior work conducted by the authors in this area, in which they noted a high proportion of respondents in health inequality aversion elicitation studies appear to be so averse to inequalities that they violate monotonicity – they choose scenarios that reduce inequalities in health even if these scenarios reduce the health of the rich at no gain to the poor, or they reduce the health of the poor, or they may reduce the health of both groups. The authors hypothesise that these monotonicity violations may be due to incomplete thinking from participants, and suggest that the quality of their thinking could be improved by two e-learning educational interventions. The primary aim of the paper is to test the impact of these interventions in a sample of the UK public (n=60).

The first e-learning intervention was an animated video that described a range of potential positions that a respondent could take (e.g. health maximisation, or maximising the health of the worst off). The second was an interactive spreadsheet-based questionnaire that presented the consequences of the participant’s choices, prior to them confirming their selection. Both interventions are available online.

The authors found that the interactive tool significantly reduced the amount of extreme egalitarian (monotonicity-violating) responses, compared to a non-interactive, paper-based version of the study. Similarly, when the video was watched before completing the paper-based exercise, the number of extreme egalitarian responses reduced. However, when the video was watched before the interactive tool there was no further decrease in extreme egalitarianism. Despite this reduction in extreme egalitarianism, the median levels of inequality aversion remained high, with implied weights of 2.6 and 7.0 for QALY gains granted to someone from the poorest fifth of society, compared to the richest fifth of society for the interactive questionnaire and video groups respectively.

This is an interesting study that provides further evidence of inequality aversion, and raises further concern about the practical dominance of distributionally-naïve approaches to economic evaluation. The public does seem to care about distribution. Furthermore, the paper demonstrates that participant responses to inequality aversion exercises are shaped by the information given to them, and the way that information is presented. I look forward to seeing more studies like this in the future.

A new method for valuing health: directly eliciting personal utility functions. The European Journal of Health Economics [PubMed] [RePEc] Published 20th July 2018

Last, but not least, for this round-up, is a paper by Devlin et al. on a new method for valuing health.

The relative valuation of health states is a pretty important topic for health economists. If we are to quantify the effectiveness, and subsequently cost-effectiveness, of an intervention, we need to understand which health states are better than others, and how much better they are. Traditionally, this is done by asking members of the public to choose between different health profiles featuring differing levels of fulfilment of a range of domains of health, in order to ‘uncover’ the relative importance the respondent places on these domains, and levels. These can then be used in order to generate social tariffs that assign a utility value to a given health state for use in economic evaluation.

The authors point out that, in the modern day, valuation studies can be conducted rapidly, and at scale, online, but at the potential cost of deliberation from participants, and the resultant risk of heuristic dominated decision making. In response to this, the authors propose a new method – direct elicitation of personal utility functions, and pilot its use for the valuation of EQ-5D in a sample of the English public (n=76).

The proposed approach differs from traditional approaches in three key ways. Firstly, instead of simply attempting to infer the relative importance that participants place on differing domains based upon choices between health profiles, the respondents are asked directly about the relative importance they place on differing domains of health, prior to validating these with profile choices. Secondly, the authors place a heavy emphasis on deliberation, and the construction, rather than uncovering, of preferences during the elicitation exercises. Thirdly, a “personal utility function” for each individual is constructed (in effect a personal EQ-5D tariff), and these individual utility functions are subsequently aggregated into a social utility function.

In the pilot, the authors find that the method appears feasible for wider use, albeit with some teething troubles associated with the computer-based tool developed to implement it, and the skills of the interviewers.

This direct method raises an interesting question for health economics – should we be inferring preferences based upon choices that differ in terms of certain attributes, or should we just ask directly about the attributes? This is a tricky question. It is possible that the preferences elicited via these different approaches could result in different preferences – if they do, on what grounds should we choose one or other? This requires a normative judgment, and at present, it appears both are (potentially) as legitimate as each other.

Whilst the authors apply this direct method to the valuation of health, I don’t see why similar approaches couldn’t be applied to any multi-attribute choice experiment. Keep your eyes out for future uses of it in valuation, and perhaps beyond? It will be interesting to see how it develops.

Credits