Journal Club Briefing: Dolan and Kahneman (2008)

Today’s Journal Club Briefing comes from the Academic Unit of Health Economics at the University of Leeds. At their journal club on 2nd August 2017, they discussed Dolan and Kahneman’s 2008 article from The Economic Journal: ‘Interpretations of utility and their implications for the valuation of health‘. If you’ve discussed an article at a recent journal club meeting at your own institution and would like to write a briefing for the blog, get in touch.

Why this paper?

Dolan and Kahneman (2008) is a paper which was published nearly ten years ago, was written several years before that, and was not published in a health-related journal. It’s hence, at first sight, a slightly curious choice for a health economics journal club. However, it raises issues which are at the heart of health economics practice. The questions raised by this article have not as yet been answered, and don’t look likely to be answered anytime soon.

Summary

Experienced vs. decision utility

The article’s point of departure is the distinction between experienced utility and decision utility, often a source of fruitful research in behavioural economics. Experienced utility is utility in the Benthamite sense, meaning the hedonic experience in the current moment: the pleasure and/or pain felt by a person at any given point in time. Decision utility is utility as taught in undergraduate economics textbooks: an objective function which the individual dispassionately acts to maximise. In the neoclassical framework of said undergraduate textbooks, this is a distinction without a difference. The individual correctly forecasts the expected flow of experienced utility given the available information and her actions, forms a decision utility function from it and acts to maximise it.

However, Thaler and Sunstein wouldn’t have sold as many books if things were so simple. Many systematic and significant instances of divergences between experienced and decision utility have been well documented, and several people (including one of the authors of this paper) have won Nobel prizes for it. The one which this article focuses on is adaptation.

Adaptation

The authors summarise a large body of evidence that shows that individuals suffer a large loss of utility after a traumatic event (e.g. the loss of a limb or loss of function), but that for many conditions they will adapt to their new situation and recover much of their utility loss. After as little as a year, their valuation of their health is very similar to that of the general population. Furthermore, the authors precis various studies which show that individuals routinely underestimate drastically the amount of adaptation that would occur should such a traumatic event befall them.

This improvement over time in the health-related utility experienced by people with many conditions is partly due to hedonic adaptation – the internal scale of pleasure/pain re-calibrates to their new situation – and partly due to behavioural change, such as finding new pastimes to replace those ruled out by their condition. While the causes of adaptation are fascinating, the focus here is not on the mechanisms behind it, but rather on the consequences for measuring utility and the implications for resource allocation.

Health valuation and adaptation

The methods health economists use to evaluate the utility of being in a given health state, such as time trade-off, standard gamble or discrete choice experiments, will tend to elicit decision utility. They are based on choices between hypothetical states and so will not capture the changes in experienced utility due to adaptation. Thus valuations of health states from the general public will tend to be lower than the valuations from people actually living in the health state.

At first glance, the consequences for resource allocation may not appear to be particularly severe. It may lead to more resources being devoted to healthcare as a whole (at least for life-improving treatments – life-extending treatments are a different case), but the overall healthcare budget is in practice largely a political decision. However, it will not lead to distortions between treatments for alternative conditions.

Yet adaptation is not a universal phenomenon. There are conditions for which little or no adaptation is seen (for example unexplained pain), and when it occurs, it occurs at different speeds and to differing extents for different conditions. The authors show that valuations of conditions with a greater initial utility loss are lower than conditions with a lesser initial loss but a lower degree of adaptation, and thus will receive a greater level of resources, despite the sum of experienced utility being the same for both. The authors argue that this is unfair, and that health economists should update their practices to better capture experienced utility.

Public vs. patient preference

A common argument in favour of the status quo is that (in many countries at least) it is public resources which are being allocated, and thus it is public preferences which should be respected. It appears legitimate to allocate resources to assuage public fears of health states, even if those health states are worse in their imagination than in reality. The authors consider this argument and reply that, in this case, the instruments of health economists are still not fit for purpose. General measures of health states, such as EQ-5D, go out of their way to describe states in abstract terms and to separate them from causes, such as cancer, which may carry an emotional affect. It cannot be argued that public valuations are justified because resources should be allocated according to public fears if the measurement of valuation deliberately tries not to elicit those fears.

The argument that adaptation causes serious problems for valuing health and for allocation of health resources is a persuasive one. It is undoubtedly true that changes in utility over time, and other violations of the neoclassical economic paradigm such as reference dependence, do not presently receive sufficient attention in health economics and policy decisions in general.

Discussion

Which yardstick?

Despite the stimulating discussion and the overall brilliance of the paper, there are some elements which can be challenged. One of them is that throughout, the authors’ arguments and recommendations are made from the standpoint that the sum over time of the flow of experienced utility from a health state is to be used as the sole measure of value. This would consist in what one of the authors calls the day reconstruction method (DRM) which consists in rating a range of feelings including happiness, worry, and frustration.

Despite the acknowledgement of some philosophical difficulties, the sum of the flow of experienced utility is treated as if it is the only true yardstick with which to measure health, without a convincing justification and no discussion on the qualitative aspect of the measurement as opposed to a truly cardinal measure of health allowing ranking of individuals’ health states.

Public vs. private preferences revisited

The authors raise the question of whether current practice can be justified by a desire to soothe public fears, and dismiss it since the elicitation tools are not suitable. However, they do not address the question of whether allocating public resources according to the public’s (incorrect) fears of given diseases or health states could be a legitimate health policy aim. One could imagine, for example, a discrete choice experiment eliciting how much the general public dreads cancer over other diseases, and make an argument that the welfare of the public is improved by allocating resources based on these results. There are myriad problems with such an approach, of course, but there seem to be no fewer problems with alternative approaches.

Intertemporal welfare

Intertemporal welfare judgements are notoriously difficult once the exponential discounting framework is left. It seems just as legitimate to base valuations on the ex post judgement of individuals who have fully adjusted to a health state as on an integration of past feelings, most of which are now distant memories. Most people would agree that the time to value their experience of a marathon is after completing it, not during the twenty-fifth mile or at the start line.

Indeed, this appears to be the position tacitly taken elsewhere by Kahneman in his work on the peak-end rule. In Redelmeier et al. (2003), it was found that the retrospective rating of the pain of a colonoscopy was based almost exclusively on the peak intensity of pain and on the pain felt at the end. Thus procedures which were extended by an extra three minutes were remembered as less painful than standard procedures, even though the total pain experienced was greater. Furthermore, those who underwent the extended procedure were more likely to state they would undergo it again. It would seem strange, in this case, to judge them as worse off.

Schelling (1984) ends his superlative discussion of the problems of intertemporal decision making with the following thought experiment. Just as with valuing health, there are no easy answers.

[S]ome anesthetics block transmission of the nervous impulses that constitute pain; others have the characteristic that the patient responds to the pain as if feeling it fully but has utterly no recollection afterwards. One of these is sodium pentothal. In my imaginary experiment we wish to distinguish the effects of the drug from the effects of the unremembered pain, and we want a healthy control subject in parallel with some painful operations that will be performed with the help of this drug. For a handsome fee you will be knocked out for an hour or two, allowed to sleep it off, then tested before you go home. You do this regularly, and one afternoon you walk into the lab a little early and find the experimenters viewing some videotape. On the screen is an experimental subject writhing, and though the audio is turned down the shrieks are unmistakably those of a person in pain. When the pain stops the victim pleads, “Don’t ever do that again. Please.”

The person is you.

Do you care?

Do you walk into your booth, lie on the couch, and hold out your arm for today’s injection?

Should I let you?

Credits

Meeting round-up: EuroQol Plenary Meeting 2017

The 34th Plenary Meeting of the EuroQol Group took place in Barcelona on 21st and 22nd September 2017. The local hosts of the meeting were Mike Herdman (UK-born but a Barcelona resident for many years), Juan Manuel Ramos-Goñi and Oliver Rivero-Arias. For the second year running, I chaired the Scientific Programme together with Anna Lugnér.

At its inception, the EuroQol Group was very much a northern European collaboration – the early versions of the EuroQol instrument (now known as the EQ-5D) were developed by researchers in the Netherlands, UK, Sweden, Finland and Norway – see here for an overview of the Group and its history. This year’s Plenary Meeting was attended by 111 participants (primarily academic researchers) representing 23 different countries spanning six continents.

As with previous Plenary Meetings, an HESG-style discussant format was followed – papers were pre-circulated to participants and presented by discussants rather than by authors. The parallel poster sessions also followed a discussant format, with approximately 10 minutes dedicated to the discussion of each poster. In total, 19 papers and 20 posters were presented. For the first time, the majority of the papers were lead-authored by women.

One of the themes of the meeting was a focus on the relationships and interactions between EQ-5D dimensions. A paper by Anna Selivanova compared health state values derived from discrete choice data both with and without interactions. Anna reported results demonstrating that interactions are important and that the interaction between mobility and self-care was the most salient. Another paper by Thor Gamst-Klaussen (represented at the meeting by co-author Jan Abel Olsen) explored the causal and effect nature of EQ-5D dimensions. The authors applied confirmatory tetrad analysis and confirmatory factor analysis to multi-country, cross-sectional data in order to test a conceptual framework depicting relationships among the five dimensions. The results suggest that the EQ-5D comprises both causal variables – mobility, pain/discomfort and anxiety/depression – and effect variables – self-care and usual activities.

An intriguing paper by John Hartman tested for differences in respondent characteristics, participation, response quality and EQ-5D-5L values depending on the device and connection used to access an online survey. The results showed systematic variability in participation and response quality, but the variability did not affect the resulting health state values. The findings could support extending the administration of valuation surveys to smaller devices (e.g. mobile phones) to obtain responses from younger, more ethnically diverse populations who have traditionally been found to be difficult to recruit.

Other topics covered in the programme included the views of UK decision makers on the role of well-being in resource allocation decisions, the development of a value set for the EQ-5D-Y (a version of the EQ-5D designed for use in children and adolescents), and the prevalence and impact of so-called ‘implausible’ health states.

The Plenary Meeting concluded with a guest presentation by Janel Hanmer of the University of Pittsburgh, followed by a reception at a restaurant on the Montjuïc hill overlooking the Barcelona harbour. The next EuroQol conference will be the Academy Meeting, which takes place in Budapest on 6-8 March 2018.

Credits

Thesis Thursday: Lidia Engel

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Lidia Engel who graduated with a PhD from Simon Fraser University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Going beyond health-related quality of life for outcome measurement in economic evaluation
Supervisors
David Whitehurst, Scott Lear, Stirling Bryan
Repository link
https://theses.lib.sfu.ca/thesis/etd10264

Your thesis explores the potential for expanding the ‘evaluative space’ in economic evaluation. Why is this important?

I think there are two answers to this question. Firstly, methods for economic evaluation of health care interventions have existed for a number of years but these evaluations have mainly been applied to more narrowly defined ‘clinical’ interventions, such as drugs. Interventions nowadays are more complex, where benefits cannot be simply measured in terms of health. You can think of areas such as public health, mental health, social care, and end-of-life care, where interventions may result in broader benefits, such as increased control over daily life, independence, or aspects related to the process of health care delivery. Therefore, I believe there is a need to re-think the way we measure and value outcomes when we conduct an economic evaluation. Secondly, ignoring broader outcomes of health care interventions that go beyond the narrow focus of health-related quality of life can potentially lead to misallocation of scarce health care resources. Evidence has shown that the choice of outcome measure (such as a health outcome or a broader measure of wellbeing) can have a significant influence on the conclusions drawn from an economic evaluation.

You use both qualitative and quantitative approaches. Was this key to answering your research questions?

I mainly applied quantitative methods in my thesis research. However, Chapter 3 draws upon some qualitative methodology. To gain a better understanding of ‘benefits beyond health’, I came across a novel approach, called Critical Interpretive Synthesis. It is similar to meta-ethnography (i.e. a synthesis of qualitative research), with the difference that the synthesis is not of qualitative literature but of methodologically diverse literature. It involves an iterative approach, where searching, sampling, and synthesis go hand in hand. It doesn’t only produce a summary of existing literature but enables the development of new interpretations that go beyond those originally offered in the literature. I really liked this approach because it enabled me to synthesise the evidence in a more effective way compared with a conventional systematic review. Defining and applying codes and themes, as it is traditionally done in qualitative research, allowed me to organize the general idea of non-health benefits into a coherent thematic framework, which in the end provided me with a better understanding of the topic overall.

What data did you analyse and what quantitative methods did you use?

I conducted three empirical analyses in my thesis research, which all made use of data from the ICECAP measures (ICECAP-O and ICECAP-A). In my first paper, I used data from the ‘Walk the Talk (WTT)‘ project to investigate the complementarity of the ICECAP-O and the EQ-5D-5L in a public health context using regression analyses. My second paper used exploratory factor analysis to investigate the extent of overlap between the ICECAP-A and five preference-based health-related quality of life measures, using data from the Multi Instrument Comparison (MIC) project. I am currently finalizing submission of my third empirical analysis, which reports findings from a path analysis using cross-sectional data from a web-based survey. The path analysis explores three outcome measurement approaches (health-related quality of life, subjective wellbeing, and capability wellbeing) through direct and mediated pathways in individuals living with spinal cord injury. Each of the three studies addressed different components of the overall research question, which, collectively, demonstrated the added value of broader outcome measures in economic evaluation when compared with existing preference-based health-related quality of life measures.

Thinking about the different measures that you considered in your analyses, were any of your findings surprising or unexpected?

In my first paper, I found that the ICECAP-O is more sensitive to environmental features (i.e. social cohesion and street connectivity) when compared with the EQ-5D-5L. As my second paper has shown, this was not surprising, as the ICECAP-A (a measure for adults rather than older adults) and the EQ-5D-5L measure different constructs and had only limited overlap in their descriptive classification systems. While a similar observation was made when comparing the ICECAP-A with three other preference-based health-related quality of life measures (15D, HUI-3, and SF-6D), a substantial overlap was observed between the ICECAP-A and the AQoL-8D, which suggests that it is possible for broader benefits to be captured by preference-based health-related measures (although some may not consider the AQoL-8D to be exclusively ‘health-related’, despite the label). The findings from the path analysis confirmed the similarities between the ICECAP-A and the AQoL-8D. However, the findings do not imply that the AQoL-8D and ICECAP-A are interchangeable instruments, as a mediation effect was found that requires further research.

How would you like to see your research inform current practice in economic evaluation? Is the QALY still in good health?

I am aware of the limitations of the QALY and although there are increasing concerns that the QALY framework does not capture all benefits of health care interventions, it is important to understand that the evaluative space of the QALY is determined by the dimensions included in preference-based measures. From a theoretical point of view, the QALY can embrace any characteristics that are important for the allocation of health care resources. However, in practice, it seems that QALYs are currently defined by what is measured (e.g. the dimensions and response options of EQ-5D instruments) rather than the conceptual origin. Therefore, although non-health benefits have been largely ignored when estimating QALYs, one should not dismiss the QALY framework but rather develop appropriate instruments that capture such broader benefits. I believe the findings of my thesis have particular relevance for national HTA bodies that set guidelines for the conduct of economic evaluation. While the need to maintain methodological consistency is important, the assessment of the real benefits of some health care interventions would be more accurate if we were less prescriptive in terms of which outcome measure to use when conducting an economic evaluation. As my thesis has shown, some preference-based measures already adopt a broad evaluative space but are less frequently used.