Journal Club Briefing: Dolan and Kahneman (2008)

Today’s Journal Club Briefing comes from the Academic Unit of Health Economics at the University of Leeds. At their journal club on 2nd August 2017, they discussed Dolan and Kahneman’s 2008 article from The Economic Journal: ‘Interpretations of utility and their implications for the valuation of health‘. If you’ve discussed an article at a recent journal club meeting at your own institution and would like to write a briefing for the blog, get in touch.

Why this paper?

Dolan and Kahneman (2008) is a paper which was published nearly ten years ago, was written several years before that, and was not published in a health-related journal. It’s hence, at first sight, a slightly curious choice for a health economics journal club. However, it raises issues which are at the heart of health economics practice. The questions raised by this article have not as yet been answered, and don’t look likely to be answered anytime soon.

Summary

Experienced vs. decision utility

The article’s point of departure is the distinction between experienced utility and decision utility, often a source of fruitful research in behavioural economics. Experienced utility is utility in the Benthamite sense, meaning the hedonic experience in the current moment: the pleasure and/or pain felt by a person at any given point in time. Decision utility is utility as taught in undergraduate economics textbooks: an objective function which the individual dispassionately acts to maximise. In the neoclassical framework of said undergraduate textbooks, this is a distinction without a difference. The individual correctly forecasts the expected flow of experienced utility given the available information and her actions, forms a decision utility function from it and acts to maximise it.

However, Thaler and Sunstein wouldn’t have sold as many books if things were so simple. Many systematic and significant instances of divergences between experienced and decision utility have been well documented, and several people (including one of the authors of this paper) have won Nobel prizes for it. The one which this article focuses on is adaptation.

Adaptation

The authors summarise a large body of evidence that shows that individuals suffer a large loss of utility after a traumatic event (e.g. the loss of a limb or loss of function), but that for many conditions they will adapt to their new situation and recover much of their utility loss. After as little as a year, their valuation of their health is very similar to that of the general population. Furthermore, the authors precis various studies which show that individuals routinely underestimate drastically the amount of adaptation that would occur should such a traumatic event befall them.

This improvement over time in the health-related utility experienced by people with many conditions is partly due to hedonic adaptation – the internal scale of pleasure/pain re-calibrates to their new situation – and partly due to behavioural change, such as finding new pastimes to replace those ruled out by their condition. While the causes of adaptation are fascinating, the focus here is not on the mechanisms behind it, but rather on the consequences for measuring utility and the implications for resource allocation.

Health valuation and adaptation

The methods health economists use to evaluate the utility of being in a given health state, such as time trade-off, standard gamble or discrete choice experiments, will tend to elicit decision utility. They are based on choices between hypothetical states and so will not capture the changes in experienced utility due to adaptation. Thus valuations of health states from the general public will tend to be lower than the valuations from people actually living in the health state.

At first glance, the consequences for resource allocation may not appear to be particularly severe. It may lead to more resources being devoted to healthcare as a whole (at least for life-improving treatments – life-extending treatments are a different case), but the overall healthcare budget is in practice largely a political decision. However, it will not lead to distortions between treatments for alternative conditions.

Yet adaptation is not a universal phenomenon. There are conditions for which little or no adaptation is seen (for example unexplained pain), and when it occurs, it occurs at different speeds and to differing extents for different conditions. The authors show that valuations of conditions with a greater initial utility loss are lower than conditions with a lesser initial loss but a lower degree of adaptation, and thus will receive a greater level of resources, despite the sum of experienced utility being the same for both. The authors argue that this is unfair, and that health economists should update their practices to better capture experienced utility.

Public vs. patient preference

A common argument in favour of the status quo is that (in many countries at least) it is public resources which are being allocated, and thus it is public preferences which should be respected. It appears legitimate to allocate resources to assuage public fears of health states, even if those health states are worse in their imagination than in reality. The authors consider this argument and reply that, in this case, the instruments of health economists are still not fit for purpose. General measures of health states, such as EQ-5D, go out of their way to describe states in abstract terms and to separate them from causes, such as cancer, which may carry an emotional affect. It cannot be argued that public valuations are justified because resources should be allocated according to public fears if the measurement of valuation deliberately tries not to elicit those fears.

The argument that adaptation causes serious problems for valuing health and for allocation of health resources is a persuasive one. It is undoubtedly true that changes in utility over time, and other violations of the neoclassical economic paradigm such as reference dependence, do not presently receive sufficient attention in health economics and policy decisions in general.

Discussion

Which yardstick?

Despite the stimulating discussion and the overall brilliance of the paper, there are some elements which can be challenged. One of them is that throughout, the authors’ arguments and recommendations are made from the standpoint that the sum over time of the flow of experienced utility from a health state is to be used as the sole measure of value. This would consist in what one of the authors calls the day reconstruction method (DRM) which consists in rating a range of feelings including happiness, worry, and frustration.

Despite the acknowledgement of some philosophical difficulties, the sum of the flow of experienced utility is treated as if it is the only true yardstick with which to measure health, without a convincing justification and no discussion on the qualitative aspect of the measurement as opposed to a truly cardinal measure of health allowing ranking of individuals’ health states.

Public vs. private preferences revisited

The authors raise the question of whether current practice can be justified by a desire to soothe public fears, and dismiss it since the elicitation tools are not suitable. However, they do not address the question of whether allocating public resources according to the public’s (incorrect) fears of given diseases or health states could be a legitimate health policy aim. One could imagine, for example, a discrete choice experiment eliciting how much the general public dreads cancer over other diseases, and make an argument that the welfare of the public is improved by allocating resources based on these results. There are myriad problems with such an approach, of course, but there seem to be no fewer problems with alternative approaches.

Intertemporal welfare

Intertemporal welfare judgements are notoriously difficult once the exponential discounting framework is left. It seems just as legitimate to base valuations on the ex post judgement of individuals who have fully adjusted to a health state as on an integration of past feelings, most of which are now distant memories. Most people would agree that the time to value their experience of a marathon is after completing it, not during the twenty-fifth mile or at the start line.

Indeed, this appears to be the position tacitly taken elsewhere by Kahneman in his work on the peak-end rule. In Redelmeier et al. (2003), it was found that the retrospective rating of the pain of a colonoscopy was based almost exclusively on the peak intensity of pain and on the pain felt at the end. Thus procedures which were extended by an extra three minutes were remembered as less painful than standard procedures, even though the total pain experienced was greater. Furthermore, those who underwent the extended procedure were more likely to state they would undergo it again. It would seem strange, in this case, to judge them as worse off.

Schelling (1984) ends his superlative discussion of the problems of intertemporal decision making with the following thought experiment. Just as with valuing health, there are no easy answers.

[S]ome anesthetics block transmission of the nervous impulses that constitute pain; others have the characteristic that the patient responds to the pain as if feeling it fully but has utterly no recollection afterwards. One of these is sodium pentothal. In my imaginary experiment we wish to distinguish the effects of the drug from the effects of the unremembered pain, and we want a healthy control subject in parallel with some painful operations that will be performed with the help of this drug. For a handsome fee you will be knocked out for an hour or two, allowed to sleep it off, then tested before you go home. You do this regularly, and one afternoon you walk into the lab a little early and find the experimenters viewing some videotape. On the screen is an experimental subject writhing, and though the audio is turned down the shrieks are unmistakably those of a person in pain. When the pain stops the victim pleads, “Don’t ever do that again. Please.”

The person is you.

Do you care?

Do you walk into your booth, lie on the couch, and hold out your arm for today’s injection?

Should I let you?

Credits

Are we estimating the effects of health care expenditure correctly?

It is a contentious issue in philosophy whether an omission can be the cause of an event. At the very least it seems we should consider causation by omission differently from ‘ordinary’ causation. Consider Sarah McGrath’s example. Billy promised Alice to water the plant while she was away, but he did not water it. Billy not watering the plant caused its death. But there are good reasons to suppose that Billy did not cause its death. If Billy’s lack of watering caused the death of the plant, it may well be reasonable to assume that Vladimir Putin and indeed anyone else who did not water the plant were also a cause. McGrath argues that there is a normative consideration here: Billy ought to have watered the plant and that’s why we judge his omission as a cause and not anyone else’s. Similarly, the example from L.A. Paul and Ned Hall’s excellent book Causation: A User’s GuideBilly and Suzy are playing soccer on rival teams. One of Suzy’s teammates scores a goal. Both Billy and Suzy were nearby and could have easily prevented the goal. But our judgement is that the goal should only be credited to Billy’s failure to block the goal as Suzy had no responsibility to.

These arguments may appear far removed from the world of health economics. But, they have practical implications. Consider the estimation of the effect that increasing health care expenditure has on public health outcomes. The government, or relevant health authority, makes a decision about how the budget is allocated. It is often the case that there are allocative inefficiencies: greater gains could be had by reallocating the budget to more effective programs of care. In this case there would seem to be a relevant omission; the budget has not been spent where it could have provided benefits. These omissions are often seen as causes of a loss of health. Karl Claxton wrote of the Cancer Drugs Fund, a pool of money diverted from the National Health Service to provide cancer drugs otherwise considered cost-ineffective, that it was associated with

a net loss of at least 14,400 quality adjusted life years in 2013/14.

Similarly, an analysis of the lack of spending on effective HIV treatment and prevention by the Mbeki administration in South Africa wrote that

More than 330,000 lives or approximately 2.2 million person-years were lost because a feasible and timely ARV treatment program was not implemented in South Africa.

But our analyses of the effects of health care expenditure typically do not take these omissions into account.

Causal inference methods are founded on a counterfactual theory of causation. The aim of a causal inference method is to estimate the potential outcomes that would have been observed under different treatment regimes. In our case this would be what would have happened under different levels of expenditure. This is typically estimated by examining the relationship between population health and levels of expenditure, perhaps using some exogenous determinant of expenditure to identify the causal effects of interest. But this only identifies those changes caused by expenditure and not those changes caused by not spending.

Consider the following toy example. There are two causes of death in the population a and b with associated programs of care and prevention A and B. The total health care expenditure is x of which a proportion p: p\in P \subseteq [0,1] is spent on A and 1-p on B. The deaths due to each cause are y_a and y_b and so the total deaths are y = y_a + y_b. Finally, the effect of a unit increase in expenditure in each program are \beta_a and \beta_b. The question is to determine what the causal effect of expenditure is. If Y_x is the potential outcome for level of expenditure x then the average treatment effect is given by E(\frac{\partial Y_x}{\partial x}).

The country has chosen an allocation between the programmes of care of p_0. If causation by omission is not a concern then, given linear, additive models (and that all the model assumptions are met), y_a = \alpha_a + \beta_a p x + f_a(t) + u_a and y_b = \alpha_b + \beta_b (1-p) x + f_b(t) + u_b, the causal effect is E(\frac{\partial Y_x}{\partial x}) = \beta = \beta_a p_0 + \beta_b (1-p_0). But if causation by omission is relevant, then the net effect of expenditure is the lives gained \beta_a p_0 + \beta_b (1-p_0) less the lives lost. The lives lost are those under all possible things we did not do, so the estimator of the causal effect is \beta' = \beta_a p_0 + \beta_b (1-p_0) -  \int_{P/p_0} [ \beta_ap + \beta_b(1-p) ] dG(p). Now, clearly \beta \neq \beta' unless P/p_0 is the empty set, i.e. there was no other option. Indeed, the choice of possible alternatives involves a normative judgement as we’ve suggested. For an omission to count as a cause, there needs to be a judgement about what ought to have been done. For health care expenditure this may mean that the only viable alternative is the allocatively efficient distribution, in which case all allocations will result in a net loss of life unless they are allocatively efficient, which some may argue is reasonable. An alternative view is simply that the government simply has to not do worse than in the past and perhaps it is also reasonable for the government not to make significant changes to the allocation, for whatever reason. In that case we might say that P \in [p_0,1] and g(p) might be a distribution truncated below p_0 with most mass around p_0 and small variance.

The problem is that we generally do not observe the effect of expenditure in each program of care nor do we know the distribution of possible budget allocations. The normative judgements are also a contentious issue. Claxton clearly believes the government ought not to have initiated the Cancer Drugs Fund, but he does not go so far as to say any allocative inefficiency results in a net loss of life. Some working out of the underlying normative principles is warranted. But if it’s not possible to estimate these net causal effects, why discuss it? Perhaps it’s due to the lack of consistency. We estimate the ‘ordinary’ causal effect in our empirical work, but we often discuss opportunity costs and losses due to inefficiencies as being due to or caused by the spending decisions that are made. As the examples at the beginning illustrate, the normative question of responsibility seeps into our judgments about whether an omission is the cause of an outcome. For health care expenditure the government or other health care body does have a relevant responsibility. I would argue then that causation by omission is important and perhaps we need to reconsider the inferences that we make.

Credits

Chris Sampson’s journal round-up for 14th March 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Discretely integrated condition event (DICE) simulation for pharmacoeconomics. PharmacoEconomics [PubMedPublished 9th March 2016

Markov state transition? Discrete event simulation? Put down that taxonomy of model structures – all you need is DICE! Oh, and sell your fancy simulation software – all you need is Excel or R. In this paper, Jaime Caro proposes discretely integrated condition event simulation as a unifying modelling approach. Two concepts form the basis of the approach: “conditions” and “events”. Events are self-explanatory; they represent those things that might occur at any given point in time. Conditions represent the characteristics that persist; for example, disease severity, clinical markers, the age of the individual, the sex of the cohort. And of course, these may change over time and in response to particular events. Both conditions and events can be associated with values; for example utilities or costs. So far, so familiar. Caro states that the essence of DICE is that conditions and events are integrated in the sense that changes in conditions are associated with events (and with time) in a simplified way. In health economics, we are never able to fully capture the continuous interactions between different conditions. So we can accept this simplification. The paper outlines how an analyst might go about building a DICE. DICE is made by health economist for health economists. It isn’t designed to cope well with queuing and capacity constraints. It isn’t ideal for modelling interactions between people. But most of the time we don’t do that anyway. The appeal of DICE is that it could support simplicity and comparability in modelling. It can be implemented in spreadsheet software or a general programming language (if it takes too long to run in Excel – for example – you can try R). The paper also comes with an online appendix containing an example implemented in Excel. I’m yet to dig into it, but I’m looking forward to doing so. I am no modelling expert, and I hope we’ll see some feedback from others on the validity of the DICE approach. But if it’s all it’s cracked up to be then there’s a simpler, more consistent, more transparent future out there for decision analytic modelling.

Heterogeneity in the effect of common shocks on healthcare expenditure growth. Health Economics [PubMedPublished 4th March 2016

Growth in healthcare expenditure (bearing in mind Baumol’s ‘cost disease’) is a major political and economic challenge. Understanding what either causes or slows down expenditure growth is therefore an important pursuit. Furthermore, it’s reasonable to expect that while particular shocks might affect all countries, the magnitude of these effects is likely to differ. This study uses a common factor model that accounts for the possibility that some of the heterogeneity in the effect of shocks on different countries could be explained by observable factors. The source of data is the OECD Health Statistics and the authors are able to test 43 determinants of healthcare expenditure growth in 34 countries between 1980 and 2012. Central to the study are the authors’ attempts to address the shortcomings of previous studies due to unobserved heterogeneity, model uncertainty and missing data, and the authors implement a number of novel techniques to attain better estimates. But after all that, the authors find little difference between their preferred model and a standard fixed effects model. Factors that are identified as growth-slowing include: less reliance on public financing; competitive pressures in insurance markets; substitution from inpatient to outpatient care; regulation of pharmaceutical costs and reduced administration.

Societal preferences for interventions with the same efficiency: assessment and application to decision making. Applied Health Economics and Health Policy [PubMedPublished 3rd March 2016

Most people who aren’t economists care about more than efficiency in the allocation of healthcare resources. One way to capture the extent to which people value other attributes could be to elicit preferences for technologies that have the same efficiency. In this study, the authors carry out both a budget allocation survey and a discrete choice experiment with more than 1000 Japanese respondents for each. The attributes considered were age, objective of care (treatment/prevention), disease severity, prior medical care, cause of disease and disease rarity. All scenarios had equivalent cost and QALY outcomes. Some familiar findings come out: preference for younger people and more severe disease and for treatment rather than prevention. In the budget allocation experiment, more than half of respondents supported the prioritisation of younger people. For most other attributes the largest proportion of respondents supported equal budget allocation. Both experiments provided similar results. The authors also introduce the concept of the preference-adjusted threshold (PAT), which reflects people’s preferences within a decision maker’s threshold range. The idea is that that this can then be used to adjust the cost-per-QALY threshold accordingly, which perhaps has more validity than more arbitrary weightings. The authors estimate marginal PATs for the attributes; for example, from these results the additional willingness to pay for a QALY for a younger cohort should be around $20,000.

An analysis of the complementarity of ICECAP-A and EQ-5D-3L in an adult population of patients with knee pain. Health and Quality of Life Outcomes [PubMedPublished 3rd March 2016

Some readers may know that I’ve been a little critical of the ICECAP measures in the past. One of my concerns is that they are not fundamentally any different to the likes of the EQ-5D. Previous work has considered the complementarity of the EQ-5D and the ICECAP-O (for older people). This study is designed to shed some light on how the ICECAP-A (for all adults) and EQ-5D-3L should be used together, and whether they measure different constructs. The authors used data from a randomised controlled trial in which both measures were completed by 442 people. Association was assessed using Spearman’s rank correlation, which showed a moderate correlation of 0.49. Exploratory factor analysis was used to assess whether or not both measures were describing the same underlying unobserved constructs. This analysis showed that a two factor model was optimal, and that the two measures largely described these two factors separately. If you’re familiar with the two questionnaires then this shouldn’t come as too much of a surprise. However, it takes a leap of faith (which the authors do take) to then conclude that each of these factors represent separate constructs in the sense that one represents “physical health” (mostly EQ-5D) and one represents “wellbeing” (mostly ICECAP). Nevertheless, the results do show that the ICECAP-A and EQ-5D-3L should not be used as substitutes as they are not measuring the same thing.

Economic evaluation of mental health interventions: a guide to costing approaches. PharmacoEconomics [PubMedPublished 27th February 2016

When it comes to comparability of cost-effectiveness results, I often find that the biggest problem is in deciphering which costs have and haven’t been included. Any efforts to encourage good practice are very welcome. In this paper, James Shearer and colleagues seek to provide practical guidance on costing approaches in mental health treatment settings. People in receipt of treatment for mental health problems often receive a diverse range of services. There are additional challenges because 3rd sector providers and other (often short lived) community-based specialist services can play an important role. It’s very important that a perspective be clearly defined along with the reasons for including or excluding particular costs. The authors run through some of the types of professionals that might be involved in mental health care and the types of resource use that might be relevant. They highlight numerous issues that – while not unique to mental health – represent particular challenges in this context. For example, it is particularly important to take into account training costs. Psychotherapies in particular often undergo development over time and therefore may require new training for providers. It’s also important to determine where care takes place, and whether the practitioner is working in the office or face-to-face with the patient. The authors identify five cost categories that need to be considered: social care, informal care, production losses, crime and education. They provide guidance on each of these in the context of mental health. Health economists might find the guidance on crime and education particularly helpful, as I suspect most of us are less familiar with identifying the relevant components of these types of costs. The paper isn’t designed to be a complete reference on costing methodology, but if you’re setting up a trial-based economic evaluation in mental health you’ll want to have this paper to hand.