Chris Sampson’s journal round-up for 8th May 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Verification of decision-analytic models for health economic evaluations: an overview. PharmacoEconomics [PubMed] Published 29th April 2017

Increasingly, it’s expected that model-based economic evaluations can be validated and shown to be fit-for-purpose. However, up to now, discussions have focussed on scientific questions about conceptualisation and external validity, rather than technical questions, such as whether the model is programmed correctly and behaves as expected. This paper looks at how things are done in the software industry with a view to creating guidance for health economists. Given that Microsoft Excel remains one of the most popular software packages for modelling, there is a discussion of spreadsheet errors. These might be errors in logic, simple copy-paste type mistakes and errors of omission. A variety of tactics is discussed. In particular, the authors describe unit testing, whereby individual parts of the code are demonstrated to be correct. Unit testing frameworks do not exist for application to spreadsheets, so the authors recommend the creation of a ‘Tests’ spreadsheet with tests for parameter assignments, functions, equations and exploratory items. Independent review by another modeller is also recommended. Six recommendations are given for taking model verification forward: i) the use of open source models, ii) standardisation in model storage and communication (anyone for a registry?), iii) style guides for script, iv) agency and journal mandates, v) training and vi) creation of an ISPOR/SMDM task force. This is a worthwhile read for any modeller, with some neat tactics that you can build into your workflow.

How robust are value judgments of health inequality aversion? Testing for framing and cognitive effects. Medical Decision Making [PubMed] Published 25th April 2017

Evidence shows that people are often extremely averse to health inequality. Sometimes these super-egalitarian responses imply such extreme preferences that monotonicity is violated. The starting point for this study is the idea that these findings are probably influenced by framing effects and cognitive biases, and that they may therefore not constitute a reliable basis for policy making. The authors investigate 4 hypotheses that might indicate the presence of bias: i) realistic small health inequality reductions vs larger one, ii) population- vs individual-level descriptions, iii) concrete vs abstract intervention scenarios and iv) online vs face-to-face administration. Two samples were recruited: one with a face-to-face discussion (n=52) and the other online (n=83). The questionnaire introduced respondents to health inequality in England before asking 4 questions in the form of a choice experiment, with 20 paired choices. Responses are grouped according to non-egalitarianism, prioritarianism and strict egalitarianism. The main research question is whether or not the alternative strategies resulted in fewer strict egalitarian responses. Not much of an effect was found with regard to large gains or to population-level descriptions. There was evidence that the abstract scenarios resulted in a greater proportion of people giving strong egalitarian responses. And the face-to-face sample did seem to exhibit some social desirability bias, with more egalitarian responses. But the main take-home message from this study for me is that it is not easy to explain-away people’s extreme aversion to health inequality, which is heartening. Yet, as with all choice experiments, we see that the mode of administration – and cognitive effects induced by the question – can be very important.

Adaptation to health states: sick yet better off? Health Economics [PubMed] Published 20th April 2017

Should patients or the public value health states for the purpose of resource allocation? It’s a question that’s cropped up plenty of times on this blog. One of the trickier challenges is understanding and dealing with adaptation. This paper has a pretty straightforward purpose – to look for signs of adaptation in a longitudinal dataset. The authors’ approach is to see whether there is a positive relationship between the length of time a person has an illness and the likelihood of them reporting better health. I did pretty much the same thing (for SF-6D and satisfaction with life) in my MSc dissertation, and found little evidence of adaptation, so I’m keen to see where this goes! The study uses 4 waves of data from the British Cohort Study, looking at self-assessed health (on a 4-point scale) and self-reported chronic illness and health shocks. Latent self-assessed health is modelled using a dynamic ordered probit model. In short, there is evidence of adaptation. People who have had a long-standing illness for a greater duration are more likely to report a higher level of self-assessed health. An additional 10 years of illness is associated with an 8 percentage point increase in the likelihood of reporting ‘excellent’ health. The study is opaque about sample sizes, but I’d guess that finding is based on not-that-many people. Further analyses are conducted to show that adaptation seems to become important only after a relatively long duration (~20 years) and that better health before diagnosis may not influence adaptation. The authors also look at specific conditions, finding that some (e.g. diabetes, anxiety, back problems) are associated with adaptation, while others (e.g. depression, cancer, Crohn’s disease) are not. I have a bit of a problem with this study though, in that it’s framed as being relevant to health care resource allocation and health technology assessment. But I don’t think it is. Self-assessed health in the ‘how healthy are you’ sense is very far removed from the process by which health state utilities are obtained using the EQ-5D. And they probably don’t reflect adaptation in the same way.


Chris Sampson’s journal round-up for 14th November 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Weighing clinical evidence using patient preferences: an application of probabilistic multi-criteria decision analysis. PharmacoEconomics [PubMedPublished 10th November 2016

There are at least two ways in which preferences determine the allocation of health care resources (in a country with a HTA agency, at least). One of them we think about a lot; the (societal) valuation of health states as defined by a multi-attribute measure (like the EQ-5D). The other relates to patient preferences that determine whether or not a specific individual (and their physician) will choose to use a particular technology, given its expected clinical outcomes for that individual. A drug may very well make sense at the aggregate level but be a very bad choice for a particular individual when compared with alternatives. It’s right that this process should be deliberative and not solely driven by an algorithm, but it’s also important to maintain transparent and consistent decision making. Multi-criteria decision analysis (MCDA) has been proposed as a means of achieving this, and it can be used to take into account the uncertainty associated with clinical outcomes. In this study the authors present an approach that also incorporates random preference variation along with parameter uncertainty in both preferences and clinical evidence. The model defines a value function and estimates the impact of uncertainty using probabilistic Monte Carlo simulation, which in turn estimates the mean value of each possible treatment in the population. Treatments can therefore be ranked according to patients’ preferences, along with an estimate of the uncertainty associated with this ranking. To demonstrate the utility of the model it is applied to an example for the relative value of HAARTs for HIV, with parameters derived from clinical evaluations and stated preferences studies. It’s nice to see that the authors also provide their R script. One headline finding seems to be that this approach is likely to demonstrate just how much uncertainty is involved that might not previously have been given much attention. It could therefore help steer us towards more valuable research in the future. And it could be used to demonstrate that optimal decisions might change when all sources of uncertainty are considered. Clearly a potential application of this method is in the realm of personalised medicine, which is slowly but inevitably reaching beyond the confines of pharmacogenomics.

Communal sharing and the provision of low-volume high-cost health services: results of a survey. PharmacoEconomics – Open Published 4th November 2016

One of the distributional concerns we might have about the QALY-maximisation approach is its implications for people with rare diseases. Drugs for rare diseases are often expensive (because the marginal cost is likely to be higher) and therefore less cost-effective. There is mixed evidence about whether or not people exhibit a preference for redistributive allocation of QALY-creating resources according to rarity. Of course, the result you get from such studies is dependent on the question you ask. In order to ask the right question it’s important to understand the mechanisms by which people might prefer allocation of additional resources to services for rare diseases. One suggestion in the literature is the preservation of hope. This study presents another, based on the number of people sharing the cost. So imagine a population of 1000 people, and all those people share the cost of health care. For a rare disease, more people will share the cost of the treatment per person treated. So if 10 people have the disease, that’s 100 payers per recipient. If 100 people have the disease then it’s just 10 payers per recipient. The idea is that people prefer a situation in which more people share the cost, and on that basis prefer to allocate resources to rare diseases. A web-based survey was conducted in Australia in which 702 people were asked to divide a budget between a small patient group with a high-cost illness and a large patient group with a low-cost illness. There were also a set of questions in which respondents indicated the importance of 6 possible influences on their decisions. The findings show that people did choose to allocate more funds to the rarer disease, despite the reduced overall health gain. This suggests that people do have a preference for wider cost sharing, which could explain extra weight being given to rare diseases. I think it’s a good idea that deserves more research, but for me there are a few problems with the study. Much of the effect could be explained by people’s non-linear valuations of risk, as the scenario highlighted that the respondents themselves would be at risk of the disease. We also can’t clearly differentiate between an effect due to the rarity of the disease (and associated cost sharing) and an effect due to the severity of the disease.

The challenge of conditional reimbursement: stopping reimbursement can be more difficult than not starting in the first place! Value in Health Published 3rd November 2016

If anything’s going to make me read a paper, it’s an exclamation mark! Conditional reimbursement of technologies that are probably effective but probably not cost-effective can be conducted in a rational way in order to generate research findings and benefit social welfare in the long run. But that can only hold true if those technologies subsequently found (through more research) to be ineffective or too costly are then made unavailable. Otherwise conditional reimbursement agreements will do more harm than good. This study uses discrete choice experiments to compare public (n=1169) and potential policymaker (n=90) values associated with the removal of an available treatment compared with non-reimbursement of a new treatment. The results showed (in addition to some other common findings) that both the public and policymakers preferred reimbursement of an existing treatment over the reimbursement of a new treatment, and were willing to accept an ICER of more than €7,000 higher for an existing treatment. Though the DCE found it to be a significant determinant, 60% of policymakers reported that they thought that reimbursement status was unimportant, so there may be some cognitive dissonance going on there. The most obvious (and probably most likely) explanation for the observed preference for currently reimbursed treatments is loss aversion. But it could also be that people recognise real costs associated with ending reimbursement that are not reflected in either the QALY estimates or the costs to the health system. Whatever the explanation, HTA agencies need to bear this in mind when using conditional reimbursement agreements.

Head-to-head comparison of health-state values derived by a probabilistic choice model and scores on a visual analogue scale. The European Journal of Health Economics [PubMed] Published 2nd November 2016

I’ve always had a fondness for a good old VAS as a direct measure of health state (dare we say utility) values, despite the limitations of the approach. This study compares discrete choices for EQ-5D-5L states with VAS valuations – thus comparing indirect and direct health state valuations – in Canada, the USA, England and The Netherlands (n=1775). Each respondent had to make a forced choice between two EQ-5D-5L health states and then assess both states on a single VAS. Ten different pairs were completed by each respondent. The two different approaches correlated strongly within and across countries, as we might expect. And pairs of EQ-5D-5L states that were valued relatively low or high in the discrete choice model were also valued accordingly in the VAS. But the relationship between the two approaches was non-linear in that values differed more at the ends of the scale, with poor health states valued more differently in the choice model and good health states valued more differently on the VAS. This probably just reflects some of the biases observed in the use of VAS that are already well-documented, particularly context bias and end-state aversion. This study clearly suggests (though does not by itself prove) that discrete choice models are a better choice for health state valuation… but the VAS ain’t dead yet.


Public or patient preferences: ex ante, ex post… extraneous?

As alluded to in yesterday’s journal round-up, on reading a recent article by Versteegh and Brouwer, I have had some thoughts about the way we think about the the debate between the use of either patient or public preferences for health state valuation.

When it comes to valuing health states, NICE (and some of their counterparts) advise the use of preferences from the general public. An alternative argument is that we might use patient preferences, because the public probably do not have an accurate understanding of what it’s like to live in a particular health state. In their new paper, Versteegh and Brouwer outline the key arguments in favour of using public preferences but highlight the limited nature of these arguments. One thing they discuss is the notion that public preferences are ex ante while patient preferences are ex post. It’s analogous to preferences vs satisfaction, or decision utility vs experienced utility. The authors outline some limitations to this interpretation. In this blog post I’d like to build on this discussion. My main focus is on defining what we actually mean when we talk about ‘patient preferences’.

Before and after what?

Ex ante means ‘before the event’, and ex post after it. But when we are valuing health states there is no event before or after which utility can be estimated. We are trying to value a state, not preferences regarding an event. We may contrive an event – such as the onset of a particular health state – but that is theoretically quite a different thing to value. Indeed, this contrivance of an event taking place may be a problem.

We should probably do away with these terms and just speak in English, but let’s be realistic. At the very least, we need to be clear about what ex ante and ex post mean in this context; the ‘event’ in question is experience of the given health state.

But then, health state valuation isn’t about just one health state – it’s only possible to value health states in relation to one another and in particular ‘full health’ and a state equivalent to being dead. Furthermore, there is little doubt that a person’s valuation of past or future health states relates to their current health state. Chances are that any individual completing a health state valuation will be valuing some states from an ex ante position and some from an ex post position, both of which are influenced by their current health status.

Ultimately, whether preferences being elicited are ex ante or ex post has nothing to do with whom is being asked, and everything to do with what they are being asked about.


But that isn’t the crux of the matter anyway. What we really want to do here is differentiate between ‘patient preferences’ and ‘public preferences’. ‘The public’ is easy to define. It’s everyone. We usually try to get a representative sample because we cannot ask everyone to do a TTO exercise. But we need to be clearer about how we define patients. Patients are not ex ante – that we can agree on. Or can we? What if we ask an individual about an inevitable future health state associated with disease progression, of which they have a good understanding? What’s worse, patients might also not be ex post, depending on our definition of these terms.

It seems far more intuitive and accurate to describe patients as ex tempore: essentially meaning ‘at the time’. Patients’ health state preferences are neither retrospective nor prospective, but explicitly in relation to their current health state. Crucially, it is that current health state that we are trying to value.

So, a person valuing their own health state is doing so ex tempore, and that’s usually what we mean by ‘patient preferences’. But I hope it’s clear by this point that an individual patient’s preferences need not necessarily be ex tempore either.

People who have never experienced a given health state are necessarily stating their preferences ex ante, but they could still be a patient or not. Meanwhile, somebody who does have experience of a health state could be valuing it from any of the alternative temporal positions. They may, for example, be valuing a future in a health state that they have previously experienced. Versteegh and Brouwer provide a nice taxonomy of the arguments for the use of public preferences. I’d like to provide my own taxonomy here, of the different types of preferences we might elicit. I see it as follows:

Experience of health state No experience of health state
  ex ante ex tempore ex post ex ante ex tempore ex post
Patient A1 A2 A3 B
Non-patient C1 C2 C3 D

There are 4 types of responder (A, B, C and D), determined by whether they are a patient and whether they have previously experienced the health state currently being valued. Similarly, there are 3 different types of health state valuation, depending on whether the state being valued is a past, present or future state. For any given person valuing any given health state, the elicited preferences will be one of the labelled boxes. Ask that same person to value a different health state, or ask a different person to value the same health state, and the elicited preference may well differ.

There may of course be other ways in which individuals differ, such as the extent to which they have adapted to their current health state. But while that’s an important consideration in determining from whom we ought to elicit preferences, I don’t think it’s a key question in identifying patient preferences as opposed to public preferences.

Patient vs patients

One implication of this is that we have (at least) two types of patient preferences. Patient preferences could be A+B. That is, we value a particular health state in all patients, regardless of their current health state. That might be done in a sample representative of the current population of people considered to be a patient, however that might be defined. It strikes me that this is the true definition of patienthood as might be used in other contexts.

The kind of patient we talk about when we discuss ‘patient preferences’ is, I think, just those people falling into ‘A2’; patients valuing their own current health state.

Versteegh and Brouwer seem to suggest that any valuation of current health – that is, ex tempore – represents patient preferences. In practice this will likely be ‘A2’ through the identification of participants, but it’s important to consider the existence of ‘C2’. Just because a person is experiencing the health state of interest does not necessarily make them a patient in any practical sense of the word.

For what it’s worth, I think that public preferences are the least bad option for now. But Versteegh and Brouwer’s suggestion that we should report both is a good one, which could lead to more research that may very well change my mind. I think it will also force this issue of clearer definition of ‘patient preferences’.

Photo credit: Tori Cat (CC BY-NC-ND 2.0)