Chris Sampson’s journal round-up for 16th December 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

MCDA-based deliberation to value health states: lessons learned from a pilot study. Health and Quality of Life Outcomes [PubMed] Published 1st July 2019

The rejection of the EQ-5D-5L value set for England indicates something of a crisis in health state valuation. Evidently, there is a lack of trust in the quantitative data and methods used. This is despite decades of methodological development. Perhaps we need a completely different approach. Could we instead develop a value set using qualitative methods?

A value set based on qualitative research aligns with an idea forwarded by Daniel Hausman, who has argued for the use of deliberative approaches. This could circumvent the problems associated with asking people to give instant (and possibly ill-thought-out) responses to preference elicitation surveys. The authors of this study report on the first ever (pilot) attempt to develop a consensus value set using methods of multi-criteria decision analysis (MCDA) and deliberation. The study attempts to identify a German value set for the SF-6D.

The study included 34 students in a one-day conference setting. A two-step process was followed for the MCDA using MACBETH (the Measuring Attractiveness by a Categorical Based Evaluation Technique), which uses pairwise comparisons to derive numerical scales without quantitative assessments. First, a scoring procedure was conducted for each of the six dimensions. Second, a weighting was identified for each dimension. After an introductory session, participants were allocated into groups of five or six and each group was tasked with scoring one SF-6D dimension. Within each group, consensus was achieved. After these group sessions, all participants were brought together to present and validate the results. In this deliberation process, consensus was achieved for all domains except pain. Then the weighting session took place, but resulted in no consensus. Subsequent to the one-day conference, a series of semi-structured interviews were conducted with moderators. All the sessions and interviews were recorded, transcribed, and analysed qualitatively.

In short, the study failed. A consensus value set could not be identified. Part of the problem was probably in the SF-6D descriptive system, particularly in relation to pain, which was interpreted differently by different people. But the main issue was that people had different opinions and didn’t seem willing to move towards consensus with a societal perspective in mind. Participants broadly fell into three groups – one in favour of prioritising pain and mental health, one opposed to trading-off SF-6D dimensions and favouring equal weights, and another group that was not willing to accept any trade-offs.

Despite its apparent failure, this seems like an extremely useful and important study. The authors provide a huge amount of detail regarding what they did, what went well, and what might be done differently next time. I’m not sure it will ever be possible to get a group of people to reach a consensus on a value set. The whole point of preference-based measures is surely that different people have different priorities, and they should be expected to disagree. But I think we should expect that the future of health state valuation lies in mixed methods. There might be more success in a qualitative and deliberative approach to scoring combined with a quantitative approach to weighting, or perhaps a qualitative approach informed by quantitative data that demands trade-offs. Whatever the future holds, this study will be a valuable guide.

Preference-based health-related quality of life outcomes associated with preterm birth: a systematic review and meta-analysis. PharmacoEconomics [PubMed] Published 9th December 2019

Premature and low birth weight babies can experience a whole host of negative health outcomes. Most studies in this context look at short-term biomedical assessments or behavioural and neurodevelopmental indicators. But some studies have sought to identify the long-term consequences on health-related quality of life by identifying health state utility values. This study provides us with a review and meta-analysis of such values.

The authors screened 2,139 articles from their search and included 20 in the review. Lots of data were extracted from the articles, which is helpfully tabulated in the paper. The majority of the studies included adolescents and focussed on children born very preterm or at very low birth weight.

For the meta-analysis, the authors employed a linear mixed-effects meta-regression, which is an increasingly routine approach in this context. The models were used to estimate the decrement in utility values associated with preterm birth or low birth weight, compared with matched controls. Conveniently, all but one of the studies used a measure other than the HUI2 or HUI3, so the analysis was restricted to these two measures. Preterm birth was associated with an average decrement of 0.066 and extremely low birth weight with a decrement of 0.068. The mean estimated utility scores for the study groups was 0.838, compared with 0.919 for the control groups.

Reviews of utility values are valuable as they provide modellers with a catalogue of potential parameters that can be selected in a meaningful and transparent way. Even though this is a thorough and well-reported study, it’s a bit harder to see how its findings will be used. Most reviews of utility values relate to a particular disease, which might be prevented or ameliorated by treatment, and the value of this treatment depends on the utility values selected. But how will these utility values be used? The avoidance of preterm or low-weight birth is not the subject of most evaluations in the neonatal setting. Even if it was, how valuable are estimates from a single point in adolescence? The authors suggest that future research should seek to identify a trajectory of utility values over the life course. But, even if we could achieve this, it’s not clear to me how this should complement utility values identified in relation to the specific health problems experienced by these people.

The new and non-transparent Cancer Drugs Fund. PharmacoEconomics [PubMed] Published 12th December 2019

Not many (any?) health economists liked the Cancer Drugs Fund (CDF). It was set-up to give special treatment to cancer drugs, which weren’t assessed on the same basis as other drugs being assessed by NICE. In 2016, the CDF was brought within NICE’s remit, with medicines available through the CDF requiring a managed access agreement. This includes agreements on data collection and on payments by the NHS during the period. In this article, the authors contend that the new CDF process is not sufficiently transparent.

Three main issued are raised: i) lack of transparency relating to the value of CDF drugs, ii) lack of transparency relating to the cost of CDF drugs, and iii) the amount of time that medicines remain on the CDF. The authors tabulate the reporting of ICERs according to the decisions made, showing that the majority of treatment comparisons do not report ICERs. Similarly, the time in the CDF is tabulated, with many indications being in the CDF for an unknown amount of time. In short, we don’t know much about medicines going through the CDF, except that they’re probably costing a lot.

I’m a fan of transparency, in almost all contexts. I think it is inherently valuable to share information widely. It seems that the authors of this paper do too. A lack of transparency in NICE decision-making is a broader problem that arises from the need to protect commercially sensitive pricing agreements. But what this paper doesn’t manage to do is to articulate why anybody who doesn’t support transparency in principle should care about the CDF in particular. Part of the authors’ argument is that the lack of transparency prevents independent scrutiny. But surely NICE is the independent scrutiny? The authors argue that it is a problem that commissioners and the public cannot assess the value of the medicines, but it isn’t clear why that should be a problem if they are not the arbiters of value. The CDF has quite rightly faced criticism over the years, but I’m not convinced that its lack of transparency is its main problem.

Credits

Chris Sampson’s journal round-up for 2nd December 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The treatment decision under uncertainty: the effects of health, wealth and the probability of death. Journal of Health Economics Published 16th November 2019

It’s important to understand how people make decisions about treatment. At the end of life, the question can become a matter of whether to have treatment or to let things take their course such that you end up dead. In order to consider this scenario, the author of this paper introduces the probability of death to some existing theoretical models of decision-making under uncertainty.

The diagnostic risk model and the therapeutic risk model can be used to identify risk thresholds that determine decisions about treatment. The diagnostic model relates to the probability that disease is present and the therapeutic model relates to the probability that treatment is successful. The new model described in this paper builds on these models to consider the impact on the decision thresholds of i) initial health state, ii) probability of death, and iii) wealth. The model includes wealth after death, in the form of a bequest. Limited versions of the model are also considered, excluding the bequest and excluding wealth (described as a ‘QALY model’). Both an individual perspective and an aggregate perspective are considered by excluding and including the monetary cost of diagnosis and treatment, to allow for a social insurance type setting.

The comparative statics show a lot of ambiguity, but there are a few things that the model can tell us. The author identifies treatment as having an ‘insurance effect’, by reducing diagnostic risk, a ‘protective effect’, by lowering the probability of death, and a risk-increasing effect associated with therapeutic risk. A higher probability of death increases the propensity for treatment in both the no-bequest model and the QALY model, because of the protective effect of treatment. In the bequest model, the impact is ambiguous, because treatment costs reduce the bequest. In the full model, wealthier individuals will choose to undergo treatment at a lower probability of success because of a higher marginal utility for survival, but the effect becomes ambiguous if the marginal utility of wealth depends on health (which it obviously does).

I am no theoretician, so it can take me a long time to figure these things out in my head. For now, I’m not convinced that it is meaningful to consider death in this way using a one-period life model. In my view, the very definition of death is a loss of time, which plays little or no part in this model. But I think my main bugbear is the idea that anybody’s decision about life saving treatment is partly determined by the amount of money they will leave behind. I find this hard to believe. The author links the finding that a higher probability of death increases treatment propensity to NICE’s end of life premium. Though I’m not convinced that the model has anything to do with NICE’s reasoning on this matter.

Moving toward evidence-based policy: the value of randomization for program and policy implementation. JAMA [PubMed] Published 15th November 2019

Evidence-based policy is a nice idea. We should figure out whether something works before rolling it out. But decision-makers (especially politicians) tend not to think in this way, because doing something is usually seen to be better than doing nothing. The authors of this paper argue that randomisation is the key to understanding whether a particular policy creates value.

Without evidence based on random allocation, it’s difficult to know whether a policy works. This, the authors argue, can undermine the success of effective interventions and allow harmful policies to persist. A variety of positive examples are provided from US healthcare, including trials of Medicare bundled payments. Apparently, such trials increased confidence in the programmes’ effects in a way that post hoc evaluations cannot, though no evidence of this increased confidence is actually provided. Policy evaluation is not always easy, so the authors describe four preconditions for the success of such studies: i) early engagement with policymakers, ii) willingness from policy leaders to support randomisation, iii) timing the evaluation in line with policymakers’ objectives, and iv) designing the evaluation in line with the realities of policy implementation.

These are sensible suggestions, but it is not clear why the authors focus on randomisation. The paper doesn’t do what it says on the tin, i.e. describe the value of randomisation. Rather, it explains the value of pre-specified policy evaluations. Randomisation may or may not deserve special treatment compared with other analytical tools, but this paper provides no explanation for why it should. The authors also suggest that people are becoming more comfortable with randomisation, as large companies employ experimental methods, particularly on the Internet with A/B testing. I think this perception is way off and that most people feel creeped out knowing that the likes of Facebook are experimenting on them without any informed consent. In the authors’ view, it being possible to randomise is a sufficient basis on which to randomise. But, considering the ethics, as well as possible methodological contraindications, it isn’t clear that randomisation should become the default.

A new tool for creating personal and social EQ-5D-5L value sets, including valuing ‘dead’. Social Science & Medicine Published 30th November 2019

Nobody can agree on the best methods for health state valuation. Or, at least, some people have disagreed loud enough to make it seem that way. Novel approaches to health state valuation are therefore welcome. Even more welcome is the development and testing of methods that you can try at home.

This paper describes the PAPRIKA method (Potentially All Pairwise RanKings of all possible Alternatives) of discrete choice experiment, implemented using 1000Minds software. Participants are presented with two health states that are defined in terms of just two dimensions, each lasting for 10 years, and asked to choose between them. Using the magical power of computers, an adaptive process identifies further choices, automatically ranking states using transitivity so that people don’t need to complete unnecessary tasks. In order to identify where ‘dead’ sits on the scale, a binary search procedure asks participants to compare EQ-5D states with being dead. What’s especially cool about this process is that everybody who completes it is able to view their own personal value set. These personal value sets can then be averaged to identify a social value set.

The authors used their tool to develop an EQ-5D-5L value set for New Zealand (which is where the researchers are based). They recruited 5,112 people in an online panel, such that the sample was representative of the general public. Participants answered 20 DCE questions each, on average, and almost half of them said that they found the questions difficult to answer. The NZ value set showed that anxiety/depression was associated with the greatest disutility, though each dimension has a notably similar level of impact at each level. The value set correlates well with numerous existing value sets.

The main limitation of this research seems to be that only levels 1, 3, and 5 of each EQ-5D-5L domain were included. Including levels 2 and 4 would more than double the number of questions that would need to be answered. It is also concerning that more than half of the sample was excluded due to low data quality. But the authors do a pretty good job of convincing us that this is for the best. Adaptive designs of this kind could be the future of health state valuation, especially if they can be implemented online, at low cost. I expect we’ll be seeing plenty more from PAPRIKA.

Credits

David Mott’s journal round-up for 16th September 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Opening the ‘black box’: an overview of methods to investigate the decision‑making process in choice‑based surveys. The Patient [PubMed] Published 5th September 2019

Choice-based surveys using methods such as discrete choice experiments (DCEs) and best-worst scaling (BWS) exercises are increasingly being used in health to understand people’s preferences. A lot of time and energy is spent on analysing the data that come out from these surveys but increasingly there is an interest in better understanding respondents’ decision-making processes. Whilst many will be aware of ‘think aloud’ interviews (often used for piloting), other methods may be less familiar as they’re not applied frequently in health. That’s where this fascinating paper by Dan Rigby and colleagues comes in. It provides an overview of five different methods of what they call ‘pre-choice process analysis’ of decision-making, describing the application, state of knowledge, and future research opportunities.

Eye-tracking has been used in health recently. It’s intuitive and provides an insight into where the participants’ focus is (or isn’t). The authors explained that one of the ways it has been used is to explore attribute non-attendance (ANA), which essentially occurs when people are ignoring attributes either because they’re irrelevant to them, or simply because it makes the task easier. However, surprisingly, it has been suggested that ‘visual ANA’ (not looking at the attribute) doesn’t always align with ‘stated ANA’ (participants stating that they ignored the attribute) – which raises some interesting questions!

However, the real highlight for me was the overview of the use of brain imaging techniques to explore choices being made in DCEs. One study highlighted by the authors – which was a DCE about eggs and is now at least #2 on my list of the bizarre preference study topics after this oddly specific one on Iberian ham – predicted choices from an initial ‘passive viewing’ using functional magnetic resonance imaging (fMRI). They found that incorporating changes in blood flow (prompted by changes in attribute levels during ‘passive viewing’) into a random utility model accounted for a lot of the variation in willingness to pay for eggs – pretty amazing stuff.

Whilst I’ve highlighted the more unusual methods here, after reading this overview I have to admit that I’m an even bigger advocate for the ‘think aloud’ technique now. Although it may have some limitations, the amount of insight offered combined with its practicality is hard to beat. Though maybe I’m biased because I know that I won’t get my hands on any eye-tracking or brain imaging devices any time soon. In any case, I highly recommend that any researchers conducting preference studies give this paper a read as it’s really well written and will surely be of interest.

Disentangling public preferences for health gains at end-of-life: further evidence of no support of an end-of-life premium. Social Science & Medicine [PubMed] Published 21st June 2019

The end of life (EOL) policy introduced by NICE in 2009 [PDF] has proven controversial. The policy allows treatments that are not cost-effective within the usual range to be considered for approval, provided that certain criteria are met. Specifically, that the treatment targets patients with a short life expectancy (≤24 months), offers a life extension (of ≥3 months) and is for a ‘small patient population’. One of the biggest issues with this policy is that it is unclear whether the general population actually supports the idea of valuing health gains (specifically life extension) at EOL more than other health gains.

Numerous academic studies, usually involving some form of stated preference exercise, have been conducted to test whether the public might support this EOL premium. A recent review by Koonal Shah and colleagues summarised the existing published studies (up to October 2017), highlighting that evidence is extremely mixed. This recently published Danish study, by Lise Desireé Hansen and Trine Kjær, adds to this literature. The authors conducted an incredibly thorough stated preference exercise to test whether quality of life (QOL) gains and life extension (LE) at EOL are valued differently from other similarly sized health gains. Not only that, but the study also explored the effect of perspective on results (social vs individual), the effect of age (18-35 vs. 65+), and impact of initial severity (25% vs. 40% initial QOL) on results.

Overall, they did not find evidence of support for an EOL premium for QOL gains or for LEs (regardless of perspective) but their results do suggest that QOL gains are preferred over LE. In some scenarios, there was slightly more support for EOL in the social perspective variant, relative to the individual perspective – which seems quite intuitive. Both age and initial severity had an impact on results, with respondents preferring to treat the young and those with worse QOL at baseline. One of the most interesting results for me was within their subgroup analyses, which suggested that women and those with a relation to a terminally ill patient had a significantly positive preference for EOL – but only in the social perspective scenarios.

This is a really well-designed study, which covers a lot of different concepts. This probably doesn’t end the debate on NICE’s use of the EOL criteria – not least because the study wasn’t conducted in England and Wales – but it contributes a lot. I’d consider it a must-read for anyone interested in this area.

How should we capture health state utility in dementia? Comparisons of DEMQOL-Proxy-U and of self- and proxy-completed EQ-5D-5L. Value in Health Published 26th August 2019

Capturing quality of life (QOL) in dementia and obtaining health state utilities is incredibly challenging; which is something that I’ve started to really appreciate recently upon getting involved in a EuroQol-funded ‘bolt-ons’ project. The EQ-5D is not always able to detect meaningful changes in cognitive function and condition-specific preference-based measures (PBMs), such as the DEMQOL, may be preferred as a result. However, this isn’t the only challenge because in many cases patients are not in a position to complete the surveys themselves. This means that proxy-reporting is often required, which could be done by either a professional (formal) carer, or a friend or family member (informal carer). Researchers that want to use a PBM in this population therefore have a lot to consider.

This paper compares the performance of the EQ-5D-5L and the DEMQOL-Proxy when completed by care home residents (EQ-5D-5L only), formal carers and informal carers. The impressive dataset that the authors use contains 1,004 care home residents, across up to three waves, and includes a battery of different cognitive and QOL measures. The overall objective was to compare the performance of the EQ-5D-5L and DEMQOL-Proxy, across the three respondent groups, based on 1) construct validity, 2) criterion validity, and 3) responsiveness.

The authors found that self-reported EQ-5D-5L scores were larger and less responsive to changes in the cognitive measures, but better at capturing residents’ self-reported QOL (based on a non-PBM) relative to proxy-reported scores. It is unclear whether this is a case of adaptation as seen in many other patient groups, or if the residents’ cognitive impairments prevent them from reliably assessing their current status. The proxy-reported EQ-5D-5L scores were generally more responsive to changes in the cognitive measures relative to the DEMQOL-Proxy (irrespective of which type of proxy), which the authors note is probably due to the fact that the DEMQOL-Proxy focuses more on the emotional impact of dementia rather than functional impairment.

Overall, this is a really interesting paper, which highlights the challenges well and illustrates that there is value in collecting these data from both patients and proxies. In terms of the PBM comparison, whilst the authors do not explicitly state it, it does seem that the EQ-5D-5L may have a slight upper hand due to its responsiveness, as well as for pragmatic reasons (the DEMQOL-Proxy has >30 questions). Perhaps a cognition ‘bolt-on’ to the EQ-5D-5L might help to improve the situation in future?

Credits