David Mott’s journal round-up for 16th September 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Opening the ‘black box’: an overview of methods to investigate the decision‑making process in choice‑based surveys. The Patient [PubMed] Published 5th September 2019

Choice-based surveys using methods such as discrete choice experiments (DCEs) and best-worst scaling (BWS) exercises are increasingly being used in health to understand people’s preferences. A lot of time and energy is spent on analysing the data that come out from these surveys but increasingly there is an interest in better understanding respondents’ decision-making processes. Whilst many will be aware of ‘think aloud’ interviews (often used for piloting), other methods may be less familiar as they’re not applied frequently in health. That’s where this fascinating paper by Dan Rigby and colleagues comes in. It provides an overview of five different methods of what they call ‘pre-choice process analysis’ of decision-making, describing the application, state of knowledge, and future research opportunities.

Eye-tracking has been used in health recently. It’s intuitive and provides an insight into where the participants’ focus is (or isn’t). The authors explained that one of the ways it has been used is to explore attribute non-attendance (ANA), which essentially occurs when people are ignoring attributes either because they’re irrelevant to them, or simply because it makes the task easier. However, surprisingly, it has been suggested that ‘visual ANA’ (not looking at the attribute) doesn’t always align with ‘stated ANA’ (participants stating that they ignored the attribute) – which raises some interesting questions!

However, the real highlight for me was the overview of the use of brain imaging techniques to explore choices being made in DCEs. One study highlighted by the authors – which was a DCE about eggs and is now at least #2 on my list of the bizarre preference study topics after this oddly specific one on Iberian ham – predicted choices from an initial ‘passive viewing’ using functional magnetic resonance imaging (fMRI). They found that incorporating changes in blood flow (prompted by changes in attribute levels during ‘passive viewing’) into a random utility model accounted for a lot of the variation in willingness to pay for eggs – pretty amazing stuff.

Whilst I’ve highlighted the more unusual methods here, after reading this overview I have to admit that I’m an even bigger advocate for the ‘think aloud’ technique now. Although it may have some limitations, the amount of insight offered combined with its practicality is hard to beat. Though maybe I’m biased because I know that I won’t get my hands on any eye-tracking or brain imaging devices any time soon. In any case, I highly recommend that any researchers conducting preference studies give this paper a read as it’s really well written and will surely be of interest.

Disentangling public preferences for health gains at end-of-life: further evidence of no support of an end-of-life premium. Social Science & Medicine [PubMed] Published 21st June 2019

The end of life (EOL) policy introduced by NICE in 2009 [PDF] has proven controversial. The policy allows treatments that are not cost-effective within the usual range to be considered for approval, provided that certain criteria are met. Specifically, that the treatment targets patients with a short life expectancy (≤24 months), offers a life extension (of ≥3 months) and is for a ‘small patient population’. One of the biggest issues with this policy is that it is unclear whether the general population actually supports the idea of valuing health gains (specifically life extension) at EOL more than other health gains.

Numerous academic studies, usually involving some form of stated preference exercise, have been conducted to test whether the public might support this EOL premium. A recent review by Koonal Shah and colleagues summarised the existing published studies (up to October 2017), highlighting that evidence is extremely mixed. This recently published Danish study, by Lise Desireé Hansen and Trine Kjær, adds to this literature. The authors conducted an incredibly thorough stated preference exercise to test whether quality of life (QOL) gains and life extension (LE) at EOL are valued differently from other similarly sized health gains. Not only that, but the study also explored the effect of perspective on results (social vs individual), the effect of age (18-35 vs. 65+), and impact of initial severity (25% vs. 40% initial QOL) on results.

Overall, they did not find evidence of support for an EOL premium for QOL gains or for LEs (regardless of perspective) but their results do suggest that QOL gains are preferred over LE. In some scenarios, there was slightly more support for EOL in the social perspective variant, relative to the individual perspective – which seems quite intuitive. Both age and initial severity had an impact on results, with respondents preferring to treat the young and those with worse QOL at baseline. One of the most interesting results for me was within their subgroup analyses, which suggested that women and those with a relation to a terminally ill patient had a significantly positive preference for EOL – but only in the social perspective scenarios.

This is a really well-designed study, which covers a lot of different concepts. This probably doesn’t end the debate on NICE’s use of the EOL criteria – not least because the study wasn’t conducted in England and Wales – but it contributes a lot. I’d consider it a must-read for anyone interested in this area.

How should we capture health state utility in dementia? Comparisons of DEMQOL-Proxy-U and of self- and proxy-completed EQ-5D-5L. Value in Health Published 26th August 2019

Capturing quality of life (QOL) in dementia and obtaining health state utilities is incredibly challenging; which is something that I’ve started to really appreciate recently upon getting involved in a EuroQol-funded ‘bolt-ons’ project. The EQ-5D is not always able to detect meaningful changes in cognitive function and condition-specific preference-based measures (PBMs), such as the DEMQOL, may be preferred as a result. However, this isn’t the only challenge because in many cases patients are not in a position to complete the surveys themselves. This means that proxy-reporting is often required, which could be done by either a professional (formal) carer, or a friend or family member (informal carer). Researchers that want to use a PBM in this population therefore have a lot to consider.

This paper compares the performance of the EQ-5D-5L and the DEMQOL-Proxy when completed by care home residents (EQ-5D-5L only), formal carers and informal carers. The impressive dataset that the authors use contains 1,004 care home residents, across up to three waves, and includes a battery of different cognitive and QOL measures. The overall objective was to compare the performance of the EQ-5D-5L and DEMQOL-Proxy, across the three respondent groups, based on 1) construct validity, 2) criterion validity, and 3) responsiveness.

The authors found that self-reported EQ-5D-5L scores were larger and less responsive to changes in the cognitive measures, but better at capturing residents’ self-reported QOL (based on a non-PBM) relative to proxy-reported scores. It is unclear whether this is a case of adaptation as seen in many other patient groups, or if the residents’ cognitive impairments prevent them from reliably assessing their current status. The proxy-reported EQ-5D-5L scores were generally more responsive to changes in the cognitive measures relative to the DEMQOL-Proxy (irrespective of which type of proxy), which the authors note is probably due to the fact that the DEMQOL-Proxy focuses more on the emotional impact of dementia rather than functional impairment.

Overall, this is a really interesting paper, which highlights the challenges well and illustrates that there is value in collecting these data from both patients and proxies. In terms of the PBM comparison, whilst the authors do not explicitly state it, it does seem that the EQ-5D-5L may have a slight upper hand due to its responsiveness, as well as for pragmatic reasons (the DEMQOL-Proxy has >30 questions). Perhaps a cognition ‘bolt-on’ to the EQ-5D-5L might help to improve the situation in future?

Credits

Chris Sampson’s journal round-up for 19th August 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Paying for kidneys? A randomized survey and choice experiment. American Economic Review [RePEc] Published August 2019

This paper starts with a quote from Alvin Roth about ‘repugnant transactions’, of which markets for organs provide a prime example. This idea of ‘repugnant transactions’ has been hijacked by some pop economists to represent the stupid opinions of non-economists. If you ask me, markets for organs aren’t repugnant, they just seem like a very bad idea in terms of both efficiency and equity. But it doesn’t matter what I think; it matters what the people of the United States think.

The authors of this study conducted an online survey with a representative sample of 2,666 Americans. Each respondent was randomised to evaluate one of eight systems compared with the current system. The eight systems differed with respect to i) cash or non-cash compensation of ii) different sizes ($30,000 or $100,000), iii) paid by either a public agency or the organ recipient. Participants made five binary choices that differed according to the gain – in transplants generated – associated with the new system. Half of the participants were also asked to express moral judgements.

Both the system features (e.g. who pays) and the outcomes of the new system influenced people’s choices. Broadly speaking, the results suggest that people aren’t opposed to donors being paid, but are opposed to patients paying. (Remember, we’re talking about the US here!). Around 21% of respondents opposed payment no matter what, 46% were in favour no matter what, and 18% were sensitive to the gain in the number of transplants. A 10% point increase in transplants resulted in a 2.6% point increase in support. Unsurprisingly, individuals’ moral judgements were predictive of the attitudes they expressed, particularly with respect to fairness. The authors describe their results as exhibiting ‘strong polarisation’, which is surely inevitable for questions that involve moral judgement.

Being in AER, this is a long meandering paper with extensive analyses and thoroughly reported results. There’s lots of information and findings that I can’t share here. It’s a valuable study with plenty of food for thought, but I can’t help but think that it is, methodologically, a bit weak. If we want to understand the different views in society, surely some Q methodology would be more useful than a basic online survey. And if we want to elicit stated preferences, surely a discrete choice experiment with a well-thought-out efficient design would give us more meaningful results.

Estimating local need for mental healthcare to inform fair resource allocation in the NHS in England: cross-sectional analysis of national administrative data linked at person level. The British Journal of Psychiatry [PubMed] Published 8th August 2019

The need to fairly (and efficiently) allocate NHS resources across the country played an important part in the birth of health economics in the UK, and resulted in resource allocation formulas. Since 1996 there has been a separate formula for mental health services, which is periodically updated. This study describes the work undertaken for the latest update.

The model is based on predicting service use and total mental health care costs observed in 2015 from predictors in the years 2013-2014, to inform allocations in 2019-2024. Various individual-level data sources available to the NHS were used for 43.7 million people registered with a GP practice and over the age of 20. The cost per patient who used mental health services ranged from £94 to over one million, averaging around £2,000. The predictor variables included individual indicators such as age, sex, ethnicity, physical diagnoses, and household type (e.g. number of adults and kids). The model also used variables observed at the local or GP practice level, such as the proportion of people receiving out-of-work benefits and the distance from the mental health trust. All of this got plugged into a good old OLS regression. From individual-level predictions, the researchers created aggregated indices of need for each clinical commission group (CCG).

A lot went into the model, which explained 99% of the variation in costs between CCGs. A key way in which this model differs from previous versions is that it relies on individual-level indicators rather than those observed at the level of GP practice or CCG. There was a lot of variation in the CCG need indices, ranging from 0.65 for Surrey Heath to 1.62 for Southwark, where 1.00 is the average. You’ll need to check the online appendices for your own CCG’s level of need (Lewisham: 1.52). As one might expect, the researchers observed a strong correlation between a CCG’s need index and the CCG’s area’s level of deprivation. Compared with previous models, this new model indicates a greater allocation of resources to more deprived and older populations.

Measuring, valuing and including forgone childhood education and leisure time costs in economic evaluation: methods, challenges and the way forward. Social Science & Medicine [PubMed] Published 7th August 2019

I’m a ‘societal perspective’ sceptic, not because I don’t care about non-health outcomes (though I do care less) but because I think it’s impossible to capture everything that is of value to society, and that capturing just a few things will introduce a lot of bias and noise. I would also deny that time has any intrinsic value. But I do think we need to do a better job of evaluating interventions for children. So I expected this paper to provide me with a good mix of satisfaction and exasperation.

Health care often involves a loss of leisure or work time, which can constitute an opportunity cost and is regularly included in economic evaluations – usually proxied by wages – for adults. The authors outline the rationale for considering ‘time-related’ opportunity costs in economic evaluations and describe the nature of lost time for children. For adults, the distinction is generally between paid or unpaid work and leisure time. Arguably, this distinction is not applicable to children. Two literature reviews are described. One looked at economic evaluations in the context of children’s health, to see how researchers have valued lost time. The other sought to identify ideas about the value of lost time for children from a broader literature.

The authors do a nice job of outlining how difficult it is to capture non-health-related costs and outcomes in the context of childhood. There is a handful of economic evaluations that have tried to measure and value children’s foregone time. The valuations generally focussed on the costs of childcare rather than the costs to the child, though one looked at the rate of return to education. There wasn’t a lot to go off in the non-health literature, which mostly relates to adults. From what there is, the recommendation is to capture absence from formal education and foregone leisure time. Of course, consideration needs to be given to the importance of lost time and thus the value of capturing it in research. We also need to think about the risk of double counting. When it comes to measurement, we can probably use similar methods as we would for adults, such as diaries. But we need very different approaches to valuation. On this, the authors found very little in the way of good examples to follow. More research needed.

Credits

Thesis Thursday: David Mott

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr David Mott who has a PhD from Newcastle University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
How do preferences for public health interventions differ? A case study using a weight loss maintenance intervention
Supervisors
Luke Vale, Laura Ternent
Repository link
http://hdl.handle.net/10443/4197

Why is it important to understand variation in people’s preferences?

It’s not all that surprising that people’s preferences for health care interventions vary, but we don’t have a great understanding of what might drive these differences. Increasingly, preference information is being used to support regulatory decisions and, to a lesser but increasing extent, health technology assessments. It could be the case that certain subgroups of individuals would not accept the risks associated with a particular health care intervention, whereas others would. Therefore, identifying differences in preferences is important. However, it’s also useful to try to understand why this heterogeneity might occur in the first place.

The debate on whose preferences to elicit for health state valuation has traditionally focused on those with experience (e.g. patients) and those without (e.g. the general population). Though this dichotomy is problematic; it has been shown that health state utilities systematically differ between these two groups, presumably due to the difference in relative experience. My project aimed to explore whether experience also affects people’s preferences for health care interventions.

How did you identify different groups of people, whose preferences might differ?

The initial plan for the project was to elicit preferences for a health care intervention from general population and patient samples. However, after reviewing the literature, it seemed highly unlikely that anyone would advocate for preferences for treatments to be elicited from general population samples. It has long been suggested that discrete choice experiments (DCEs) could be used to incorporate patient preferences into decision-making, and it turned out that patients were the focus of the majority of the DCE studies that I reviewed. Given this, I took a more granular approach in my empirical work.

We recruited a very experienced group of ‘service users’ from a randomised controlled trial (RCT). In this case, it was a novel weight loss maintenance intervention aimed at helping obese adults that had lost at least 5% of their overall weight to maintain their weight loss. We also recruited an additional three groups from an online panel. The first group were ‘potential service users’ – those that met the trial criteria but could not have experienced the intervention. The second group were ‘potential beneficiaries’ – those that were obese or overweight and did not meet the trial criteria. The final group were ‘non-users’ – those with a normal BMI.

What can your study tell us about preferences in the context of a weight loss maintenance intervention?

The empirical part of my study involved a DCE and an open-ended contingent valuation (CV) task. The DCE was focused on the delivery of the trial intervention, which was a technology-assisted behavioural intervention. It had a number of different components but, briefly, it involved participants weighing themselves regularly on a set of ‘smart scales’, which enabled the trial team to access and monitor the data. Participants received text messages from the trial team with feedback, reminders to weigh themselves (if necessary), and links to online tools and content to support the maintenance of their weight loss.

The DCE results suggested that preferences for the various components of the intervention varied significantly between individuals and between the different groups – and not all were important. In contrast, the efficacy and cost attributes were important across the board. The CV results suggested that a very significant proportion of individuals would be willing to pay for an effective intervention (i.e. that avoided weight regain), with very few respondents expressing a willingness to pay for an intervention that led to more than 10-20% weight regain.

Do alternative methods for preference elicitation provide a consistent picture of variation in preferences?

Existing evidence suggests that willingness to pay (WTP) estimates from CV tasks might differ from those derived from DCE data, but there aren’t a lot of empirical studies on this in health. Comparisons were planned in my study, but the approach taken in the end was suboptimal and ultimately inconclusive. The original plan was to obtain WTP estimates for an entire WLM intervention using the DCE and to compare this with the estimates from the CV task. Due to data limitations, it wasn’t possible to make this comparison. However, the CV task was a bit unusual because we asked for respondents’ WTP at various different efficacy levels. So instead the comparison made was between average WTP values for a percentage point of weight re-gain. The differences were statistically insignificant.

Are some people’s preferences ‘better defined’ than others’?

We hypothesised that those with experience of the trial intervention would have ‘better defined’ preferences. To explore this, we compared the data quality across the different user groups. From a quick glance at the DCE results, it is pretty clear that the data were much better for the most experienced group; the coefficients were larger, and a much higher proportion was statistically significant. However, more interestingly, we found that the most experienced group were 23% more likely to have passed all of the rationality tests that were embedded in the DCE. Therefore, if you accept that better quality data is an indicator of ‘better defined’ preferences, then the data do seem reasonably supportive of the hypothesis. That being said, there were no significant differences between the other three groups, begging the question: was it the difference in experience, or some other difference between RCT participants and online panel respondents?

What does your research imply for the use of preferences in resource allocation decisions?

While there are still many unanswered questions, and there is always a need for further research, the results from my PhD project suggest that preferences for health care interventions can differ significantly between respondents with differing levels of experience. Had my project been applied to a more clinical intervention that is harder for an average person to imagine experiencing, I would expect the differences to have been much larger. I’d love to see more research in this area in future, especially in the context of benefit-risk trade-offs.

The key message is that the level of experience of the participants matters. It is quite reasonable to believe that a preference study focusing on a particular subgroup of patients will not be generalisable to the broader patient population. As preference data, typically elicited from patients, is increasingly being used in decision-making – which is great – it is becoming increasingly important for researchers to make sure that their respondent samples are appropriate to support the decisions that are being made.