Chris Sampson’s journal round-up for 7th January 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Overview, update, and lessons learned from the international EQ-5D-5L valuation work: version 2 of the EQ-5D-5L valuation protocol. Value in Health Published 2nd January 2019

Insofar as there is any drama in health economics, the fallout from the EQ-5D-5L value set for England was pretty dramatic. If you ask me, the criticisms are entirely ill-conceived. Regardless of that, one of the main sticking points was that the version of the EQ-5D-5L valuation protocol that was used was flawed. England was one of the first countries to get a valuation, so it used version 1.0 of the EuroQol Valuation Technique (EQ-VT). We’re now up to version 2.1. This article outlines the issues that arose in using the first version, what EuroQol did to try and solve them, and describes the current challenges in valuation.

EQ-VT 1.0 includes the composite time trade-off (cTTO) task to elicit values for health states better and worse than dead. Early valuation studies showed some unusual patterns. Research into the causes of this showed that in many cases there was very little time spent on the task. Some interviewers had a tendency to skip parts of the explanation for completing the worse-than-dead bit of the cTTO, resulting in no values worse than dead. EQ-VT 1.1 added three practise valuations along with greater monitoring of interviewer performance and a quality control procedure. This dramatically reduced interviewer effects and the likelihood of inconsistent responses. Yet further improvements could be envisioned. And so EQ-VT 2.0 added a feedback module. The feedback module shows respondents the ranking of states implied by their valuations, with which respondents can then agree or disagree. 2.0 was tested against 1.1 and showed further reductions in inconsistencies thanks to the feedback module. Other modifications were not supported by the evaluation. EQ-VT 2.1 added a dynamic question to further improve the warm-up tasks.

There are ongoing challenges with the cTTO, mostly to do with how to model the data. The authors provide a table setting out causes, consequences, and possible solutions for various issues that might arise in the modelling of cTTO data. And then there’s the discrete choice experiment (DCE), which is included in addition to the cTTO, but which different valuation studies used (or did not use) differently in modelling values. Research is ongoing that will probably lead to developments beyond EQ-VT 2.1. This might involve abandoning the cTTO altogether. Or, at least, there might be a reduction in cTTO tasks and a greater reliance on DCE. But more research is needed before duration can be adequately incorporated into DCEs.

Helpfully, the paper includes a table with a list of countries and specification of the EQ-VT versions used. This demonstrates the vast amount of knowledge that has been accrued about EQ-5D-5L valuation and the lack of wisdom in continuing to support the (relatively under-interrogated) EQ-5D-3L MVH valuation.

Do time trade-off values fully capture attitudes that are relevant to health-related choices? The European Journal of Health Economics [PubMed] Published 31st December 2018

Different people have different preferences, so values for health states elicited using TTO should vary from person to person. This study is concerned with how personal circumstances and beliefs influence TTO values and whether TTO entirely captures the impact of these on preferences for health states.

The authors analysed data from an online survey with a UK-representative sample of 1,339. Participants were asked about their attitudes towards quality and quantity of life, before completing some TTO tasks based on the EQ-5D-5L. Based on their response, they were shown two ‘lives’ that – given their TTO response – they should have considered to be of equivalent value. The researchers constructed generalised estimating equations to model the TTO values and logit models for the subsequent choices between states. Age, marital status, education, and attitudes towards trading quality and quantity of life all determined TTO values in addition to the state that was being valued. In the modelling of the decisions about the two lives, attitudes influenced decisions through the difference between the two lives in the number of life years available. That is, an interaction term between the attitudes variable and years variables showed that people who prefer quantity of life over quality of life were more likely to choose the state with a greater number of years.

The authors’ interpretation from this is that TTO reflects people’s attitudes towards quality and quantity of life, but only partially. My interpretation would be that the TTO exercise would have benefitted from the kind of refinement described above. The choice between the two lives is similar to the feedback module of the EQ-VT 2.0. People often do not understand the implications of their TTO valuations. The study could also be interpreted as supportive of ‘head-to-head’ choice methods (such as DCE) rather than making choices involving full health and death. But the design of the TTO task used in this study was quite dissimilar to others, which makes it difficult to say anything generally about TTO as a valuation method.

Exploring the item sets of the Recovering Quality of Life (ReQoL) measures using factor analysis. Quality of Life Research [PubMed] Published 21st December 2018

The ReQoL is a patient-reported outcome measure for use with people experiencing mental health difficulties. The ReQoL-10 and ReQoL-20 both ask questions relating to seven domains: six mental, one physical. There’s been a steady stream of ReQoL research published in recent years and the measures have been shown to have acceptable psychometric properties. This study concerns the factorial structure of the ReQoL item sets, testing internal construct validity and informing scoring procedures. There’s also a more general methodological contribution relating to the use of positive and negative factors in mental health outcome questionnaires.

At the outset of this study, the ReQoL was based on 61 items. These were reduced to 40 on the basis of qualitative and quantitative analysis reported in other papers. This paper reports on two studies – the first group (n=2,262) completed the 61 items and the second group (n=4,266) completed 40 items. Confirmatory factor analysis and exploratory factor analysis were conducted. Six-factor (according to ReQoL domains), two-factor (negative/positive) and bi-factor (global/negative/positive) models were tested. In the second study, participants were either presented with a version that jumbled up the positively and negatively worded questions or a version that showed a block of negatives followed by a block of positives. The idea here is that if a two-factor structure is simply a product of the presentation of questions, it should be more pronounced in the jumbled version.

The results were much the same from the two study samples. The bi-factor model demonstrated acceptable fit, with much higher factor loadings on the general quality of life factor that loaded on all items. The results indicated sufficient unidimensionality to go ahead with reducing the number of items and the two ordering formats didn’t differ, suggesting that the negative and positive loadings weren’t just an artefact of the presentation. The findings show that the six dimensions of the ReQoL don’t stand as separate factors. The justification for maintaining items from each of the six dimensions, therefore, seems to be a qualitative one.

Some outcome measurement developers have argued that items should all be phrased in the same direction – as either positive or negative – to obtain high-quality data. But there’s good reason to think that features of mental health can’t reliably be translated from negative to positive, and this study supports the inclusion (and intermingling) of both within a measure.

Credits

Chris Sampson’s journal round-up for 31st December 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Perspectives of patients with cancer on the quality-adjusted life year as a measure of value in healthcare. Value in Health Published 29th December 2018

Patients should have the opportunity to understand how decisions are made about which treatments they are and are not allowed to use, given their coverage. This study reports on a survey of cancer patients and survivors, with the aim of identifying patients’ awareness, understanding, and opinions about the QALY as a measure of value.

Participants were recruited from a (presumably US-based) patient advocacy group and 774 mostly well-educated, mostly white, mostly women responded. The online survey asked about cancer status and included a couple of measures of health literacy. Fewer than 7% of participants had ever heard of the QALY – more likely for those with greater health literacy. The survey explained the QALY to the participants and then asked if the concept of the QALY makes sense. Around half said it did and 24% thought that it was a good way to measure value in health care. The researchers report a variety of ‘significant’ differences in tendencies to understand or support the use of QALYs, but I’m not convinced that they’re meaningful because the differences aren’t big and the samples are relatively small.

At the end of the survey, respondents were asked to provide opinions on QALYs and value in health care. 165 people provided responses and these were coded and analysed qualitatively. The researchers identified three themes from this one free-text question: i) measuring value, ii) opinions on QALY, and iii) value in health care and decision making. I’m not sure that they’re meaningful themes that help us to understand patients’ views on QALYs. A significant proportion of respondents rejected the idea of using numbers to quantify value in health care. On the other hand, some suggested that the QALY could be a useful decision aid for patients. There was opposition to ‘external decision makers’ having any involvement in health care decision making. Unless you’re paying for all of your care out of pocket, that’s tough luck. But the most obvious finding from the qualitative analysis is that respondents didn’t understand what QALYs were for. That’s partly because health economists in general need to be better at communicating concepts like the QALY. But I think it’s also in large part because the authors failed to provide a clear explanation. They didn’t even use my lovely Wikipedia graphic. Many of the points made by respondents are entirely irrelevant to the appropriateness of QALYs as they’re used (or in the case of the US, aren’t yet used) in practice. For example, several discussed the use of QALYs in clinical decision making. Patients think that they should maintain autonomy, which is fair enough but has nothing to do with how QALYs are used to assess health technologies.

QALYs are built on the idea of trade-offs. They measure the trade-off between life extension and life improvement. They are used to guide trade-offs between different treatments for different people. But the researchers didn’t explain how or why QALYs are used to make trade-offs, so the elicited views aren’t well-informed.

Measuring multivariate risk preferences in the health domain. Journal of Health Economics Published 27th December 2018

Health preferences research is now a substantial field in itself. But there’s still a lot of work left to be done on understanding risk preferences with respect to health. Gradually, we’re coming round to the idea that people tend to be risk-averse. But risk preferences aren’t (necessarily) so simple. Recent research has proposed that ‘higher order’ preferences such as prudence and temperance play a role. A person exhibiting univariate prudence for longevity would be better able to cope with risk if they are going to live longer. Univariate temperance is characterised by a preference for prospects that disaggregate risk across different possible outcomes. Risk preferences can also be multivariate – across health and wealth, for example – determining the relationship between univariate risk preferences and other attributes. These include correlation aversion, cross-prudence, and cross-temperance. Many articles from the Arthur Attema camp demand a great deal of background knowledge. This paper isn’t an exception, but it does provide a very clear and intuitive description of the various kinds of uni- and multivariate risk preferences that the researchers are considering.

For this study, an experiment was conducted with 98 people, who were asked to make 69 choices, corresponding to 3 choices about each risk preference trait being tested, for both gains and losses. Participants were told that they had €240,000 in wealth and 40 years of life to play with. The number of times that an individual made choices in line with a particular trait was used as an indicator of their strength of preference.

For gains, risk aversion was common for both wealth and longevity, and prudence was a common trait. There was no clear tendency towards temperance. For losses, risk aversion and prudence tended to neutrality. For multivariate risk preferences, a majority of people were correlation averse for gains and correlation seeking for losses. For gains, 76% of choices were compatible with correlation aversion, suggesting that people prefer to disaggregate fixed wealth and health gains. For losses, the opposite was true in 68% of choices. There was evidence for cross-prudence in wealth gains but not longevity gains, suggesting that people prefer health risk if they have higher wealth. For losses, the researchers observed cross-prudence and cross-temperance neutrality. The authors go on to explore associations between different traits.

A key contribution is in understanding how risk preferences differ in the health domain as compared with the monetary domain (which is what most economists study). Conveniently, there are a lot of similarities between risk preferences in the two domains, suggesting that health economists can learn from the wider economics literature. Risk aversion and prudence seem to apply to longevity as well as monetary gains, with a shift to neutrality in losses. The potential implications of these findings are far-reaching, but this is just a small experimental study. More research needed (and anticipated).

Prospective payment systems and discretionary coding—evidence from English mental health providers. Health Economics [PubMed] Published 27th December 2018

If you’ve conducted an economic evaluation in the context of mental health care in England, you’ll have come across mental health care clusters. Patients undergoing mental health care are allocated to one of 20 clusters, classed as either ‘psychotic’, ‘non-psychotic’, or ‘organic’, which forms the basis of an episodic payment model. In 2013/14, these episodes were associated with an average cost of between £975 and £9,354 per day. Doctors determine the clusters and the clusters determine reimbursement. Perverse incentives abound. Or do they?

This study builds on the fact that patients are allocated by clinical teams with guidance from the algorithm-based Mental Health Clustering Tool (MHCT). Clinical teams might exhibit upcoding, whereby patients are allocated to clusters that attract a higher price than that recommended by the MHCT. Data were analysed for 148,471 patients from the Mental Health Services Data Set for 2011-2015. For each patient, their allocated cluster is known, along with a variety of socioeconomic indicators and the HoNoS and SARN instruments, which go into the MHCT algorithm. Mixed-effects logistic regression was used to look at whether individual patients were or were not allocated to the cluster recommended as ‘best fit’ by the MHCT, controlling for patient and provider characteristics. Further to this, multilevel multinomial logit models were used to categorise decisions that don’t match the MHCT as either under- or overcoding.

Average agreement across clusters between the MHCT and clinicians was 36%. In most cases, patients were allocated to a cluster either one step higher or one step lower in terms of the level of need, and there isn’t an obvious tendency to overcode. The authors are able to identify a few ways in which observable provider and patient characteristics influence the tendency to under- or over-cluster patients. For example, providers with higher activity are less likely to deviate from the MHCT best fit recommendation. However, the dominant finding – identified by using median odds ratios for the probability of a mismatch between two random providers – seems to be that unobserved heterogeneity determines variation in behaviour.

The study provides clues about the ways in which providers could manipulate coding to their advantage and identifies the need for further data collection for a proper assessment. But reimbursement wasn’t linked to clustering during the time period of the study, so it remains to be seen how clinicians actually respond to these potentially perverse incentives.

Credits

Thesis Thursday: Ernest Law

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Ernest Law who has a PhD from the University of Illinois at Chicago. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Examining sources of variation in developing a societal health state value set
Supervisors
Simon Pickard, Todd Lee, Surrey Walton, Alan Schwartz, Feng Xie
Repository link
http://hdl.handle.net/10027/23037

How did you come to study EQ-5D valuation methods, and why are they important?

I came across health preferences research after beginning my studies at UIC with my thesis supervisor, Prof. Simon Pickard. Before this, I was a clinical pharmacist who spent a lot of time helping patients and their families navigate the trade-offs between the benefits and harms of pharmacotherapy. So, when I was introduced to a set of methods that seeks to quantify such trade-offs, I was quickly captivated and set on a path to understanding more. I continued on to expand my interests in valuation methods pertinent to health-system decision-making. Naturally, I collided with societal health state value sets – important tools developed from generic preference-based measures, such as the EQ-5D.

During my studies at UIC, our group received a grant (PI: Simon Pickard) from the EuroQol Research Foundation to develop the United States EQ-5D-5L value set. While developing the study protocol, we built in additional data elements (e.g., EQ-5D-3L valuation tasks, advance directive status) that would help answer important questions in explaining variation in value sets. By understanding these sources of variation, we could inform researchers and policymakers alike on the development and application of EQ-5D value sets.

What does your thesis add to the debate about EQ-5D-3L and -5L value sets?

As a self-reported measure, the literature appears reasonably clear regarding the 5L’s advantages over the 3L: reduced ceiling effects, more unique self-reported health states, and improved discriminatory power. However, less was known on how differences in descriptive systems impact direct valuations.

Previous comparisons focused on differences in index scores and QALYs generated from existing value sets. But these value sets differed in substantive ways: preferences from different respondents, in different time periods, from different geographic locations, using different study protocols. This makes it difficult to isolate the differences due to the descriptive system.

In our study, we asked respondents in the US EQ-5D valuation study to complete time trade-off tasks for 3L and 5L health states. By doing so, we were able to hold many of the aforementioned factors constant except the valued health state. From a research perspective, we provide strong evidence on how even small changes in the descriptive system can have a profound impact on the valuations. From a policy perspective, and an HTA agency deciding specifically between the 3L and 5L, we’ve provided critical insight into the kind of value set one might expect to obtain using either descriptive system.

Why are health state valuations by people with advance directives particularly interesting?

The interminable debate over “whose preferences” should be captured when obtaining preferences for the purposes of generating QALYs is well-known among health outcomes researchers and policy-makers. Two camps typically emerge, those that argue for capturing preferences from the general population and those that argue for patients to be the primary source. The supporting arguments for both sides have been well-documented. One additional approach has recently emerged which may reconcile some of the differences by using informed preferences. Guidance from influential groups in the US, such as the First and Second Panels of Cost-Effectiveness in Health and Medicine have also maintained that “the best articulation of a society’s preferences… would be gathered from a representative sample of fully informed members”.

We posited that individuals with advance directives may represent a group that had reflected substantially on their current health state, as well as the experience and consequences of a range of (future) health states. Individuals who complete an advance directive undergo a process that includes discussion and documentation of an individual’s preferences concerning their goals of care in the event they are unable to do so themselves. So we set out to examine this relationship between advance directives and stated preferences, and whether the completion of an advance directive was associated with differences in health state preferences (spoiler: it was).

Is there evidence that value sets should be updated over time?

We sought to address this literature gap by using respondent-level data from the US EQ-5D-3L study that collected TTO values in 2002 and from our EQ-5D-5L study, which also collected 3L TTO values in 2017. However there were inherent challenges with using these data collected so many years apart: demographics shift, new methods and modes of administration are implemented, etc.

So, we attempted to account for what was possible by controlling for respondent characteristics and restricting health state values to those obtained using the same preference elicitation technique (i.e., conventional TTO). We found that values in 2017 were modestly higher, implying that the average adult in the US in 2017 was less willing to trade time for quality of life than in 2002, i.e. 6 months over a 10-year time-horizon. Our research suggests that time-specific differences in societal preferences exist and that the time period in which values were elicited may be an important factor to consider when selecting or applying a value set.

Based on your research, do you have any recommendations for future valuation studies?

I would encourage researchers conducting future valuation studies, particularly societal value sets, to consider some of the following:

1) Consider building in small but powerful methodological sub-aims into your study. Of course, you must balance resource constraints, data quality, and respondent burden against such add-ons, but a balance can be struck!

2) Pay attention to important developments in the population being sampled; for example, we incorporated advance directives because it is becoming an important topic in the US healthcare debate, in addition to contributing to the discussion surrounding informed preferences.

3) Take a close look at the most commonly utilized health state values sets representing your health-system/target population. Is it possible that existing value sets are “outdated”? If so, a proposal to update this value set might fill a very important need. While you’re at it, consider an analysis to compare current and previous values. The evidence is scarce (and difficult to study!) so it’s important to continue building evidence that can inform the broader scientific and HTA community as to the role that time plays in changes to societal preferences.