Chris Sampson’s journal round-up for 23rd December 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The Internet and children’s psychological wellbeing. Journal of Health Economics Published 13th December 2019

Here at the blog, we like the Internet. We couldn’t exist without it. We vie for your attention along with all of the other content factories (or “friends”). But there’s a well-established sense that people – especially children – should moderate their consumption of Internet content. The Internet is pervasive and is now a fundamental part of our day-to-day lives, not simply an information source to which we turn when we need it. Almost all 12-15 year olds in the UK use the Internet. The ubiquity of the Internet makes it difficult to test its effects. But this paper has a good go at it.

This study is based on the idea that broadband speeds are a good proxy for Internet use. In England, a variety of public and private sector initiatives have resulted in a distorted market with quasi-random assigment of broadband speeds. The authors provide a very thorough explanation of children’s wellbeing in relation to the Internet, outlining a range of potential mechanisms.

The analysis combines data from the UK’s pre-eminent household panel survey (Understanding Society) with broadband speed data published by the UK regulator Ofcom. Six wellbeing outcomes are analysed from children’s self-reported responses. The questions ask children how they feel about their lives – measured on a seven-point scale – in relation to school work, appearance, family, friends, school attended, and life as a whole. An unbalanced panel of 6,310 children from 2012-2017 provides 13,938 observations from 3,765 different Lower Layer Super Output Areas (LSOA), with average broadband speeds for each LSOA for each year. Each of the six wellbeing outcomes is modelled with child-, neighbourhood- and time-specific fixed effects. The models’ covariates include a variety of indicators relating to the child, their parents, their household, and their local area.

A variety of models are tested, and the overall finding is that higher broadband speeds are negatively associated with all of the six wellbeing indicators. Wellbeing in relation to appearance shows the strongest effect; a 1% increase in broadband speed reduces happiness with appearance by around 0.6%. The authors explore a variety of potential mechanisms by running pairs of models between broadband speeds and the mechanism and between the mechanism and the outcomes. A key finding is that the data seem to support the ‘crowding out’ hypothesis. Higher broadband speeds are associated with children spending less time on activities such as sports, clubs, and real world social interactions, and these activities are in turn positively associated with wellbeing. The authors also consider different subgroups, finding that the effects are more detrimental for girls.

Where the paper falls down is that it doesn’t do anything to convince us that broadband speeds represent a good proxy for Internet use. It’s also not clear exactly what the proxy is meant to be for – use (e.g. time spent on the Internet) or access (i.e. having the option to use the Internet) – though the authors seem to be interested in the former. If that’s the case, the logic of the proxy is not obvious. If I want to do X on the Internet then higher speeds will enable me to do it in less time, in which case the proxy would capture the inverse of the desired indicator. The other problem I think we have is in the use of self-reported measures in this context. A key supposed mechanism for the effect is through ‘social comparison theory’, which we might reasonably expect to influence the way children respond to questions as well as – or instead of – their underlying wellbeing.

One-way sensitivity analysis for probabilistic cost-effectiveness analysis: conditional expected incremental net benefit. PharmacoEconomics [PubMed] Published 16th December 2019

Here we have one of those very citable papers that clearly specifies a part of cost-effectiveness analysis methodology. A better title for this paper could be Make one-way sensitivity analysis great again. The authors start out by – quite rightly – bashing the tornado diagram, mostly on the basis that it does not intuitively characterise the information that a decision-maker needs. Instead, the authors propose an approach to probabilistic one-way sensitivity analysis (POSA) that is a kind of simplified version of EVPPI (expected value of partially perfect information) analysis. Crucially, this approach does not assume that the various parameters of the analysis are independent.

The key quantity created by this analysis is the conditional expected incremental net monetary benefit (cINMB), conditional, that is, on the value of the parameter of interest. There are three steps to creating a plot of the POSA results: 1) rank the costs and outcomes for the sampled values of the parameter – say from the first to the last centile; 2) plug in a cost-effectiveness threshold value to calculate the cINMB at each sampled value; and 3) record the probability of observing each value of the parameter. You could use this information to present a tornado-style diagram, plotting the credible range of the cINMB. But it’s more useful to plot a line graph showing the cINMB at the different values of the parameter of interest, taking into account the probability that the values will actually be observed.

The authors illustrate their method using three different parameters from a previously published cost-effectiveness analysis, in each case simulating 15,000 Monte Carlo ‘inner loops’ for each of the 99 centiles. It took me a little while to get my head around the results that are presented, so there’s still some work to do around explaining the visuals to decision-makers. Nevertheless, this approach has the potential to become standard practice.

A head-on ordinal comparison of the composite time trade-off and the better-than-dead method. Value in Health Published 19th December 2019

For years now, methodologists have been trying to find a reliable way to value health states ‘worse than dead’. The EQ-VT protocol, used to value the EQ-5D-5L, includes the composite time trade-off (cTTO). The cTTO task gives people the opportunity to trade away life years in good health to avoid having to subsequently live in a state that they have identified as being ‘worse than dead’ (i.e. they would prefer to die immediately than to live in it). An alternative approach to this is the better-than-dead method, whereby people simply compare given durations in a health state to being dead. But are these two approaches measuring the same thing? This study sought to find out.

The authors recruited a convenience sample of 200 students and asked them to value seven different EQ-5D-5L health states that were close to zero in the Dutch tariff. Each respondent completed both a cTTO task and a better-than-dead task (the order varied) for each of the seven states. The analysis then looked at the extent to which there was agreement between the two methods in terms of whether states were identified as being better or worse than dead. Agreement was measured using counts and using polychoric correlations. Unsurprisingly, agreement was higher for those states that lay further from zero in the Dutch tariff. Around zero, there was quite a bit of disagreement – only 65% agreed for state 44343. Both approaches performed similarly with respect to consistency and test-retest reliability. Overall, the authors interpret these findings as meaning that the two methods are measuring the same underlying preferences.

I don’t find that very convincing. States were more often identified as worse than dead in the better-than-dead task, with 55% valued as such, compared with 37% in the cTTO. That seems like a big difference. The authors provide a variety of possible explanations for the differences, mostly relating to the way the tasks are framed. Or it might be that the complexity of the worse-than-dead task in the cTTO is so confusing and counterintuitive that respondents (intentionally or otherwise) avoid having to do it. For me, the findings reinforce the futility of trying to value health states in relation to being dead. If a slight change in methodology prevents a group of biomedical students from giving consistent assessments of whether or not a state is worse than being dead, what hope do we have?

Credits

Chris Sampson’s journal round-up for 7th January 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Overview, update, and lessons learned from the international EQ-5D-5L valuation work: version 2 of the EQ-5D-5L valuation protocol. Value in Health Published 2nd January 2019

Insofar as there is any drama in health economics, the fallout from the EQ-5D-5L value set for England was pretty dramatic. If you ask me, the criticisms are entirely ill-conceived. Regardless of that, one of the main sticking points was that the version of the EQ-5D-5L valuation protocol that was used was flawed. England was one of the first countries to get a valuation, so it used version 1.0 of the EuroQol Valuation Technique (EQ-VT). We’re now up to version 2.1. This article outlines the issues that arose in using the first version, what EuroQol did to try and solve them, and describes the current challenges in valuation.

EQ-VT 1.0 includes the composite time trade-off (cTTO) task to elicit values for health states better and worse than dead. Early valuation studies showed some unusual patterns. Research into the causes of this showed that in many cases there was very little time spent on the task. Some interviewers had a tendency to skip parts of the explanation for completing the worse-than-dead bit of the cTTO, resulting in no values worse than dead. EQ-VT 1.1 added three practise valuations along with greater monitoring of interviewer performance and a quality control procedure. This dramatically reduced interviewer effects and the likelihood of inconsistent responses. Yet further improvements could be envisioned. And so EQ-VT 2.0 added a feedback module. The feedback module shows respondents the ranking of states implied by their valuations, with which respondents can then agree or disagree. 2.0 was tested against 1.1 and showed further reductions in inconsistencies thanks to the feedback module. Other modifications were not supported by the evaluation. EQ-VT 2.1 added a dynamic question to further improve the warm-up tasks.

There are ongoing challenges with the cTTO, mostly to do with how to model the data. The authors provide a table setting out causes, consequences, and possible solutions for various issues that might arise in the modelling of cTTO data. And then there’s the discrete choice experiment (DCE), which is included in addition to the cTTO, but which different valuation studies used (or did not use) differently in modelling values. Research is ongoing that will probably lead to developments beyond EQ-VT 2.1. This might involve abandoning the cTTO altogether. Or, at least, there might be a reduction in cTTO tasks and a greater reliance on DCE. But more research is needed before duration can be adequately incorporated into DCEs.

Helpfully, the paper includes a table with a list of countries and specification of the EQ-VT versions used. This demonstrates the vast amount of knowledge that has been accrued about EQ-5D-5L valuation and the lack of wisdom in continuing to support the (relatively under-interrogated) EQ-5D-3L MVH valuation.

Do time trade-off values fully capture attitudes that are relevant to health-related choices? The European Journal of Health Economics [PubMed] Published 31st December 2018

Different people have different preferences, so values for health states elicited using TTO should vary from person to person. This study is concerned with how personal circumstances and beliefs influence TTO values and whether TTO entirely captures the impact of these on preferences for health states.

The authors analysed data from an online survey with a UK-representative sample of 1,339. Participants were asked about their attitudes towards quality and quantity of life, before completing some TTO tasks based on the EQ-5D-5L. Based on their response, they were shown two ‘lives’ that – given their TTO response – they should have considered to be of equivalent value. The researchers constructed generalised estimating equations to model the TTO values and logit models for the subsequent choices between states. Age, marital status, education, and attitudes towards trading quality and quantity of life all determined TTO values in addition to the state that was being valued. In the modelling of the decisions about the two lives, attitudes influenced decisions through the difference between the two lives in the number of life years available. That is, an interaction term between the attitudes variable and years variables showed that people who prefer quantity of life over quality of life were more likely to choose the state with a greater number of years.

The authors’ interpretation from this is that TTO reflects people’s attitudes towards quality and quantity of life, but only partially. My interpretation would be that the TTO exercise would have benefitted from the kind of refinement described above. The choice between the two lives is similar to the feedback module of the EQ-VT 2.0. People often do not understand the implications of their TTO valuations. The study could also be interpreted as supportive of ‘head-to-head’ choice methods (such as DCE) rather than making choices involving full health and death. But the design of the TTO task used in this study was quite dissimilar to others, which makes it difficult to say anything generally about TTO as a valuation method.

Exploring the item sets of the Recovering Quality of Life (ReQoL) measures using factor analysis. Quality of Life Research [PubMed] Published 21st December 2018

The ReQoL is a patient-reported outcome measure for use with people experiencing mental health difficulties. The ReQoL-10 and ReQoL-20 both ask questions relating to seven domains: six mental, one physical. There’s been a steady stream of ReQoL research published in recent years and the measures have been shown to have acceptable psychometric properties. This study concerns the factorial structure of the ReQoL item sets, testing internal construct validity and informing scoring procedures. There’s also a more general methodological contribution relating to the use of positive and negative factors in mental health outcome questionnaires.

At the outset of this study, the ReQoL was based on 61 items. These were reduced to 40 on the basis of qualitative and quantitative analysis reported in other papers. This paper reports on two studies – the first group (n=2,262) completed the 61 items and the second group (n=4,266) completed 40 items. Confirmatory factor analysis and exploratory factor analysis were conducted. Six-factor (according to ReQoL domains), two-factor (negative/positive) and bi-factor (global/negative/positive) models were tested. In the second study, participants were either presented with a version that jumbled up the positively and negatively worded questions or a version that showed a block of negatives followed by a block of positives. The idea here is that if a two-factor structure is simply a product of the presentation of questions, it should be more pronounced in the jumbled version.

The results were much the same from the two study samples. The bi-factor model demonstrated acceptable fit, with much higher factor loadings on the general quality of life factor that loaded on all items. The results indicated sufficient unidimensionality to go ahead with reducing the number of items and the two ordering formats didn’t differ, suggesting that the negative and positive loadings weren’t just an artefact of the presentation. The findings show that the six dimensions of the ReQoL don’t stand as separate factors. The justification for maintaining items from each of the six dimensions, therefore, seems to be a qualitative one.

Some outcome measurement developers have argued that items should all be phrased in the same direction – as either positive or negative – to obtain high-quality data. But there’s good reason to think that features of mental health can’t reliably be translated from negative to positive, and this study supports the inclusion (and intermingling) of both within a measure.

Credits

Thesis Thursday: Ernest Law

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Ernest Law who has a PhD from the University of Illinois at Chicago. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Examining sources of variation in developing a societal health state value set
Supervisors
Simon Pickard, Todd Lee, Surrey Walton, Alan Schwartz, Feng Xie
Repository link
http://hdl.handle.net/10027/23037

How did you come to study EQ-5D valuation methods, and why are they important?

I came across health preferences research after beginning my studies at UIC with my thesis supervisor, Prof. Simon Pickard. Before this, I was a clinical pharmacist who spent a lot of time helping patients and their families navigate the trade-offs between the benefits and harms of pharmacotherapy. So, when I was introduced to a set of methods that seeks to quantify such trade-offs, I was quickly captivated and set on a path to understanding more. I continued on to expand my interests in valuation methods pertinent to health-system decision-making. Naturally, I collided with societal health state value sets – important tools developed from generic preference-based measures, such as the EQ-5D.

During my studies at UIC, our group received a grant (PI: Simon Pickard) from the EuroQol Research Foundation to develop the United States EQ-5D-5L value set. While developing the study protocol, we built in additional data elements (e.g., EQ-5D-3L valuation tasks, advance directive status) that would help answer important questions in explaining variation in value sets. By understanding these sources of variation, we could inform researchers and policymakers alike on the development and application of EQ-5D value sets.

What does your thesis add to the debate about EQ-5D-3L and -5L value sets?

As a self-reported measure, the literature appears reasonably clear regarding the 5L’s advantages over the 3L: reduced ceiling effects, more unique self-reported health states, and improved discriminatory power. However, less was known on how differences in descriptive systems impact direct valuations.

Previous comparisons focused on differences in index scores and QALYs generated from existing value sets. But these value sets differed in substantive ways: preferences from different respondents, in different time periods, from different geographic locations, using different study protocols. This makes it difficult to isolate the differences due to the descriptive system.

In our study, we asked respondents in the US EQ-5D valuation study to complete time trade-off tasks for 3L and 5L health states. By doing so, we were able to hold many of the aforementioned factors constant except the valued health state. From a research perspective, we provide strong evidence on how even small changes in the descriptive system can have a profound impact on the valuations. From a policy perspective, and an HTA agency deciding specifically between the 3L and 5L, we’ve provided critical insight into the kind of value set one might expect to obtain using either descriptive system.

Why are health state valuations by people with advance directives particularly interesting?

The interminable debate over “whose preferences” should be captured when obtaining preferences for the purposes of generating QALYs is well-known among health outcomes researchers and policy-makers. Two camps typically emerge, those that argue for capturing preferences from the general population and those that argue for patients to be the primary source. The supporting arguments for both sides have been well-documented. One additional approach has recently emerged which may reconcile some of the differences by using informed preferences. Guidance from influential groups in the US, such as the First and Second Panels of Cost-Effectiveness in Health and Medicine have also maintained that “the best articulation of a society’s preferences… would be gathered from a representative sample of fully informed members”.

We posited that individuals with advance directives may represent a group that had reflected substantially on their current health state, as well as the experience and consequences of a range of (future) health states. Individuals who complete an advance directive undergo a process that includes discussion and documentation of an individual’s preferences concerning their goals of care in the event they are unable to do so themselves. So we set out to examine this relationship between advance directives and stated preferences, and whether the completion of an advance directive was associated with differences in health state preferences (spoiler: it was).

Is there evidence that value sets should be updated over time?

We sought to address this literature gap by using respondent-level data from the US EQ-5D-3L study that collected TTO values in 2002 and from our EQ-5D-5L study, which also collected 3L TTO values in 2017. However there were inherent challenges with using these data collected so many years apart: demographics shift, new methods and modes of administration are implemented, etc.

So, we attempted to account for what was possible by controlling for respondent characteristics and restricting health state values to those obtained using the same preference elicitation technique (i.e., conventional TTO). We found that values in 2017 were modestly higher, implying that the average adult in the US in 2017 was less willing to trade time for quality of life than in 2002, i.e. 6 months over a 10-year time-horizon. Our research suggests that time-specific differences in societal preferences exist and that the time period in which values were elicited may be an important factor to consider when selecting or applying a value set.

Based on your research, do you have any recommendations for future valuation studies?

I would encourage researchers conducting future valuation studies, particularly societal value sets, to consider some of the following:

1) Consider building in small but powerful methodological sub-aims into your study. Of course, you must balance resource constraints, data quality, and respondent burden against such add-ons, but a balance can be struck!

2) Pay attention to important developments in the population being sampled; for example, we incorporated advance directives because it is becoming an important topic in the US healthcare debate, in addition to contributing to the discussion surrounding informed preferences.

3) Take a close look at the most commonly utilized health state values sets representing your health-system/target population. Is it possible that existing value sets are “outdated”? If so, a proposal to update this value set might fill a very important need. While you’re at it, consider an analysis to compare current and previous values. The evidence is scarce (and difficult to study!) so it’s important to continue building evidence that can inform the broader scientific and HTA community as to the role that time plays in changes to societal preferences.