Thesis Thursday: Ernest Law

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Ernest Law who has a PhD from the University of Illinois at Chicago. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Examining sources of variation in developing a societal health state value set
Simon Pickard, Todd Lee, Surrey Walton, Alan Schwartz, Feng Xie
Repository link

How did you come to study EQ-5D valuation methods, and why are they important?

I came across health preferences research after beginning my studies at UIC with my thesis supervisor, Prof. Simon Pickard. Before this, I was a clinical pharmacist who spent a lot of time helping patients and their families navigate the trade-offs between the benefits and harms of pharmacotherapy. So, when I was introduced to a set of methods that seeks to quantify such trade-offs, I was quickly captivated and set on a path to understanding more. I continued on to expand my interests in valuation methods pertinent to health-system decision-making. Naturally, I collided with societal health state value sets – important tools developed from generic preference-based measures, such as the EQ-5D.

During my studies at UIC, our group received a grant (PI: Simon Pickard) from the EuroQol Research Foundation to develop the United States EQ-5D-5L value set. While developing the study protocol, we built in additional data elements (e.g., EQ-5D-3L valuation tasks, advance directive status) that would help answer important questions in explaining variation in value sets. By understanding these sources of variation, we could inform researchers and policymakers alike on the development and application of EQ-5D value sets.

What does your thesis add to the debate about EQ-5D-3L and -5L value sets?

As a self-reported measure, the literature appears reasonably clear regarding the 5L’s advantages over the 3L: reduced ceiling effects, more unique self-reported health states, and improved discriminatory power. However, less was known on how differences in descriptive systems impact direct valuations.

Previous comparisons focused on differences in index scores and QALYs generated from existing value sets. But these value sets differed in substantive ways: preferences from different respondents, in different time periods, from different geographic locations, using different study protocols. This makes it difficult to isolate the differences due to the descriptive system.

In our study, we asked respondents in the US EQ-5D valuation study to complete time trade-off tasks for 3L and 5L health states. By doing so, we were able to hold many of the aforementioned factors constant except the valued health state. From a research perspective, we provide strong evidence on how even small changes in the descriptive system can have a profound impact on the valuations. From a policy perspective, and an HTA agency deciding specifically between the 3L and 5L, we’ve provided critical insight into the kind of value set one might expect to obtain using either descriptive system.

Why are health state valuations by people with advance directives particularly interesting?

The interminable debate over “whose preferences” should be captured when obtaining preferences for the purposes of generating QALYs is well-known among health outcomes researchers and policy-makers. Two camps typically emerge, those that argue for capturing preferences from the general population and those that argue for patients to be the primary source. The supporting arguments for both sides have been well-documented. One additional approach has recently emerged which may reconcile some of the differences by using informed preferences. Guidance from influential groups in the US, such as the First and Second Panels of Cost-Effectiveness in Health and Medicine have also maintained that “the best articulation of a society’s preferences… would be gathered from a representative sample of fully informed members”.

We posited that individuals with advance directives may represent a group that had reflected substantially on their current health state, as well as the experience and consequences of a range of (future) health states. Individuals who complete an advance directive undergo a process that includes discussion and documentation of an individual’s preferences concerning their goals of care in the event they are unable to do so themselves. So we set out to examine this relationship between advance directives and stated preferences, and whether the completion of an advance directive was associated with differences in health state preferences (spoiler: it was).

Is there evidence that value sets should be updated over time?

We sought to address this literature gap by using respondent-level data from the US EQ-5D-3L study that collected TTO values in 2002 and from our EQ-5D-5L study, which also collected 3L TTO values in 2017. However there were inherent challenges with using these data collected so many years apart: demographics shift, new methods and modes of administration are implemented, etc.

So, we attempted to account for what was possible by controlling for respondent characteristics and restricting health state values to those obtained using the same preference elicitation technique (i.e., conventional TTO). We found that values in 2017 were modestly higher, implying that the average adult in the US in 2017 was less willing to trade time for quality of life than in 2002, i.e. 6 months over a 10-year time-horizon. Our research suggests that time-specific differences in societal preferences exist and that the time period in which values were elicited may be an important factor to consider when selecting or applying a value set.

Based on your research, do you have any recommendations for future valuation studies?

I would encourage researchers conducting future valuation studies, particularly societal value sets, to consider some of the following:

1) Consider building in small but powerful methodological sub-aims into your study. Of course, you must balance resource constraints, data quality, and respondent burden against such add-ons, but a balance can be struck!

2) Pay attention to important developments in the population being sampled; for example, we incorporated advance directives because it is becoming an important topic in the US healthcare debate, in addition to contributing to the discussion surrounding informed preferences.

3) Take a close look at the most commonly utilized health state values sets representing your health-system/target population. Is it possible that existing value sets are “outdated”? If so, a proposal to update this value set might fill a very important need. While you’re at it, consider an analysis to compare current and previous values. The evidence is scarce (and difficult to study!) so it’s important to continue building evidence that can inform the broader scientific and HTA community as to the role that time plays in changes to societal preferences.

Meeting round-up: ISPOR Europe 2018 (part 2)

Have you missed ISPOR Europe 2018 but are eager to know all about it? Time to continue reading! In yesterday’s post, I wrote about ISPOR’s outstanding short-course on causal inference and the superb sessions I had attended on day 1. This blog post is about day 2, Tuesday 13th, which was another big day.

The second plenary session was on fairness in pharmaceutical pricing. It was moderated by Sarah Garner, with presentations by many key stakeholders. The thought-provoking discussion highlighted the importance of pharmaceutical pricing policy and the large role that HTA can have in shaping it.

Communicating cost-effectiveness analysis was the next session, where myself, together with Rob Hettle, Gabriel Rogers and Mike Drummond, discussed the pitfalls and approaches to explaining cost-effectiveness models to non-health economists. This was a hugely popular session! We were delighted by the incredibly positive feedback we received, which reassured us that we are clearly not alone in finding it difficult to communicate cost-effectiveness analysis to a lay audience. We certainly feel incentivised to continue working on this topic. The slides are available here, and for the audience’s feedback, search on twitter #communicateCEA.

The lunch was followed by the open meeting of ISPOR Women in HEOR Initiative with Shelby Reed, Olivia Wu and Louise Timlin. It is really encouraging to see ISPOR taking a proactive stance to gender balance!

The most popular session in the afternoon was Valuing a cure: Are new approaches needed, with Steve Pearson, Jens Grueger, Sarah Garner and Mark Sculpher. The panel showed the various perspectives on the pricing of curative therapies. Payers call for a sustainable pricing model, whilst pharma warns that pricing policy is necessarily linked to the incentives for investment in research. I agree with Mark in that these challenges are not unique to curative therapies. As pharmaceutical therapies have greater health benefits but at large costs, it is pressing that cost-effectiveness assessments are also able to consider the opportunity cost of funding more costly treatments. See here for a roundup of the estimates already available.

I then attended the excellent session on Drug disinvestment: is it needed and how could it work, moderated by Richard Macaulay. Andrew Walker explained that HTA agencies’ advice does not always go down well with local payers, highlighting this with an amusing imaginary dialogue between NICE and a hospital. Detlev Parow argued that payers find that prices are often unaffordable, hence payment schemes should consider other options, such as treatment success, risk-sharing agreements and payment by instalments. Bettina Ryll made an impressive case from the patients’ perspective, for whom these decisions have a real impact.

The conference continued late into the evening and, I suspect, long into the early hours of Wednesday, with the ever-popular conference dinner. Wednesday was another day full of fascinating sessions. The plenary was titled Budget Impact and Expenditure Caps: Potential or Pitfall, moderated by Guillem López-Casasnovas. It was followed by inspiring sessions that explored a wide range of topics, presented by the top experts in the relevant fields. These really delved into the nitty-gritty on subjects, such as using R to build decision models, the value of diagnostic information, and expert elicitation, just to name a few.

I don’t think I’m just speaking personally when I say that ISPOR Barcelona was an absolutely brilliant conference! I’ve mentioned here a few of the most outstanding sessions, but there were many, many more. There were so many sessions at the same time that it was physically impossible to attend all of those with a direct relevance to my research. But fortunately, we can access all the presentations by downloading them from the ISPOR website. I’ll leave the suggestion to ISPOR here, that they should think about filming some of the key sessions and broadcasting them as webinars after the conference. This could create a further key resource for our sector.

As in previous editions, ISPOR Barcelona truly confirms ISPOR Europe in the top HTA conferences in Europe, if not the world. It expertly combines cutting-edge methodological research with outstanding applied work, all with the view to better inform decision making. As I’m sure you can guess, I’m already looking forward to the next ISPOR Europe in Copenhagen on the 2nd-6th November 2019, and the amazing sessions which will indubitably be featured!


Meeting round-up: ISPOR Europe 2018 (part 1)

ISPOR Europe 2018, which took place in Barcelona on the 10th-14th November, was an exceptional conference. It had a jam-packed programme on the latest developments and most pressing challenges in health technology assessment (HTA), economic evaluation and outcomes research. In two blog posts, I’ll tell you about the outstanding sessions and thought-provoking discussions in this always superb conference.

For me, proceedings started on Sunday, with the excellent short-course Adjusting for Time-Dependent Confounding and Treatment Switching Bias in Observational Studies and Clinical Trials: Purpose, Methods, Good Practices and Acceptance in HTA, by Uwe Siebert, Felicitas Kühne and Nick Latimer. Felicitas Kühne explained that causal inference methods aim to estimate the effect of a treatment, risk factor etc. on our outcome of interest, controlling for other exposures that may affect it and hence bias our estimate. Uwe Siebert and Nick Latimer provided a really useful overview of the methods to overcome this challenge in observational studies and RCTs with treatment switching. This was an absolutely brilliant course. Highly recommended to any health economist!

ISPOR conferences usually start early and finish late with loads of exceptional sessions. On Monday, I started the conference proper with the plenary Joint Assessment of Relative Effectiveness: “Trick or Treat” for Decision Makers in EU Member States, moderated by Finn Børlum Kristensen. There were presentations from representatives of payers, HTA agencies, EUnetHTA, pharmaceutical industry and patients. The prevailing mood seemed to be of cautious anticipation. Avoiding duplication of efforts in the clinical assessment was greatly welcomed, but there were some concerns voiced about the practicalities of implementation. The proposal was due to be discussed soon by the European Commission, so undoubtedly we can look forward to knowing more in the near future.


My next session was the fascinating panel on the perils and opportunities of advanced computing techniques with the tongue-in-cheek title Will machines soon make health economists obsolete?, by David Thompson, Bill Marder, Gerry Oster and Mike Drummond. Don’t panic yet as, despite the promises of artificial intelligence, I’d wager that our jobs are quite safe. For example, Gerry Oster predicted that demand for health economic models is actually likely to increase, as computers make our models quicker and cheaper to build. Mike Drummond finished with the sensible suggestion to simply keep calm and carry on modelling, as computing advances will liberate our time to explore other areas, such as the interface with decision-makers. This session left us all in a very positive mood as we headed for a well-earned lunch!

There were many interesting sessions in the afternoon. I chose to pop over to the ISPOR Medical Device and Diagnostic Special Interest Group Open Meeting, the ISPOR Portugal chapter meeting, along with taking in the podium presentations on conceptual papers. Many of the presentations will be made available in the ISPOR database, which I recommend exploring. I had a wonderful experience moderating the engaging podium session on cancer models, with outstanding presentations delivered by Hedwig Blommestein, Ash Bullement, and Isle van Oostrum.

The workshop Adjusting for post-randomisation confounding and switching in phase 3 and pragmatic trials to get the estimands right: needs, methods, sub-optimal use, and acceptance in HTA by Uwe Siebert, Felicitas Kühne, Nick Latimer and Amanda Adler is one worth highlighting. The panellists showed that some HTAs do not include any adjustments for treatment switching, whilst adjustments can sometimes be incorrectly applied. It reinforced the idea that we need to learn more about these methods, to be able to apply them in practice and critically appraise them.

The afternoon finished with the second session of the day on posters. Alessandro Grosso, Laura Bojke and I had a poster on the impact of structural uncertainty in the expected value of perfect information. Alessandro did an amazing job encapsulating the poster and presenting it live to camera, which you can watch here.


In tomorrow’s blog post, I’ll tell you about day 2 of ISPOR Europe 2018 in Barcelona. Tuesday was another big day, with loads of outstanding sessions on the key topics in HTA. It featured my very own workshop, with Rob Hettle, Gabriel Rogers and Mike Drummond on communicating cost-effectiveness analysis. I hope you will stay tuned for the ISPOR meeting round-up part 2!