Meeting round-up: ISPOR Europe 2019

For many health economists, November is ISPOR Europe month, and this year was no exception! We gathered in the fantastic Bella Center in Copenhagen to debate, listen and breathe health economics and outcomes research from the 2nd to the 6th November. Missed it? Would like a recap? Stay tuned for the #ISPOREurope 2019 round-up!

Bella Center

My ISPOR week started with the fascinating course ‘Tools for reproducible real-world data analysis’ by Blythe Adamson and Rachael Sorg. My key take-home messages? Use an interface like R-markdown to produce a document with code and results automatically. Use a version control platform like Phabricator to make code review easy. Write a detailed protocol, write the code to follow the protocol, and then check the code side by side with the protocol.

Monday started with the impressive workshop on translating oncology clinical trial endpoints to real-world data (RWD) for decision making.

Keith Abrams set the scene. Electronic health records (EHRs) may be used to derive the overall survival (OS) benefit given the observed benefit on progression-free survival (PFS). Sylwia Bujkiewicz showed an example where a bivariate meta-analysis of RCTs was used to estimate the surrogate relationship between PFS and OS (paper here). Jessica Davies discussed some of the challenges, such as the lack of data on exposure to treatments in a way that matches the data recorded in trials. Federico Felizzi presented a method to determine the optimal treatment duration of a cancer drug (see here for the code).

Next up, the Women in HEOR session! Women in HEOR is an ISPOR initiative that aims to support the growth, development, and contribution of women. It included various initiatives at ISPOR Europe, such as dinners, receptions and, of course, this session.

Shelby Reed introduced, and Olivia Wu presented on the overwhelming evidence on the benefits of diversity and on how to foster it in our work environment. Nancy Berg presented on ISPOR’s commitment to diversity and equality. We then heard from Sabina Hutchison about how to network in a conference environment, how to develop a personal brand and present our pitch. Have a look at my twitter thread for the tips. For more information on the Women in HEOR activities at ISPOR Europe, search #WomenInHEOR on twitter. Loads of cool information!

My Monday afternoon started with the provocatively titled ‘Time for change? Has time come for the pharma industry to accept modest prices?’. Have a look here for my live twitter thread. Kate Dion started by noting that the pressure is on for the pharmaceutical industry to reduce drug prices. Sarah Garner argued that lower prices lead to more patients being able to access the drug, which in turn increases the company’s income. Michael Schröter argued that innovative products should have a premium price, such as with Hemlibra. Lastly, Jens Grueger supported the implementation of value-based price, given the cost-effectiveness threshold.

Keeping with the drug pricing theme, my next session was on indication-based pricing. Mireia Jofre Bonet tackled the question of whether a single price is stifling innovation. Adrian Towse was supportive of indication-based pricing because it allows for the price to depend on the value of each indication and expand access to the full licensed population. Andrew Briggs argued against indication-based pricing for three reasons. First, it would give companies the maximum value-based price across all indications. Second, it would lead to greater drug expenditure, leading to greater opportunity costs. Third, it would be difficult to enforce, given that it would require cooperation of all payers. Francis Arickx explained the pricing system in Belgium. Remarkably, prices can be renegotiated over time depending on new entrants to market and new evidence. Another excellent session at ISPOR Europe!

My final session on Monday was about the timely and important topic of approaches for OS extrapolation. Elisabeth Fenwick introduced the session by noting that innovations in oncology have given rise to different patterns of survival, with implications for extrapolation. Sven Klijn presented on the various available methods for survival extrapolation. John Whalen focused on mixture cure models for cost-effectiveness analysis. Steve Palmer argued that, although new methods, such as mixture cure models, may provide additional insight, the approach should be justified, evidence-based and alternatives explored. In sum, there is no single optimal method.

On Tuesday, my first session was the impressive workshop on estimating cost-effectiveness thresholds based on the opportunity cost (twitter thread). Nancy Devlin set the scene by explaining the importance of getting the cost-effectiveness threshold right. James Lomas explained how to estimate the opportunity cost to the health care system following the seminal work by Karl Claxton et al and also touching on some of James’s recent work. Martin Henriksson noted that, by itself, the opportunity cost is not sufficient to define the threshold if we wish to consider solidarity and need alongside cost-effectiveness. The advantage of knowing the opportunity cost is that we can make informed trade-offs between health maximisation and other elements of value. Danny Palnoch finished the panel by explaining the challenges when deciding what to pay for a new treatment.

Clearly there is a tension between the price that pharmaceutical companies feel is reasonable, the opportunity cost to the health care service, and the desire by stakeholders to use the drug. I feel this in every session of the NICE appraisal committee!

My next session was the compelling panel on the use of RWD to revisit the HTA decision (twitter thread). Craig Brooks-Rooney noted that, as regulators increasingly license technologies based on weaker evidence, HTA agencies are under pressure to adapt their methods to the available evidence. Adrian Towse proposed a conceptual framework to use RWD to revisit decisions based on value of information analysis. Jeanette Kusel went through examples where RWD has been used to inform NICE decisions, such as brentuximab vendotin. Anna Halliday discussed the many practical challenges to implement RWD collection to inform re-appraisals. Anna finished with the caution against prolonging negotiations and appraisals, which could lead to delays to patient access.

My Wednesday started with the stimulating panel on drugs with tumour agnostic indications. Clarissa Higuchi Zerbini introduced the panel and proposed some questions to be addressed. Rosa Giuliani contributed with the clinical perspective. Jacoline Bouvy discussed the challenges faced by NICE and ways forward in appraising tumour-agnostic drugs. Marc van den Bulcke finished the panel with an overview of how next generation sequencing has been implemented in Belgium.

My last session was the brilliant workshop on HTA methods for antibiotics.

Mark Sculpher introduced the topic. Antibiotic resistance is a major challenge for humanity, but the development of new antibiotics is declining. Beth Woods presented a new framework for HTA of antibiotics. The goal is to reflect the full value of antibiotics whilst accounting for the opportunity cost and uncertainties in the evidence (see this report for more details). Angela Blake offered the industry perspective. She argued that revenues should be delinked to volume, to be holistic in the value assessment, and to be mindful of the incentives faced by drug companies. Nick Crabb finished by introducing a new project, by NICE and NHS England, on the feasibility of innovative value assessments for antibiotics.

And this is the end of the absolutely outstanding ISPOR Europe 2019! If you’re eager for more, have a look at the video below with my conference highlights!

Chris Sampson’s journal round-up for 30th September 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

A need for change! A coding framework for improving transparency in decision modeling. PharmacoEconomics [PubMed] Published 24th September 2019

We’ve featured a few papers in recent round-ups that (I assume) will be included in an upcoming themed issue of PharmacoEconomics on transparency in modelling. It’s shaping up to be a good one. The value of transparency in decision modelling has been recognised, but simply making the stuff visible is not enough – it needs to make sense. The purpose of this paper is to help make that achievable.

The authors highlight that the writing of analyses, including coding, involves personal style and preferences. To aid transparency, we need a systematic framework of conventions that make the inner workings of a model understandable to any (expert) user. The paper describes a framework developed by the Decision Analysis in R for Technologies in Health (DARTH) group. The DARTH framework builds on a set of core model components, generalisable to all cost-effectiveness analyses and model structures. There are five components – i) model inputs, ii) model implementation, iii) model calibration, iv) model validation, and v) analysis – and the paper describes the role of each. Importantly, the analysis component can be divided into several parts relating to, for example, sensitivity analyses and value of information analyses.

Based on this framework, the authors provide recommendations for organising and naming files and on the types of functions and data structures required. The recommendations build on conventions established in other fields and in the use of R generally. The authors recommend the implementation of functions in R, and relate general recommendations to the context of decision modelling. We’re also introduced to unit testing, which will be unfamiliar to most Excel modellers but which can be relatively easily implemented in R. The role of various tools are introduced, including R Studio, R Markdown, Shiny, and GitHub.

The real value of this work lies in the linked R packages and other online material, which you can use to test out the framework and consider its application to whatever modelling problem you might have. The authors provide an example using a basic Sick-Sicker model, which you can have a play with using the DARTH packages. In combination with the online resources, this is a valuable paper that you should have to hand if you’re developing a model in R.

Accounts from developers of generic health state utility instruments explain why they produce different QALYs: a qualitative study. Social Science & Medicine [PubMed] Published 19th September 2019

It’s well known that different preference-based measures of health will generate different health state utility values for the same person. Yet, they continue to be used almost interchangeably. For this study, the authors spoke to people involved in the development of six popular measures: QWB, 15D, HUI, EQ-5D, SF-6D, and AQoL. Their goal was to understand the bases for the development of the measures and to explain why the different measures should give different results.

At least one original developer for each instrument was recruited, along with people involved at later stages of development. Semi-structured interviews were conducted with 15 people, with questions on the background, aims, and criteria for the development of the measure, and on the descriptive system, preference weights, performance, and future development of the instrument.

Five broad topics were identified as being associated with differences in the measures: i) knowledge sources used for conceptualisation, ii) development purposes, iii) interpretations of what makes a ‘good’ instrument, iv) choice of valuation techniques, and v) the context for the development process. The online appendices provide some useful tables that summarise the differences between the measures. The authors distinguish between measures based on ‘objective’ definitions (QWB) and items that people found important (15D). Some prioritised sensitivity (AQoL, 15D), others prioritised validity (HUI, QWB), and several focused on pragmatism (SF-6D, HUI, 15D, EQ-5D). Some instruments had modest goals and opportunistic processes (EQ-5D, SF-6D, HUI), while others had grand goals and purposeful processes (QWB, 15D, AQoL). The use of some measures (EQ-5D, HUI) extended far beyond what the original developers had anticipated. In short, different measures were developed with quite different concepts and purposes in mind, so it’s no surprise that they give different results.

This paper provides some interesting accounts and views on the process of instrument development. It might prove most useful in understanding different measures’ blind spots, which can inform the selection of measures in research, as well as future development priorities.

The emerging social science literature on health technology assessment: a narrative review. Value in Health Published 16th September 2019

Health economics provides a good example of multidisciplinarity, with economists, statisticians, medics, epidemiologists, and plenty of others working together to inform health technology assessment. But I still don’t understand what sociologists are talking about half of the time. Yet, it seems that sociologists and political scientists are busy working on the big questions in HTA, as demonstrated by this paper’s 120 references. So, what are they up to?

This article reports on a narrative review, based on 41 empirical studies. Three broad research themes are identified: i) what drove the establishment and design of HTA bodies? ii) what has been the influence of HTA? and iii) what have been the social and political influences on HTA decisions? Some have argued that HTA is inevitable, while others have argued that there are alternative arrangements. Either way, no two systems are the same and it is not easy to explain differences. It’s important to understand HTA in the context of other social tendencies and trends, and that HTA influences and is influenced by these. The authors provide a substantial discussion on the role of stakeholders in HTA and the potential for some to attempt to game the system. Uncertainty abounds in HTA and this necessarily requires negotiation and acts as a limit on the extent to which HTA can rely on objectivity and rationality.

Something lacking is a critical history of HTA as a discipline and the question of what HTA is actually good for. There’s also not a lot of work out there on culture and values, which contrasts with medical sociology. The authors suggest that sociologists and political scientists could be more closely involved in HTA research projects. I suspect that such a move would be more challenging for the economists than for the sociologists.


Chris Sampson’s journal round-up for 23rd September 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Can you repeat that? Exploring the definition of a successful model replication in health economics. PharmacoEconomics [PubMed] Published 18th September 2019

People talk a lot about replication and its role in demonstrating the validity and reliability of analyses. But what does a successful replication in the context of cost-effectiveness modelling actually mean? Does it mean coming up with precisely the same estimates of incremental costs and effects? Does it mean coming up with a model that recommends the same decision? The authors of this study sought to bring us closer to an operational definition of replication success.

There is potentially much to learn from other disciplines that have a more established history of replication. The authors reviewed literature on the definition of ‘successful replication’ across all disciplines, and used their findings to construct a variety of candidate definitions for use in the context of cost-effectiveness modelling in health. Ten definitions of a successful replication were pulled out of the cross-disciplinary review, which could be grouped into ‘data driven’ replications and ‘experimental’ replications – the former relating to the replication of analyses and the latter relating to the replication of specific observed effects. The ten definitions were from economics, biostatistics, cognitive science, psychology, and experimental philosophy. The definitions varied greatly, with many involving subjective judgments about the proximity of findings. A few studies were found that reported on replications of cost-effectiveness models and which provided some judgment on the level of success. Again, these were inconsistent and subjective.

Quite reasonably, the authors judge that the lack of a fixed definition of successful replication in any scientific field is not just an oversight. The threshold for ‘success’ depends on the context of the replication and on how the evidence will be used. This paper provides six possible definitions of replication success for use in cost-effectiveness modelling, ranging from an identical replication of the results, through partial success in replicating specific pathways within a given margin of error, to simply replicating the same implied decision.

Ultimately, ‘data driven’ replications are a solution to a problem that shouldn’t exist, namely, poor reporting. This paper mostly convinced me that overall ‘success’ isn’t a useful thing to judge in the context of replicating decision models. Replication of certain aspects of a model is useful to evaluate. Whether the replication implied the same decision is a key thing to consider. Beyond this, it is probably worth considering partial success in replicating specific parts of a model.

Differential associations between interpersonal variables and quality-of-life in a sample of college students. Quality of Life Research [PubMed] Published 18th September 2019

There is growing interest in the well-being of students and the distinct challenges involved in achieving good mental health and addressing high levels of demand for services in this group. Students go through many changes that might influence their mental health, prominent among these is the change to their social situation.

This study set out to identify the role of key interpersonal variables on students’ quality of life. The study recruited 1,456 undergraduate students from four universities in the US. The WHOQOL measure was used for quality of life and a barrage of measures were used to collect information on loneliness, social connectedness, social support, emotional intelligence, intimacy, empathic concern, and more. Three sets of analyses of increasing sophistication were conducted, from zero-order correlations between each measure and the WHOQOL, to a network analysis using a Gaussian Graphical Model to identify both direct and indirect relationships while accounting for shared variance.

In all analyses, loneliness stuck out as the strongest driver of quality of life. Social support, social connectedness, emotional intelligence, intimacy with one’s romantic partner, and empathic concern were also significantly associated with quality of life. But the impact of loneliness was greatest, with other interpersonal variables influencing quality of life through their impact on loneliness.

This is a well-researched and reported study. The findings are informative to student support and other services that seek to improve the well-being of students. There is reason to believe that such services should recognise the importance of interpersonal determinants of well-being and in particular address loneliness. But it’s important to remember that this study is only as good as the measures it uses. If you don’t think WHOQOL is adequately measuring student well-being, or you don’t think the UCLA Loneliness Scale tells us what we need to know, you might not want these findings to influence practice. And, of course, the findings may not be generalisable, as the extent to which different interpersonal variables affect quality of life is very likely dependent on the level of service provision, which varies greatly between different universities, let alone countries.

Affordability and non-perfectionism in moral action. Ethical Theory and Moral Practice [PhilPapers] Published 14th September 2019

The ‘cost-effective but unaffordable’ challenge has been bubbling for a while now, at least since sofosbuvir came on the scene. This study explores whether “we can’t afford it” is a justifiable position to take. The punchline is that, no, affordability is not a sound ethical basis on which to support or reject the provision of a health technology. I was extremely sceptical when I first read the claim. If we can’t afford it, it’s impossible, and how can there by a moral imperative in an impossibility? But the authors proceeded to convince me otherwise.

The authors don’t go into great detail on this point, but it all hinges on divisibility. The reason that a drug like sofosbuvir might be considered unaffordable is that loads of people would be eligible to receive it. If sofosbuvir was only provided to a subset of this population, it could be affordable. On this basis, the authors propose the ‘principle of non-perfectionism’. This states that not being able to do all the good we can do (e.g. provide everyone who needs it with sofosbuvir) is not a reason for not doing some of the good we can do. Thus, if we cannot support provision of a technology to everyone who could benefit from it, it does not follow (ethically) to provide it to nobody, but rather to provide it to some people. The basis for selecting people is not of consequence to this argument but could be based on a lottery, for example.

Building on this, the authors explain to us why this is wrong, with the notion of ‘numerical discrimination’. They argue that it is not OK to prioritise one group over another simply because we can meet the needs of everyone within that group as opposed to only some members of the other group. This is exactly what’s happening when we are presented with notions of (un)affordability. If the population of people who could benefit from sofosbuvir was much smaller, there wouldn’t be an issue. But the simple fact that the group is large does not make it morally permissible to deny cost-effective treatment to any individual member within that group. You can’t discriminate against somebody because they are from a large population.

I think there are some tenuous definitions in the paper and some questionable analogies. Nevertheless, the authors succeeded in convincing me that total cost has no moral weight. It is irrelevant to moral reasoning. We should not refuse any health technology to an entire population on the grounds that it is ‘unaffordable’. The authors frame it as a ‘mistake in moral mathematics’. For this argument to apply in the HTA context, it relies wholly on the divisibility of health technologies. To some extent, NICE and their counterparts are in the business of defining models of provision, which might result in limited use criteria to get around the affordability issue. Though these issues are often handled by payers such as NHS England.

The authors of this paper don’t consider the implications for cost-effectiveness thresholds, but this is where my thoughts turned. Does the principle of non-perfectionism undermine the morality of differentiating cost-effectiveness thresholds according to budget impact? I think it probably does. Reducing the threshold because the budget impact is great will result in discrimination (‘numerical discrimination’) against individuals simply because they are part of a large population that could benefit from treatment. This seems to be the direction in which we’re moving. Maybe the efficiency cart is before the ethical horse.