Chris Sampson’s journal round-up for 23rd September 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Can you repeat that? Exploring the definition of a successful model replication in health economics. PharmacoEconomics [PubMed] Published 18th September 2019

People talk a lot about replication and its role in demonstrating the validity and reliability of analyses. But what does a successful replication in the context of cost-effectiveness modelling actually mean? Does it mean coming up with precisely the same estimates of incremental costs and effects? Does it mean coming up with a model that recommends the same decision? The authors of this study sought to bring us closer to an operational definition of replication success.

There is potentially much to learn from other disciplines that have a more established history of replication. The authors reviewed literature on the definition of ‘successful replication’ across all disciplines, and used their findings to construct a variety of candidate definitions for use in the context of cost-effectiveness modelling in health. Ten definitions of a successful replication were pulled out of the cross-disciplinary review, which could be grouped into ‘data driven’ replications and ‘experimental’ replications – the former relating to the replication of analyses and the latter relating to the replication of specific observed effects. The ten definitions were from economics, biostatistics, cognitive science, psychology, and experimental philosophy. The definitions varied greatly, with many involving subjective judgments about the proximity of findings. A few studies were found that reported on replications of cost-effectiveness models and which provided some judgment on the level of success. Again, these were inconsistent and subjective.

Quite reasonably, the authors judge that the lack of a fixed definition of successful replication in any scientific field is not just an oversight. The threshold for ‘success’ depends on the context of the replication and on how the evidence will be used. This paper provides six possible definitions of replication success for use in cost-effectiveness modelling, ranging from an identical replication of the results, through partial success in replicating specific pathways within a given margin of error, to simply replicating the same implied decision.

Ultimately, ‘data driven’ replications are a solution to a problem that shouldn’t exist, namely, poor reporting. This paper mostly convinced me that overall ‘success’ isn’t a useful thing to judge in the context of replicating decision models. Replication of certain aspects of a model is useful to evaluate. Whether the replication implied the same decision is a key thing to consider. Beyond this, it is probably worth considering partial success in replicating specific parts of a model.

Differential associations between interpersonal variables and quality-of-life in a sample of college students. Quality of Life Research [PubMed] Published 18th September 2019

There is growing interest in the well-being of students and the distinct challenges involved in achieving good mental health and addressing high levels of demand for services in this group. Students go through many changes that might influence their mental health, prominent among these is the change to their social situation.

This study set out to identify the role of key interpersonal variables on students’ quality of life. The study recruited 1,456 undergraduate students from four universities in the US. The WHOQOL measure was used for quality of life and a barrage of measures were used to collect information on loneliness, social connectedness, social support, emotional intelligence, intimacy, empathic concern, and more. Three sets of analyses of increasing sophistication were conducted, from zero-order correlations between each measure and the WHOQOL, to a network analysis using a Gaussian Graphical Model to identify both direct and indirect relationships while accounting for shared variance.

In all analyses, loneliness stuck out as the strongest driver of quality of life. Social support, social connectedness, emotional intelligence, intimacy with one’s romantic partner, and empathic concern were also significantly associated with quality of life. But the impact of loneliness was greatest, with other interpersonal variables influencing quality of life through their impact on loneliness.

This is a well-researched and reported study. The findings are informative to student support and other services that seek to improve the well-being of students. There is reason to believe that such services should recognise the importance of interpersonal determinants of well-being and in particular address loneliness. But it’s important to remember that this study is only as good as the measures it uses. If you don’t think WHOQOL is adequately measuring student well-being, or you don’t think the UCLA Loneliness Scale tells us what we need to know, you might not want these findings to influence practice. And, of course, the findings may not be generalisable, as the extent to which different interpersonal variables affect quality of life is very likely dependent on the level of service provision, which varies greatly between different universities, let alone countries.

Affordability and non-perfectionism in moral action. Ethical Theory and Moral Practice [PhilPapers] Published 14th September 2019

The ‘cost-effective but unaffordable’ challenge has been bubbling for a while now, at least since sofosbuvir came on the scene. This study explores whether “we can’t afford it” is a justifiable position to take. The punchline is that, no, affordability is not a sound ethical basis on which to support or reject the provision of a health technology. I was extremely sceptical when I first read the claim. If we can’t afford it, it’s impossible, and how can there by a moral imperative in an impossibility? But the authors proceeded to convince me otherwise.

The authors don’t go into great detail on this point, but it all hinges on divisibility. The reason that a drug like sofosbuvir might be considered unaffordable is that loads of people would be eligible to receive it. If sofosbuvir was only provided to a subset of this population, it could be affordable. On this basis, the authors propose the ‘principle of non-perfectionism’. This states that not being able to do all the good we can do (e.g. provide everyone who needs it with sofosbuvir) is not a reason for not doing some of the good we can do. Thus, if we cannot support provision of a technology to everyone who could benefit from it, it does not follow (ethically) to provide it to nobody, but rather to provide it to some people. The basis for selecting people is not of consequence to this argument but could be based on a lottery, for example.

Building on this, the authors explain to us why this is wrong, with the notion of ‘numerical discrimination’. They argue that it is not OK to prioritise one group over another simply because we can meet the needs of everyone within that group as opposed to only some members of the other group. This is exactly what’s happening when we are presented with notions of (un)affordability. If the population of people who could benefit from sofosbuvir was much smaller, there wouldn’t be an issue. But the simple fact that the group is large does not make it morally permissible to deny cost-effective treatment to any individual member within that group. You can’t discriminate against somebody because they are from a large population.

I think there are some tenuous definitions in the paper and some questionable analogies. Nevertheless, the authors succeeded in convincing me that total cost has no moral weight. It is irrelevant to moral reasoning. We should not refuse any health technology to an entire population on the grounds that it is ‘unaffordable’. The authors frame it as a ‘mistake in moral mathematics’. For this argument to apply in the HTA context, it relies wholly on the divisibility of health technologies. To some extent, NICE and their counterparts are in the business of defining models of provision, which might result in limited use criteria to get around the affordability issue. Though these issues are often handled by payers such as NHS England.

The authors of this paper don’t consider the implications for cost-effectiveness thresholds, but this is where my thoughts turned. Does the principle of non-perfectionism undermine the morality of differentiating cost-effectiveness thresholds according to budget impact? I think it probably does. Reducing the threshold because the budget impact is great will result in discrimination (‘numerical discrimination’) against individuals simply because they are part of a large population that could benefit from treatment. This seems to be the direction in which we’re moving. Maybe the efficiency cart is before the ethical horse.

Credits

Sofosbuvir: a fork in the road for NICE?

NICE recently completed their appraisal of the hepatitis C drug sofosbuvir. However, as has been reported in the media, NHS England will not be complying with the guidance within the normal time period.

The cost of a 24 week course of sofosbuvir is almost £70,000. Around 160,000 people are chronically infected with the hepatitis C virus in England, so that adds up to a fair chunk of the NHS budget. Yet the drug does appear to be cost-effective. ICERs differ for different patient groups, but for most scenarios the ICER is below £30,000 per QALY. In the NICE documentation, a number of reasons are listed for NHS England’s decision. But what they ultimately boil down to – it seems – is affordability.

The problem is that NICE doesn’t account for affordability in its guidance. One need only consider that the threshold has remained unchanged for over a decade to see that this is true. How to solve this problem really depends on what we believe the job of NICE should be. Should it be NICE’s job to consider what should and shouldn’t be purchased within the existing health budget? Or, rather, should it be NICE’s job simply to figure out what is ‘worth it’ to society, regardless of affordability? This isn’t the first time that an NHS organisation has appealed against a NICE decision in some way. Surely, it won’t be the last. These instances represent a failure in the system, not least on grounds of accountability for reasonableness. Here I’d like to suggest that NICE has 3 options for dealing with this problem; one easy, one hard and one harder.

The easy option

The simplest option involves the fewest changes to the NICE process. Indeed, it would involve doing pretty much what it does now, only with slightly different (and more transparent) reasoning. In this scenario NICE would explicitly ignore the problem of affordability. Its remit would cease to be the consideration of optimality on a national level and it would ignore the budget constraint. NICE’s remit would become figuring out which health technologies are ‘worth it’; i.e. would the public be willing to purchase a given technology with a given health benefit at a given cost. To some extent, therefore, NICE would become a threshold-setter. The threshold should be based on some definition of a social value of a QALY. This is the easy option for NICE as setting the threshold would be the only additional task to what they currently do. Its threshold might not change all that much, or may be a little higher.

However, even if NICE denies responsibility, clearly someone does need to take account of affordability. Given the events associated with sofosbuvir it seems that this could become the work of NHS England. NHS England could use a threshold based on the budget and current QALY-productivity in the NHS. One might expect NHS England to be in a better position to identify the local evidence necessary to determine appropriate thresholds, which would likely be much lower than NICE’s. It would also be responsible for disinvestment decisions. Given the nationwide remit of NHS England, this would still prevent postcode lotteries. The implication here, of course, is that NICE and NHS England might use different thresholds. Any number of decision rules could be used to determine the result for technologies falling between the two. Maybe this is where considerations for innovation or non-health-related equity concerns belong. It seems probable to me that NICE’s threshold would be higher than NHS England’s, in which case NICE would effectively be advising increases in the health budget. This is something that I quite like the sound of.

The hard option

Personally, I believe that NICE’s failure to justify their threshold(s) is quite a serious failing and undermines the enterprise. The hard option will involve them defining it properly, informed by current levels of QALY-productivity in the NHS. Thus properly adopting a position as a threshold-searcher, and doing the job prescribed to NHS England in the ‘easy option’. NICE guidance would therefore be informed by the current health budget and affordability, and therefore must include guidance on disinvestment. The first stage of this work has already been done. The disinvestment guidance would be the hard part. This argument has already been much discussed, and seems to be what many economists support.

I don’t find this argument entirely compelling, at least not as a solution to the affordability problem. To solve this issue NICE would need to regularly review the current threshold and revise it in light of current productivity and the prevailing health budget. It has no experience of doing this. I believe the task could be more effectively carried out by commissioning organisations (such as NHS England), who are in a better position to oversee the collection of the appropriate data and would have a public responsibility to do so. It might also be politically useful if decisions about affordability were made independently of decisions about value.

The harder option

The harder option is for there to be a paradigm shift in the way NICE – and health economics more generally – operates. It could involve programme budgeting and marginal analysis, or the Birch and Gafni approach. This might just be the best option, but it seems unlikely to happen nationally any time soon.

It’s possible that more cost-effective but unaffordable drugs are in the pipeline. Failure to address the affordability problem soon could seriously undermine NICE.

DOI: 10.6084/m9.figshare.1291123