Chris Sampson’s journal round-up for 11th June 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

End-of-life healthcare expenditure: testing economic explanations using a discrete choice experiment. Journal of Health Economics Published 7th June 2018

People incur a lot of health care costs at the end of life, despite the fact that – by definition – they aren’t going to get much value from it (so long as we’re using QALYs, anyway). In a 2007 paper, Gary Becker and colleagues put forward a theory for the high value of life and high expenditure on health care at the end of life. This article sets out to test a set of hypotheses derived from this theory, namely: i) higher willingness-to-pay (WTP) for health care with proximity to death, ii) higher WTP with greater chance of survival, iii) societal WTP exceeds individual WTP due to altruism, and iv) societal WTP may exceed individual WTP due to an aversion to restricting access to new end-of-life care. A further set of hypotheses relating to the ‘pain of risk-bearing’ is also tested. The authors conducted an online discrete choice experiment (DCE) with 1,529 Swiss residents, which asked respondents to suppose that they had terminal cancer and was designed to elicit WTP for a life-prolonging novel cancer drug. Attributes in the DCE included survival, quality of life, and ‘hope’ (chance of being cured). Individual WTP – using out-of-pocket costs – and societal WTP – based on social health insurance – were both estimated. The overall finding is that the hypotheses are on the whole true, at least in part. But the fact is that different people have different preferences – the authors note that “preferences with regard to end-of-life treatment are very heterogeneous”. The findings provide evidence to explain the prevailing high level of expenditure in end of life (cancer) care. But the questions remain of what we can or should do about it, if anything.

Valuation of preference-based measures: can existing preference data be used to generate better estimates? Health and Quality of Life Outcomes [PubMed] Published 5th June 2018

The EuroQol website lists EQ-5D-3L valuation studies for 27 countries. As the EQ-5D-5L comes into use, we’re going to see a lot of new valuation studies in the pipeline. But what if we could use data from one country’s valuation to inform another’s? The idea is that a valuation study in one country may be able to ‘borrow strength’ from another country’s valuation data. The author of this article has developed a Bayesian non-parametric model to achieve this and has previously applied it to UK and US EQ-5D valuations. But what about situations in which few data are available in the country of interest, and where the country’s cultural characteristics are substantially different. This study reports on an analysis to generate an SF-6D value set for Hong Kong, firstly using the Hong Kong values only, and secondly using the UK value set as a prior. As expected, the model which uses the UK data provided better predictions. And some of the differences in the valuation of health states are quite substantial (i.e. more than 0.1). Clearly, this could be a useful methodology, especially for small countries. But more research is needed into the implications of adopting the approach more widely.

Can a smoking ban save your heart? Health Economics [PubMed] Published 4th June 2018

Here we have another Swiss study, relating to the country’s public-place smoking bans. Exposure to tobacco smoke can have an acute and rapid impact on health to the extent that we would expect an immediate reduction in the risk of acute myocardial infarction (AMI) if a smoking ban reduces the number of people exposed. Studies have already looked at this effect, and found it to be large, but mostly with simple pre-/post- designs that don’t consider important confounding factors or prevailing trends. This study tests the hypothesis in a quasi-experimental setting, taking advantage of the fact that the 26 Swiss cantons implemented smoking bans at different times between 2007 and 2010. The authors analyse individual-level data from Swiss hospitals, estimating the impact of the smoking ban on AMI incidence, with area and time fixed effects, area-specific time trends, and unemployment. The findings show a large and robust effect of the smoking ban(s) for men, with a reduction in AMI incidence of about 11%. For women, the effect is weaker, with an average reduction of around 2%. The evidence also shows that men in low-education regions experienced the greatest benefit. What makes this an especially nice paper is that the authors bring in other data sources to help explain their findings. Panel survey data are used to demonstrate that non-smokers are likely to be the group benefitting most from smoking bans and that people working in public places and people with less education are most exposed to environmental tobacco smoke. These findings might not be generalisable to other settings. Other countries implemented more gradual policy changes and Switzerland had a particularly high baseline smoking rate. But the findings suggest that smoking bans are associated with population health benefits (and the associated cost savings) and could also help tackle health inequalities.

Credits

Chris Sampson’s journal round-up for 25th September 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Good practices for real‐world data studies of treatment and/or comparative effectiveness: recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making. Value in Health Published 15th September 2017

I have an instinctive mistrust of buzzwords. They’re often used to avoid properly defining something, either because it’s too complicated or – worse – because it isn’t worth defining in the first place. For me, ‘real-world evidence’ falls foul. If your evidence isn’t from the real world, then it isn’t evidence at all. But I do like a good old ISPOR Task Force report, so let’s see where this takes us. Real-world evidence (RWE) and its sibling buzzword real-world data (RWD) relate to observational studies and other data not collected in an experimental setting. The purpose of this ISPOR task force (joint with the International Society for Pharmacoepidemiology) was to prepare some guidelines about the conduct of RWE/RWD studies, with a view to improving decision-makers’ confidence in them. Essentially, the hope is to try and create for RWE the kind of ecosystem that exists around RCTs, with procedures for study registration, protocols, and publication: a noble aim. The authors distinguish between 2 types of RWD: ‘Exploratory Treatment Effectiveness Studies’ and ‘Hypothesis Evaluating Treatment Effectiveness Studies’. The idea is that the latter test a priori hypotheses, and these are the focus of this report. Seven recommendations are presented: i) pre-specify the hypotheses, ii) publish a study protocol, iii) publish the study with reference to the protocol, iv) enable replication, v) test hypotheses on a separate dataset than the one used to generate the hypotheses, vi) publically address methodological criticisms, and vii) involve key stakeholders. Fair enough. But these are just good practices for research generally. It isn’t clear how they are in any way specific to RWE. Of course, that was always going to be the case. RWE-specific recommendations would be entirely contingent on whether or not one chose to define a study as using ‘real-world evidence’ (which you shouldn’t, because it’s meaningless). The authors are trying to fit a bag of square pegs into a hole of undefined shape. It isn’t clear to me why retrospective observational studies, prospective observational studies, registry studies, or analyses of routinely collected clinical data should all be treated the same, yet differently to randomised trials. Maybe someone can explain why I’m mistaken, but this report didn’t do it.

Are children rational decision makers when they are asked to value their own health? A contingent valuation study conducted with children and their parents. Health Economics [PubMed] [RePEc] Published 13th September 2017

Obtaining health state utility values for children presents all sorts of interesting practical and theoretical problems, especially if we want to use them in decisions about trade-offs with adults. For this study, the researchers conducted a contingent valuation exercise to elicit children’s (aged 7-19) preferences for reduced risk of asthma attacks in terms of willingness to pay. The study was informed by two preceding studies that sought to identify the best way in which to present health risk and financial information to children. The participating children (n=370) completed questionnaires at school, which asked about socio-demographics, experience of asthma, risk behaviours and altruism. They were reminded (in child-friendly language) about the idea of opportunity cost, and to consider their own budget constraint. Baseline asthma attack risk and 3 risk-reduction scenarios were presented graphically. Two weeks later, the parents completed similar questionnaires. Only 9% of children were unwilling to pay for risk reduction, and most of those said that it was the mayor’s problem! In some senses, the children did a better job than their parents. The authors conducted 3 tests for ‘incorrect’ responses – 14% of adults failed at least one, while only 4% of children did so. Older children demonstrated better scope sensitivity. Of course, children’s willingness to pay was much lower in absolute terms than their parents’, because children have a much smaller budget. As a percentage of the budget, parents were – on average – willing to pay more than children. That seems reassuringly predictable. Boys and fathers were willing to pay more than girls and mothers. Having experience of frequent asthma attacks increased willingness to pay. Interestingly, teenagers were willing to pay less (as a proportion of their budget) than younger children… and so were the teenagers’ parents! Children’s willingness to pay was correlated with that of their own parent’s at the higher risk reductions but not the lowest. This study reports lots of interesting findings and opens up plenty of avenues for future research. But the take-home message is obvious. Kids are smart. We should spend more time asking them what they think.

Journal of Patient-Reported Outcomes: aims and scope. Journal of Patient-Reported Outcomes Published 12th September 2017

Here we have a new journal that warrants a mention. The journal is sponsored by the International Society for Quality of Life Research (ISOQOL), making it a sister journal of Quality of Life Research. One of its Co-Editors-in-Chief is the venerable David Feeny, of HUI fame. They’ll be looking to publish research using PRO(M) data from trials or routine settings, studies of the determinants of PROs, qualitative studies in the development of PROs; anything PRO-related, really. This could be a good journal for more thorough reporting of PRO data that can get squeezed out of a study’s primary outcome paper. Also, “JPRO” is fun to say. The editors don’t mention that the journal is open access, but the website states that it is, so APCs at the ready. ISOQOL members get a discount.

Research and development spending to bring a single cancer drug to market and revenues after approval. JAMA Internal Medicine [PubMed] Published 11th September 2017

We often hear that new drugs are expensive because they’re really expensive to develop. Then we hear about how much money pharmaceutical companies spend on marketing, and we baulk. The problem is, pharmaceutical companies aren’t forthcoming with their accounts, so researchers have to come up with more creative ways to estimate R&D spending. Previous studies have reported divergent estimates. Whether R&D costs ‘justify’ high prices remains an open question. For this study, the authors looked at public data from the US for 10 companies that had only one cancer drug approved by the FDA between 2007 and 2016. Not very representative, perhaps, but useful because it allows for the isolation of the development costs associated with a single drug reaching the market. The median time for drug development was 7.3 years. The most generous estimate of the mean cost of development came in at under a billion dollars; substantially less than some previous estimates. This looks like a bargain; the mean revenue for the 10 companies up to December 2016 was over $6.5 billion. This study may seem a bit back-of-the-envelope in nature. But that doesn’t mean it isn’t accurate. If anything, it begs more confidence than some previous studies because the methods are entirely transparent.

Credits

 

 

What’s the significance of this?

A good illustration of the muddles that p-values can get us in appeared recently on HealthNewsReview.com. HealthNewsReview examines and debunks the often hyped-up claims about medicines that appear in the media. But last week they “called BS” on a claim on Novartis’ website for the drug Everolimus. Novartis claimed that in a recent trial Everolimus demonstrated benefits that were “not statistically significant but clinically meaningful.” HealthNewsReview writes:

When results aren’t statistically significant, researchers can’t be sufficiently confident that any benefit they observed is real. Such findings are considered speculative until confirmed by other studies.

Sometimes, a result that was initially “not significant” might well reach the threshold of significance in a bigger study group with more patients, which is what this promotional material seems to anticipate.

And they quote a biostatistician further on:

A result that is statistically insignificant is not meaningful, period. Thus, we cannot say a result is statistically insignificant and clinically meaningful at the same time.

A null hypothesis significance testing (NHST) framework aims to determine whether the data are compatible with a model that the coefficient on the treatment is exactly zero. For the Everolimus trial, the t-test did not hit the magical 1.96 threshold and so it has been concluded there either was an effect of exactly zero or there was insufficient power. Hence it is “not meaningful”.

This is where the problems of NHST become obvious. Everolimus is an mTOR inhibitor, a class of drugs under active development for the treatment of cancer. Hyperactivation of mTOR signalling in cancer has been widely observed and various preclinical trials have shown promising results (see more here). So why one should expect it to have an effect of exactly zero, I can’t say.

Perhaps more importantly, this is where the irrelevance of inference rears its head. A decision to use Everolimus has to be made. It cannot be deferred and the only reason we use other treatments at this point in time is an accident of history. As we discussed recently, all that matters for these decisions is the (posterior) mean net benefits. Although, in the US, costs information has been outlawed because of those pernicious “death panels”. Decisions are made on the basis of “comparative effectiveness“. But even in this case, the above comments on Everolimus do not follow this logic, seeming to imply: (i) in the absence of statistical significance we have learned nothing from the data to inform our decision; and (ii) we should only choose to implement technologies that have demonstrated statistical significance. If taken as true then we have no choice but to conflate clinical and statistical significance, since apparently we cannot conclude something is clinically significant unless it is also statistically significant. This goes against all sound advice.

I write this without good knowledge of the original Everolimus trial. It may well be flawed. Industry funded research can be biased. Indeed, it is the design and conduct of the trial should be the basis for a reasonable critique of its findings, not its statistical significance.

HealthNewsReview is a typically excellent reviewer of claims derived from often flawed studies. Their conflation of statistical and clinical significance here though is by no means unique, being used by many regulatory agencies around the world. This just goes to show how the p-value can continue to distract from a sound decision making process in health care.