Chris Sampson’s journal round-up for 12th August 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Developing open-source models for the US health system: practical experiences and challenges to date with the Open-Source Value Project. PharmacoEconomics [PubMed] Published 7th August 2019

PharmacoEconomics will soon publish a themed issue on transparency in decision modelling (to which I’ve contributed), and this paper – I assume – is one that will feature. At least one output from the Open-Source Value Project has featured in these round-ups before. The purpose of this paper is to describe the experiences of the initiative in developing and releasing two open-source models, one in rheumatoid arthritis and one in lung cancer.

The authors outline the background to the project and its goal to develop credible models that are more tuned-in to stakeholders’ needs. By sharing the R and C++ source code, developing interactive web applications, and providing extensive documentation, the models are intended to be wholly transparent and flexible. The model development process also involves feedback from experts and the public, followed by revision and re-release. It’s a huge undertaking. The paper sets out the key challenges associated with this process, such as enabling stakeholders with different backgrounds to understand technical models and each other. The authors explain how they have addressed such difficulties along the way. The resource implications of this process are also challenging, because the time and expertise required are much greater than for run-of-the-mill decision models. The advantages of the tools used by the project, such as R and GitHub, are explained, and the paper provides some ammunition for the open-source movement. One of the best parts of the paper is the authors’ challenge to those who question open-source modelling on the basis of intellectual property concerns. For example, they state that, “Claiming intellectually property on the implementation of a relatively common modeling approach in Excel or other programming software, such as a partitioned survival model in oncology, seems a bit pointless.” Agreed.

The response to date from the community has been broadly positive, though there has been a lack of engagement from US decision-makers. Despite this, the initiative has managed to secure adequate funding. This paper is a valuable read for anyone involved in open-source modelling or in establishing a collaborative platform for the creation and dissemination of research tools.

Incorporating affordability concerns within cost-effectiveness analysis for health technology assessment. Value in Health Published 30th July 2019

The issue of affordability is proving to be a hard nut to crack for health economists. That’s probably because we’ve spent a very long time conducting incremental cost-effectiveness analyses that pay little or no attention to the budget constraint. This paper sets out to define a framework that finally brings affordability into the fold.

The author sets up an example with a decision-maker that seeks to maximise population health with a fixed budget – read, HTA agency – and the motivating example is new medicines for hepatitis C. The core of the proposal is an alternative decision rule. Rather than simply comparing the incremental cost-effectiveness ratio (ICER) to a fixed threshold, it incorporates a threshold that is a function of the budget impact. At it’s most basic, a bigger budget impact (all else equal) means a greater opportunity cost and thus a lower threshold. The author suggests doing away with the ICER (which is almost impossible to work with) and instead using net health benefits. In this framework, whether or not net health benefit is greater than zero depends on the size of the budget impact at any given ICER. If we accept the core principle that budget impact should be incorporated into the decision rule, it raises two other issues – time and uncertainty – which are also addressed in the paper. The framework moves us beyond the current focus on net present value, which ignores the distribution of costs over time beyond simply discounting future expenditure. Instead, the opportunity cost ‘threshold’ depends on the budget impact in each time period. The description of the framework also addresses uncertainty in budget impact, which requires the estimation of opportunity costs in each iteration of a probabilistic analysis.

The paper is thorough in setting out the calculations needed to implement this framework. If you’re conducting an economic evaluation of a technology that could have a non-marginal (big) budget impact, you should tag this on to your analysis plan. Once researchers start producing these estimates, we’ll be able to understand how important these differences could be for resource allocation decision-making and determine whether the likes of NICE ought to incorporate it into their methods guide.

Did UberX reduce ambulance volume? Health Economics [PubMed] [RePEc] Published 24th June 2019

In London, you can probably – at most times of day – get an Uber quicker than you can get an ambulance. That isn’t necessarily a bad thing, as ambulances aren’t there to provide convenience. But it does raise an interesting question. Could the availability of super-fast, low-cost, low-effort taxi hailing reduce pressure on ambulance services? If so, we might anticipate the effect to be greatest where people have to actually pay for ambulances.

This study combines data on Uber market entry in the US, by state and city, with ambulance rates. Between Q1 2012 and Q4 2015, the proportion of the US population with access to Uber rose from 0% to almost 25%. The authors are also able to distinguish ‘lights and sirens’ ambulance rides from ‘no lights and sirens’ rides. A difference-in-differences model estimates the ambulance rate for a given city by quarter-year. The analysis suggests that there was a significant decline in ambulance rates in the years following Uber’s entry to the market, implying an average of 1.2 fewer ambulance trips per 1,000 population per quarter.

There are some questionable results in here, including the fact that a larger effect was found for the ‘lights and sirens’ ambulance rate, so it’s not entirely clear what’s going on. The authors describe a variety of robustness checks for our consideration. Unfortunately, the discussion of the results is lacking in detail and insight, so readers need to figure it out themselves. I’d be very interested to see a similar analysis in the UK. I suspect that I would be inclined to opt for an Uber over an ambulance in many cases. And I wouldn’t have the usual concern about Uber exploiting its drivers, as I dare say ambulance drivers aren’t treated much better.

Credits

Chris Sampson’s journal round-up for 5th August 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The barriers and facilitators to model replication within health economics. Value in Health Published 16th July 2019

Replication is a valuable part of the scientific process, especially if there are uncertainties about the validity of research methods. When it comes to cost-effectiveness modelling, there are endless opportunities for researchers to do things badly, even with the best intentions. Attempting to replicate modelling studies can therefore support health care decision-making. But replication studies are rarely conducted, or, at least, rarely reported. The authors of this study sought to understand the factors that can make replication easy or difficult, with a view to informing reporting standards.

The authors attempted to replicate five published cost-effectiveness modelling studies, with the aim of recreating the key results. Each replication attempt was conducted by a different author and we’re even given a rating of the replicator’s experience level. The characteristics of the models were recorded and each replicator detailed – anecdotally – the things that helped or hindered their attempt. Some replications were a resounding failure. In one case, the replicated cost per patient was more than double the original, at more than £1,000 wide of the mark. Replicators reported that having a clear diagram of the model structure was a big help, as was the provision of example calculations and explicit listing of the key assumptions. Various shortcomings made replication difficult, all relating to a lack of clarity or completeness in reporting. The impact of this on the validation attempt was exacerbated if the model either involved lots of scenarios that weren’t clearly described or if the model had a long time horizon.

The quality of each study was assessed using the Philips checklist, and all did pretty well, suggesting that the checklist is not sufficient for ensuring replicability. If you develop and report cost-effectiveness models, this paper could help you better understand how end-users will interpret your reporting and make your work more replicable. This study focusses on Markov models. They’re definitely the most common approach, so perhaps that’s OK. It might be useful to produce prescriptive guidance specific to Markov models, informed by the findings of this study.

US integrated delivery networks perspective on economic burden of patients with treatment-resistant depression: a retrospective matched-cohort study. PharmacoEconomics – Open [PubMed] Published 28th June 2019

Treatment-resistant depression can be associated high health care costs, as multiple lines of treatment are tried, with patients experiencing little or no benefit. New treatments and models of care can go some way to addressing these challenges. In the US, there’s some reason to believe that integrated delivery networks (IDNs) could be associated with lower care costs, because IDNs are based on collaborative care models and constitute a single point of accountability for patient costs. They might be particularly useful in the case of treatment-resistant depression, but evidence is lacking. The authors of this study investigated the difference in health care resource use and costs for patients with and without treatment-resistant depression, in the context of IDNs.

The researchers conducted a retrospective cohort study using claims data for people receiving care from IDNs, with up to two years follow-up from first antidepressant use. 1,582 people with treatment-resistant depression were propensity score matched to two other groups – patients without depression and patients with depression that was not classified as treatment-resistant. Various regression models were used to compare the key outcomes of all-cause and specific categories of resource use and costs. Unfortunately, there is no assessment of whether the selected models are actually any good at estimating differences in costs.

The average costs and resource use levels in the three groups ranked as you would expect: $25,807 per person per year for the treatment-resistant group versus $13,701 in the non-resistant group and $8,500 in the non-depression group. People with treatment-resistant depression used a wider range of antidepressants and for a longer duration. They also had twice as many inpatient visits as people with depression that wasn’t treatment-resistant, which seems to have been the main driver of the adjusted differences in costs.

We don’t know (from this study) whether or not IDNs provide a higher quality of care. And the study isn’t able to compare IDN and non-IDN models of care. But it does show that IDNs probably aren’t a full solution to the high costs of treatment-resistant depression.

Rabin’s paradox for health outcomes. Health Economics [PubMed] [RePEc] Published 19th June 2019

Rabin’s paradox arises from the theoretical demonstration that a risk-averse individual who turns down a 50:50 gamble of gaining £110 or losing £100 would, if expected utility theory is correct, turn down a 50:50 gamble of losing £1,000 or gaining millions. This is because of the assumed concave utility function over wealth that is used to model risk aversion and it is probably not realistic. But we don’t know about the relevance of this paradox in the health domain… until now.

A key contribution of this paper is that it considers both decision-making about one’s own health and decision-making from a societal perspective. Three different scenarios are set-up in each case, relating to gains and losses in life expectancy with different levels of health functioning. 201 students were recruited as part of a larger study on preferences and each completed all six gamble-pairs (three individual, three societal). To test for Rabin’s paradox, the participants were asked whether they would accept each gamble involving a moderate stake and a large stake.

In short, the authors observe Rabin’s proposed failure of expected utility theory. Many participants rejected small gambles but did not reject the larger gambles. The effect was more pronounced for societal preferences. Though there was a large minority for whom expected utility theory was not violated. The upshot of all this is that our models of health preferences that are based on expected utility may be flawed where uncertain outcomes are involved – as they often are in health. This study adds to a growing body of literature supporting the relevance of alternative utility theories, such as prospect theory, to health and health care.

My only problem here is that life expectancy is not health. Life expectancy is everything. It incorporates the monetary domain, which this study did not want to consider, as well as every other domain of life. When you die, your stock of cash is as useful to you as your stock of health. I think it would have been more useful if the study focussed only on health status and outcomes and excluded all considerations of death.

Credits

Chris Sampson’s journal round-up for 18th February 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

An educational review about using cost data for the purpose of cost-effectiveness analysis. PharmacoEconomics [PubMed] Published 12th February 2019

Costing can seem like a cinderella method in the health economist’s toolkit. If you’re working on an economic evaluation, estimating resource use and costs can be tedious. That is perhaps why costing methodology has been relatively neglected in the literature compared to health state valuation (for example). This paper tries to redress the balance slightly by providing an overview of the main issues in costing, explaining why they’re important, so that we can do a better job. The issues are more complex than many assume.

Supported by a formidable reference list (n=120), the authors tackle 9 issues relating to costing: i) costs vs resource use; ii) trial-based vs model-based evaluations; iii) costing perspectives; iv) data sources; v) statistical methods; vi) baseline adjustments; vii) missing data; viii) uncertainty; and ix) discounting, inflation, and currency. It’s a big paper with a lot to say, so it isn’t easily summarised. Its role is as a reference point for us to turn to when we need it. There’s a stack of papers and other resources cited in here that I wasn’t aware of. The paper itself doesn’t get technical, leaving that to the papers cited therein. But the authors provide a good discussion of the questions that ought to be addressed by somebody designing a study, relating to data collection and analysis.

The paper closes with some recommendations. The main one is that people conducting cost-effectiveness analysis should think harder about why they’re making particular methodological choices. The point is also made that new developments could change the way we collect and analyse cost data. For example, the growing use of observational data demands that greater consideration be given to unobserved confounding. Costing methods are important and interesting!

A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. PharmacoEconomics [PubMed] Published 9th February 2019

Wherever feasible, decision models should be published open-source, so that they can be reviewed, reused, recycled, or, perhaps, rejected. But open-source models are still a rare sight. Here, we have one for rheumatoid arthritis. But the paper isn’t really about the model. After all, the model and supporting documentation are already available online. Rather, the paper describes the reasoning behind publishing a model open-source, and the process for doing so in this case.

This is the first model released as part of the Open Source Value Project, which tries to convince decision-makers that cost-effectiveness models are worth paying attention to. That is, it’s aimed at the US market, where models are largely ignored. The authors argue that models need to be flexible to be valuable into the future and that, to achieve this, four steps should be followed in the development: 1) release the initial model, 2) invite feedback, 3) convene an expert panel to determine actions in light of the feedback, and 4) revise the model. Then, repeat as necessary. Alongside this, people with the requisite technical skills (i.e. knowing how to use R, C++, and GitHub) can proffer changes to the model whenever they like. This paper was written after step 3 had been completed, and the authors report receiving 159 comments on their model.

The model itself (which you can have a play with here) is an individual patient simulation, which is set-up to evaluate a variety of treatment scenarios. It estimates costs and (mapped) QALYs and can be used to conduct cost-effectiveness analysis or multi-criteria decision analysis. The model was designed to be able to run 32 different model structures based on different assumptions about treatment pathways and outcomes, meaning that the authors could evaluate structural uncertainties (which is a rare feat). A variety of approaches were used to validate the model.

The authors identify several challenges that they experienced in the process, including difficulties in communication between stakeholders and the large amount of time needed to develop, test, and describe a model of this sophistication. I would imagine that, compared with most decision models, the amount of work underlying this paper is staggering. Whether or not that work is worthwhile depends on whether researchers and policymakers make us of the model. The authors have made it as easy as possible for stakeholders to engage with and build on their work, so they should be hopeful that it will bear fruit.

EQ-5D-Y-5L: developing a revised EQ-5D-Y with increased response categories. Quality of Life Research [PubMed] Published 9th February 2019

The EQ-5D-Y has been a slow burner. It’s been around 10 years since it first came on the scene, but we’ve been without a value set and – with the introduction of the EQ-5D-5L – the questionnaire has lost some comparability with its adult equivalent. But the EQ-5D-Y has almost caught-up, and this study describes part of how that’s been achieved.

The reason to develop a 5L version for the EQ-5D-Y is the same as for the adult version – to reduce ceiling effects and improve sensitivity. A selection of possible descriptors was identified through a review of the literature. Focus groups were conducted with children between 8 and 15 years of age in Germany, Spain, Sweden, and the UK in order to identify labels that can be understood by young people. Specifically, the researchers wanted to know the words used by children and adolescents to describe the quantity or intensity of health problems. Participants ranked the labels according to severity and specified which labels they didn’t like. Transcripts were analysed using thematic content analysis. Next, individual interviews were conducted with 255 participants across the four countries, which involved sorting and response scaling tasks. Younger children used a smiley scale. At this stage, both 4L and 5L versions were being considered. In a second phase of the research, cognitive interviews were used to test for comprehensibility and feasibility.

A 5-level version was preferred by most, and 5L labels were identified in each language. The English version used terms like ‘a little bit’, ‘a lot’, and ‘really’. There’s plenty more research to be done on the EQ-5D-Y-5L, including psychometric testing, but I’d expect it to be coming to studies near you very soon. One of the key takeaways from this study, and something that I’ve been seeing more in research in recent years, is that kids are smart. The authors make this point clear, particulary with respect to the response scaling tasks that were conducted with children as young as 8. Decision-making criteria and frameworks that relate to children should be based on children’s preferences and ideas.

Credits