James Altunkaya’s journal round-up for 3rd September 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Sensitivity analysis for not-at-random missing data in trial-based cost-effectiveness analysis: a tutorial. PharmacoEconomics [PubMed] [RePEc] Published 20th April 2018

Last month, we highlighted a Bayesian framework for imputing missing data in economic evaluation. The paper dealt with the issue of departure from the ‘Missing at Random’ (MAR) assumption by using a Bayesian approach to specify a plausible missingness model from the results of expert elicitation. This was used to estimate a prior distribution for the unobserved terms in the outcomes model.

For those less comfortable with Bayesian estimation, this month we highlight a tutorial paper from the same authors, outlining an approach to recognise the impact of plausible departures from ‘Missingness at Random’ assumptions on cost-effectiveness results. Given poor adherence to current recommendations for the best practice in handling and reporting missing data, an incremental approach to improving missing data methods in health research may be more realistic. The authors supply accompanying Stata code.

The paper investigates the importance of assuming a degree of ‘informative’ missingness (i.e. ‘Missingness not at Random’) in sensitivity analyses. In a case study, the authors present a range of scenarios which assume a decrement of 5-10% in the quality of life of patients with missing health outcomes, compared to multiple imputation estimates based on observed characteristics under standard ‘Missing at Random’ assumptions. This represents an assumption that, controlling for all observed characteristics used in multiple imputation, those with complete quality of life profiles may have higher quality of life than those with incomplete surveys.

Quality of life decrements were implemented in the control and treatment arm separately, and then jointly, in six scenarios. This aimed to demonstrate the sensitivity of cost-effectiveness judgements to the possibility of a different missingness mechanism in each arm. The authors similarly investigate sensitivity to higher health costs in those with missing data than predicted based on observed characteristics in imputation under ‘Missingness at Random’. Finally, sensitivity to a simultaneous departure from ‘Missingness at Random’ in both health outcomes and health costs is investigated.

The proposed sensitivity analyses provide a useful heuristic to assess what degree of difference between missing and non-missing subjects on unobserved characteristics would be necessary to change cost-effectiveness decisions. The authors admit this framework could appear relatively crude to those comfortable with more advanced missing data approaches such as those outlined in last month’s round-up. However, this approach should appeal to those interested in presenting the magnitude of uncertainty introduced by missing data assumptions, in a way that is easily interpretable to decision makers.

The impact of waiting for intervention on costs and effectiveness: the case of transcatheter aortic valve replacement. The European Journal of Health Economics [PubMed] [RePEc] Published September 2018

This paper appears in print this month and sparked interest as one of comparatively few studies on the cost-effectiveness of waiting lists. Given interest in using constrained optimisation methods in health outcomes research, highlighted in this month’s editorial in Value in Health, there is rightly interest in extending the traditional sphere of economic evaluation from drugs and devices to understanding the trade-offs of investing in a wider range of policy interventions, using a common metric of costs and QALYs. Rachel Meacock’s paper earlier this year did a great job at outlining some of the challenges involved broadening the scope of economic evaluation to more general decisions in health service delivery.

The authors set out to understand the cost-effectiveness of delaying a cardiac treatment (TVAR) using a waiting list of up to 12 months compared to a policy of immediate treatment. The effectiveness of treatment at 3, 6, 9 & 12 months after initial diagnosis, health decrements during waiting, and corresponding health costs during wait time and post-treatment were derived from a small observational study. As treatment is studied in an elderly population, a non-ignorable proportion of patients die whilst waiting for surgery. This translates to lower modelled costs, but also lower quality life years in modelled cohorts where there was any delay from a policy of immediate treatment. The authors conclude that eliminating all waiting time for TVAR would produce population health at a rate of ~€12,500 per QALY gained.

However, based on the modelling presented, the authors lack the ability to make cost-effectiveness judgements of this sort. Waiting lists exist for a reason, chiefly a lack of clinical capacity to treat patients immediately. In taking a decision to treat patients immediately in one disease area, we therefore need some judgement as to whether the health displaced in now untreated patients in another disease area is of greater, less or equal magnitude to that gained by treating TVAR patients immediately. Alternately, modelling should include the cost of acquiring additional clinical capacity (such as theatre space) to treat TVAR patients immediately, so as not to displace other treatments. In such a case, the ICER is likely to be much higher, due to the large cost of new resources needed to reduce waiting times to zero.

Given the data available, a simple improvement to the paper would be to reflect current waiting times (already gathered from observational study) as the ‘standard of care’ arm. As such, the estimated change in quality of life and healthcare resource cost from reducing waiting times to zero from levels observed in current practice could be calculated. This could then be used to calculate the maximum acceptable cost of acquiring additional treatment resources needed to treat patients with no waiting time, given current national willingness-to-pay thresholds.

Admittedly, there remain problems in using the authors’ chosen observational dataset to calculate quality of life and cost outcomes for patients treated at different time periods. Waiting times were prioritised in this ‘real world’ observational study, based on clinical assessment of patients’ treatment need. Thus it is expected that the quality of life lost during a waiting period would be lower for patients treated in the observational study at 12 months, compared to the expected quality of life loss of waiting for the group of patients judged to need immediate treatment. A previous study in cardiac care took on the more manageable task of investigating the cost-effectiveness of different prioritisation strategies for the waiting list, investigating the sensitivity of conclusions to varying a fixed maximum wait-time for the last patient treated.

This study therefore demonstrates some of the difficulties in attempting to make cost-effectiveness judgements about waiting time policy. Given that the cost-effectiveness of reducing waiting times in different disease areas is expected to vary, based on relative importance of waiting for treatment on short and long-term health outcomes and costs, this remains an interesting area for economic evaluation to explore. In the context of the current focus on constrained optimisation techniques across different areas in healthcare (see ISPOR task force), it is likely that extending economic evaluation to evaluate a broader range of decision problems on a common scale will become increasingly important in future.

Understanding and identifying key issues with the involvement of clinicians in the development of decision-analytic model structures: a qualitative study. PharmacoEconomics [PubMed] Published 17th August 2018

This paper gathers evidence from interviews with clinicians and modellers, with the aim to improve the nature of the working relationship between the two fields during model development.

Researchers gathered opinion from a variety of settings, including industry. The main report focusses on evidence from two case studies – one tracking the working relationship between modellers and a single clinical advisor at a UK university, with the second gathering evidence from a UK policy institute – where modellers worked with up to 11 clinical experts per meeting.

Some of the authors’ conclusions are not particularly surprising. Modellers reported difficulty in recruiting clinicians to advise on model structures, and further difficulty in then engaging recruited clinicians to provide relevant advice for the model building process. Specific comments suggested difficulty for some clinical advisors in identifying representative patient experiences, instead diverting modellers’ attention towards rare outlier events.

Study responses suggested currently only 1 or 2 clinicians were typically consulted during model development. The authors recommend involving a larger group of clinicians at this stage of the modelling process, with a more varied range of clinical experience (junior as well as senior clinicians, with some geographical variation). This is intended to help ensure clinical pathways modelled are generalizable. The experience of one clinical collaborator involved in the case study based at a UK university, compared to 11 clinicians at the policy institute studied, perhaps may also illustrate a general problem of inadequate compensation for clinical time within the university system. The authors also advocate the availability of some relevant training for clinicians in decision modelling to help enhance the efficiency of participants’ time during model building. Clinicians sampled were supportive of this view – citing the need for further guidance from modellers on the nature of their expected contribution.

This study ties into the general literature regarding structural uncertainty in decision analytic models. In advocating the early contribution of a larger, more diverse group of clinicians in model development, the authors advocate a degree of alignment between clinical involvement during model structuring, and guidelines for eliciting parameter estimates from clinical experts. Similar problems, however, remain for both fields, in recruiting clinical experts from sufficiently diverse backgrounds to provide a valid sample.

Credits

 

Thesis Thursday: Frank Sandmann

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Frank Sandmann who has a PhD from the London School of Hygiene & Tropical Medicine. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
The true cost of epidemic and outbreak diseases in hospitals
Supervisors
Mark Jit, Sarah Deeny, Julie Robotham, John Edmunds
Repository link
http://researchonline.lshtm.ac.uk/4648208/

Do you refer to the ‘true’ cost because some costs are hidden in this context?

That’s a good observation. Economists use the term “true cost” as a synonym for “opportunity cost”, which can be defined as the net value of the forgone second-best use of a resource. The true value of a hospital bed is therefore determined by its second-best use, which may indeed be less easily observed and less obvious, or somewhat hidden.

In the context of infectious disease outbreaks in hospital, the most visible costs are the direct expenditures on treatments of infected cases and any measures of containment. However, they do not capture the full extent of the “alternative” costs and therefore cannot equal opportunity costs. Slightly less visible are the potential knock-on effects for visitors to the hospital who, unbeknown to them, may get infected and contribute to sustained transmission in the community. Least seen are the externalities borne by patients who have not been admitted so far but who are awaiting admission, and for whom there is no space in hospital yet due to the ongoing outbreak.

In my thesis, I provided a general overview of the historical development of the concept of opportunity costs of resources before I looked in detail at bed-days and the application for hospitals.

How should the opportunity cost of hospital stays be determined?

That depends on for whom you want to determine these costs.

For individual patients, it depends on the very subjective decision of how else they would spend their time instead, and how urgent it is to receive hospital care.

From the perspective of hospital administrators, it is straightforward to calculate the opportunity costs based on the revenues and expenditures of the inpatients, their length of stays, and the existing demand of care from the community. This is quite important because whether there are opportunity costs from forgone admissions will depend on whether there are other patients actually waiting to be admitted, which is somewhat reflected in occupancy rates and of course waiting lists.

Any other decision maker who is acting as an agent on behalf of a collective group or the public should look into the forgone health impact of patients who cannot be admitted when the beds are unavailable to them. In my thesis, I proposed a method for quantifying the opportunity costs of bed-days with the net benefit of the second-best patients forgone, which I illustrated with the example of norovirus-associated gastroenteritis.

How important are differences in methods for costing in the context of gastroenteritis and norovirus?

The results can differ quite substantially when using different costing methods. Norovirus is an ideal illness to illustrate this issue given that otherwise healthy people with gastrointestinal symptoms and no further comorbidities or complications shouldn’t be admitted to hospital in order to minimise the risk of an outbreak. Patients with norovirus are therefore often not the patient group that is benefitting the most from a hospital stay.

In one of the studies of my PhD, I was able to show that the annual burden of norovirus in public hospitals in England amounts to a mean £110 million using conventional costing methods, while the opportunity costs were two-to-three times higher of up to £300 million.

This means that there is the potential for a situation where an intervention is disadvantaged when using conventional methods for costing and ignoring the opportunity costs. When evaluating such an intervention against established decision rules of cost-effectiveness, this may lead to an incorrect decision.

What were some of the key challenges that you encountered in estimating the cost of norovirus to hospitals, and how did you overcome them?

There were at least four key challenges:

First was the number of admissions. Many inpatients with norovirus won’t get recorded as such if they haven’t been laboratory-confirmed. That is why I regressed national inpatient episodes of gastroenteritis against laboratory surveillance reports for ten different gastrointestinal pathogens to estimate the norovirus-attributable proportion.

Second was the number of bed-days used by inpatients that were infected with norovirus during their hospital stay. Using their total length of stay, or some form of propensity matching, suffers from time-dependent biases and overestimates the number of bed-days. Instead, I used a multi-state model and patient-level data from a local hospital.

Third was the bed-days that were left unoccupied for infection control. One of the datasets tracked them mandatorily for acute hospitals during winters, while another surveillance system was voluntary, but recorded outbreaks throughout the year. For a more accurate estimate, I compared both datasets with each other to explore their potential overlap.

Fourth was the forgone health of alternative admissions who had otherwise occupied the beds. I had to make assumptions about the disease progression with and without hospital treatment, for which I used health-state utilities that accounted for age, sex, and the primary medical condition.

If you could have wished for one additional set of data that wasn’t available, what would it have been?

I have been very fortunate to work with a number of colleagues at Public Health England and University College London who provided me with much of the epidemiological data that I needed. My research could have benefitted though from a dataset that tracked the time of infection for a larger patient population and for longer observation periods, and a dataset that included more robust estimates for the health gain from hospital care.

If I could make a wish about the existing datasets on norovirus that I have used, I would wish for a higher rate of reporting given that it became clear from our comparison of datasets that there is a highly-correlated trend, but the number of outbreaks reported and the details of reporting leave room for improvement. Another wish of mine for daily reporting of bed-days during winter became reality only recently; during my PhD, I had to impute missing values that were non-randomly missing at weekends and over the Christmas period. This was changed in winter 2016, and I have recently shown that the mean of our lowest-to-highest imputation scenarios is surprisingly close to the daily number of bed-days recorded since then.

Parts of your thesis are made up of journal articles that you published before submission. Was this always your intention and how did you find the experience?

I always wanted to publish parts of my thesis in separate journal articles as I believe this to be a great chance to reach different audiences. That is because my theoretical research on opportunity costs may be of broader interest than just to those who work on norovirus or bed-days given that my findings are generalisable to other diseases as well as other resources. At the same time, others may be more interested in my results for norovirus, and still others in my application of the various statistical, economic, and mathematical modelling techniques.

After all, I honestly suspect that some people may place a higher value on their next-best alternative use of time than reading my thesis from cover to cover.

Writing up my thoughts early on also helped me refine them, and the peer-review process was a great opportunity to get some additional feedback. It did require good time management skills though to keep coming back to previous studies to address the peer-reviewers’ comments while I was already busy working on the next studies.

All in all, I can recommend others to consider it and, looking back, I’d do it again this way.

Rita Faria’s journal round-up for 18th June 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Objectives, budgets, thresholds, and opportunity costs—a health economics approach: an ISPOR Special Task Force report. Value in Health [PubMedPublished 21st February 2018

The economic evaluation world has been discussing cost-effectiveness thresholds for a while. This paper has been out for a few months, but it slipped under my radar. It explains the relationship between the cost-effectiveness threshold, the budget, opportunity costs and willingness to pay for health. My take-home messages are that we should use cost-effectiveness analysis to inform decisions both for publicly funded and privately funded health care systems. Each system has a budget and a way of raising funds for that budget. The cost-effectiveness threshold should be specific for each health care system, in order to reflect its specific opportunity cost. The budget can change for many reasons. The cost-effectiveness threshold should be adjusted to reflect these changes and hence reflect the opportunity cost. For example, taxpayers can increase their willingness to pay for health through increased taxes for the health care system. We are starting to see this in the UK with the calls to raise taxes to increase the NHS budget. It is worth noting that the NICE threshold may not warrant adjustment upwards since research suggests that it does not reflect the opportunity cost. This is a welcome paper on the topic and a must read, particularly if you’re arguing for the use of cost-effectiveness analysis in settings that traditionally were reluctant to embrace it, such as the US.

Basic versus supplementary health insurance: access to care and the role of cost effectiveness. Journal of Health Economics [RePEc] Published 31st May 2018

Using cost-effectiveness analysis to inform coverage decisions not only for the public but also for the privately funded health care is also a feature of this study by Jan Boone. I’ll admit that the equations are well beyond my level of microeconomics, but the text is good at explaining the insights and the intuition. Boone grapples with the question about how the public and private health care systems should choose which technologies to cover. Boone concludes that, when choosing which technologies to cover, the most cost-effective technologies should be prioritised for funding. That the theory matches the practice is reassuring to an economic evaluator like myself! One of the findings is that cost-effective technologies which are very cheap should not be covered. The rationale being that everyone can afford them. The issue for me is that people may decide not to purchase a highly cost-effective technology which is very cheap. As we know from behaviour economics, people are not rational all the time! Boone also concludes that the inclusion of technologies in the universal basic package should consider the prevalence of the conditions in those people at high risk and with low income. The way that I interpreted this is that it is more cost-effective to include technologies for high-risk low-income people in the universal basic package who would not be able to afford these technologies otherwise, than technologies for high-income people who can afford supplementary insurance. I can’t cover here all the findings and the nuances of the theoretical model. Suffice to say that it is an interesting read, even if you avoid the equations like myself.

Surveying the cost effectiveness of the 20 procedures with the largest public health services waiting lists in Ireland: implications for Ireland’s cost-effectiveness threshold. Value in Health Published 11th June 2018

As we are on the topic of cost-effectiveness thresholds, this is a study on the threshold in Ireland. This study sets out to find out if the current cost-effectiveness threshold is too high given the ICERs of the 20 procedures with the largest waiting lists. The idea is that, if the current cost-effectiveness threshold is correct, the procedures with large and long waiting lists would have an ICER of above the cost-effectiveness threshold. If the procedures have a low ICER, the cost-effectiveness threshold may be set too high. I thought that Figure 1 is excellent in conveying the discordance between ICERs and waiting lists. For example, the ICER for extracapsular extraction of crystalline lens is €10,139/QALY and the waiting list has 10,056 people; the ICER for surgical tooth removal is €195,155/QALY and the waiting list is smaller at 833. This study suggests that, similar to many other countries, there are inefficiencies in the way that the Irish health care system prioritises technologies for funding. The limitation of the study is in the ICERs. Ideally, the relevant ICER compares the procedure with the standard care in Ireland whilst on the waiting list (“no procedure” option). But it is nigh impossible to find ICERs that meet this condition for all procedures. The alternative is to assume that the difference in costs and QALYs is generalisable from the source study to Ireland. It was great to see another study on empirical cost-effectiveness thresholds. Looking forward to knowing what the cost-effectiveness threshold should be to accurately reflect opportunity costs.

Credits