Alastair Canaway’s journal round-up for 27th November 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Elevated mortality among weekend hospital admissions is not associated with adoption of seven day clinical standards. Emergency Medicine Journal [PubMedPublished 8th November 2017

Our esteemed colleagues in Manchester brought more evidence to the seven-day NHS debate (debacle?). Patients who are admitted to hospital in an emergency at weekends have higher mortality rates than those during the week. Despite what our Secretary of State will have you believe, there is an increasing body of evidence suggesting that once case-mix is adequately adjusted for, the ‘weekend effect’ becomes negligible. This paper takes a slightly different angle for examining the same phenomenon. It harnesses the introduction of four priority clinical standards in England, which aim to reduce the number of deaths associated with the weekend effect. These are time to first consultant review; access to diagnostics; access to consultant-directed interventions; and on-going consultant review. The study uses publicly available data on the performance of NHS Trusts in relation to these four priority clinical standards. For the latest financial year (2015/16), Trusts’ weekend effect odds ratios were compared to their achievement against the four clinical standards. Data were available for 123 Trusts. The authors found that adoption of the four clinical standards was not associated with the extent to which mortality was elevated for patients admitted at the weekend. Furthermore, they found no association between the Trusts’ performance against any of the four standards and the magnitude of the weekend effect. The authors offer three reasons as to why this may be the case. First, data quality could be poor, second, it could be that the standards themselves are inadequate for reducing mortality, finally, it could be that the weekend effect in terms of mortality may be the wrong metric by which to judge the benefits of a seven-day service. They note that their previous research demonstrated that the weekend effect is driven by admission volumes at the weekend rather than the number of deaths, so it will not be impacted by care provision, and this is consistent with the findings in this study. The spectre of opportunity cost looms over the implementation of these standards; although no direct harm may arise from the introduction of these standards, resources will be diverted away from potentially more beneficial alternatives, this is a serious concern. The seven-day debate continues.

The effect of level overlap and color coding on attribute non-attendance in discrete choice experiments. Value in Health Published 16th November 2017

I think discrete choice experiments (DCE) are difficult to complete. That may be due to me not being the sharpest knife in the drawer, or it could be due to the nature of DCEs, or a bit of both. For this reason, I like best-worst scaling (BWS). BWS aside, DCEs are a common tool used in health economics research to assess and understand preferences. Given the difficulty of DCEs, people often resort to heuristics, that is, respondents often simplify choice tasks by taking shortcuts, e.g. ignoring one or more attribute (attribute non-attendance) or always selecting the option with the highest level of a certain attribute. This has downstream consequences leading to bias within preference estimates. Furthermore, difficulty with comprehension leads to high attrition rates. This RCT sought to examine whether participant dropout and attribute non-attendance could be reduced through two methods: level overlap, and colour coding. Level overlap refers to the DCE design whereby in each choice task a certain number of attributes are presented with the same level; in different choice tasks different attributes are overlapped. The idea of this is to prevent dominant attribute strategies whereby participants always choose the option with the highest level of one specific attribute and forces them to evaluate all attributes. The second method involves colour coding and the provision of other visual cues to reduce task complexity, e.g. colour coding levels to make it easy to see which levels are equal. There were five trial arms. The control arm featured no colour coding and no attribute overlap. The other four arms featured either colour coding (two different types were tested), attribute overlap, or a combination of them both. A nationally (Dutch) representative sample in relation to age, gender, education and geographic region were recruited online. In total 3394 respondents were recruited and each arm contained over 500 respondents. Familiarisation and warm-up questions were followed by 21 pairwise choice tasks in a randomised order. For the control arm (no overlap, no colour coding) 13.9% dropped out whilst only attending to on average 2.1 out of the five attributes. Colour coding reduced this to 9.6% with 2.8 attributes being attended. Combining level overlap with intensity colour coding reduced drop out further to 7.2% whilst increasing attribute attendance to four out of five. Thus, the combination of level overlap and colour coding nearly halved the dropout and doubled the attribute attendance within the DCE task. An additional, and perhaps most important benefit of the improvement in attribute attendance is that it reduces the need to model for potential attribute non-attendance post-hoc. Given the difficult of DCE completion, it seems colour coding in combination with level overlap should be implored for future DCE tasks.

Evidence on the longitudinal construct validity of major generic and utility measures of health-related quality of life in teens with depression. Quality of Life Research [PubMed] Published 17th November 2017

There appears to be increasing recognition of the prevalence and seriousness of youth mental health problems. Nearly 20% of young people will suffer depression during their adolescent years. To facilitate cost-utility analysis it is necessary to have a measure of preference based health-related quality of life (HRQL). However, there are few measures designed for use in adolescents. This study sought to examine various existing HRQL measures in relation to their responsiveness for the evaluation of interventions targeting depression in young people. This builds on previous work conducted by Brazier et al that found the EQ-5D and SF-6D performed adequately for depression in adults. In total 392 adolescents aged between 13 and 17 years joined the study, 376 of these completed follow up assessments. Assessments were taken at baseline and 12 weeks. The justification for 12 weeks is that it represented the modal time to clinical change. The following utility instruments were included: the HUI suite, the EQ-5D-3L, Quality of Well-Being Scale (QWB), and the SF-6D (derived from SF-36). Other non-preference based HRQL measures were also included: disease-specific ratings and scales, and the PedsQL 4.0. All (yes, you read that correctly) measures were found to be responsive to change in depression symptomology over the 12-week follow up period and each of the multi-attribute utility instruments was able to detect clinically meaningful change. In terms of comparing the utility instruments, the HUI-3, the QWB and the SF-6D were the most responsive whilst the EQ-5D-3L was the least responsive. In summary, any of the utility instruments could be used. One area of disappointment for me was that the CHU-9D was not included within this study – it’s one of the few instruments that has been developed by and for children and would have very much been a worthy addition. Regardless, this is an informative study for those of us working within the youth mental health sphere.

Credits

Weekend effect redux

The ‘weekend effect’ has continued to make headlines since we last posted about it. Last week an open letter to beleaguered Secretary of State for Health Jeremy Hunt was published in The Guardian by a number of prominent scientists and clinicians calling for an inquiry into the seven day NHS policy and the evidence behind it. And on Monday a further article was published in The Guardian that reported on the findings of a new study in the British Journal of Psychiatry in which it was reported that there was no increase in the risk of mortality for mental health patients in England. Rashmi Patel, the lead author of the study, was quoted as saying that in relation to increased mortality associated with weekend admission their study had shown ‘no significant difference’ and,

Our study does not support the need to have more doctors on duty at the weekends in psychiatric hospitals. In fact, if this means having to reduce the provision of doctors during the week to provide more doctors at the weekend, this could harm patient care.

But, this is not necessarily true, and it goes to show the difficulties with interpreting and translating evidence into effective policy.

I think part of the problem lies with the mindset of there either is or there isn’t a weekend effect. Perhaps this dichotomy has been ingrained into our psyches by hypothesis testing and p-values. But, it’s a bad way to think about it; care does differ between the weekend and weekdays therefore it is quite plausible that care quality differs as well. I don’t think many people believe in what we’ll call strong weekendism, which might be described as there being no patient who would experience a different overall health outcome if they are admitted at the weekend or on a weekday. However, some people may take the weak weekendism position, which might state that no patient who was admitted on a weekend and who died would have survived had they been admitted on a weekday. However, only the strong weekendism position necessarily supports a conclusion that the 7-day NHS policy is unwarranted. Thus, the aforementioned open letter to Jeremy Hunt seems to take too strong a line. However I think both the strong and weak positions are too strong, the most plausible position in my view is that care quality is worse at the weekend. It’s just a question of how much.

Once the question is reframed about the magnitude of the effect rather than its existence, we can start debating whether a policy to remedy care quality differences is worth it. Contrary to Rashmi Patel’s claims above, their study may or may not support increasing clinical staff provision at the weekend. The evidence we have previously covered on this blog provides preliminary evidence that it is highly unlikely that the 7-day NHS policy will be cost-effective by any standard measure. This should be the focus of the debate, not whether a weekend effect exists or not.

Credits

The economics of a 7-day NHS

The recently delivered Queen’s speech set out the government’s plan for “a 7-day NHS”. This vision is a reaction to alarming statistics that mortality rates are increased by 11% for patients admitted to hospital on a Saturday, and 16% if admitted on a Sunday, compared to patients admitted during the week. In a recent paper, I (along with my co-authors) examine the evidence base being used to support this policy move in more detail, and estimate the economic consequences in terms of the potential costs and benefits. The paper focuses on emergency hospital admissions, as this is the area in which the majority of these deaths occur and has been the focus of much of the policy debate.

The evidence base for seven day services

The highly quoted figure of a 16% increase in the risk of mortality is in fact a relative risk, which we all know too well can be misleading. When interpreting risk statistics the key piece of information is the baseline level of risk; figures which are omitted from the case for seven-day services. The most recent figures from England put the elevated mortality risk experienced by patients admitted to hospital in an emergency during the weekend at 0.3 percentage points. Whilst by no means trivial, it is doubtful that this alternative interpretation of the statistics would have summoned quite the same passion for a reorganisation of the entire English healthcare system.

The classic confusion between correlation and causation is the next mistake made when interpreting the ‘weekend effect’ literature. The association between reduced staffing levels in hospitals at weekends and elevated mortality has been cited as the root of the problem, despite a lack of causal evidence to this effect. In spite of this absence of supportive evidence, making routine services available seven days a week has been declared as the solution to tackling the observed weekend effect. The crucial question then, is what are the likely costs and benefits of such service extensions?

“it’s about saving lives”

As economists we are familiar with the concept of opportunity cost, yet sadly it appears that politicians and policy makers have yet to grasp this key notion. Regardless of whether seven-day services are funded through a redistribution of current NHS budgets or an injection of new cash, this decision implicitly diverts potential resources away from patients admitted during the week. The average daily volume of patients admitted to hospital in an emergency is significantly higher on weekdays than during weekends. This means that staff would be diverted away from working at times of high patient volumes to times when there are fewer patients needing treatment. Yet these patients from whom resources are diverted away are never mentioned in arguments of fairness or equity. If, as the government suggest, staffing levels really are the key to reducing mortality, then the introduction of seven-day services may well narrow the gap between weekday and weekend mortality rates. However, it could easily do so by causing the weekday death rate to rise.

Potential benefits and costs of seven-day services

As healthcare policies such as seven-day services are funded from the same NHS budget as new treatments, they should be subject to the same cost-effectiveness evaluation as technologies seeking NICE approval. This requires rigorous evaluation of hard evidence, something seemingly neglected in favour of headline-hitting policy promises. In the paper we use the available evidence, albeit somewhat rudimentary, on the costs and benefits of introducing seven-day services in this setting to assess whether the policy change would likely pass a NICE assessment. We do so under the most optimistic assumption that this service change has the potential to completely eradicate the weekend effect.

Using methods described in detail in the paper, we estimate that reducing the mortality rate experienced by patients admitted in an emergency at the weekends to that observed during the week would result in an annual reduction of between 4,355 and 5,353 deaths occurring nationally (ceteris paribus, of course). This translates into a potential health gain of 29,727 – 36,539 QALYs per year if all of these deaths could be averted. Using the NICE threshold of £20,000 per QALY, the NHS should spend no more than £595m – £731m to achieve a health gain of this size.

Whilst the potential benefits of extending services appear large, they must be compared with the additional costs of doing so. Although caution was emphasised when producing the figures, the best available estimates of the costs of implementing seven-day services are those published by the NHS Seven Days a Week Forum. They estimate this to be 1.5% to 2% of total hospital income, equivalent to a 5% to 6% increase in the cost of emergency admissions. This translates to an annual cost of between £1.07bn and £1.43bn, exceeding our estimates of the maximum amount that the NHS should spend to eradicate the weekend effect by a factor of 1.5 to 2.4, or between £339m and £831m. To make matters worse, all of these calculations take place under the rather optimistic assumption that benefits to patients admitted at the weekend could be achieved without any detrimental effect on outcomes for those admitted during the week.

The way forward

Although alarming, the statistics on elevated weekend mortality are insufficient by themselves to justify a policy change towards extending normal hours of operation into the weekend. There is as yet no clear evidence: that seven-day working will, in isolation, reduce the weekend death rate; that lower weekend mortality rates can be achieved without increasing weekday death rates; or that such reorganisation is cost-effective.

A move towards a fully operational NHS service seven days a week has the potential to have impacts beyond reducing mortality, but these must be evidenced if the policy is to be supported. Mere suggestions that it may reduce factors such as readmission rates and hospital length of stay are not enough to justify a policy change, just as the verbal reassurance of a drug manufacturer that their product was able to cure cancer would not alone secure them NICE approval. Rigorous evidence and evaluation is needed in the policy sphere if we are truly to get the best use from our limited NHS resources. Evaluations of the implementation of seven-day services in the thirteen early adopters should be performed before national implementation is considered, just as any potential new treatment would be trialled before approval.

Disclaimer: The views and opinions expressed are those of the author and do not necessarily reflect those of the HS&DR programme, NIHR, NHS or the Department of Health.