Weekend effect redux

The ‘weekend effect’ has continued to make headlines since we last posted about it. Last week an open letter to beleaguered Secretary of State for Health Jeremy Hunt was published in The Guardian by a number of prominent scientists and clinicians calling for an inquiry into the seven day NHS policy and the evidence behind it. And on Monday a further article was published in The Guardian that reported on the findings of a new study in the British Journal of Psychiatry in which it was reported that there was no increase in the risk of mortality for mental health patients in England. Rashmi Patel, the lead author of the study, was quoted as saying that in relation to increased mortality associated with weekend admission their study had shown ‘no significant difference’ and,

Our study does not support the need to have more doctors on duty at the weekends in psychiatric hospitals. In fact, if this means having to reduce the provision of doctors during the week to provide more doctors at the weekend, this could harm patient care.

But, this is not necessarily true, and it goes to show the difficulties with interpreting and translating evidence into effective policy.

I think part of the problem lies with the mindset of there either is or there isn’t a weekend effect. Perhaps this dichotomy has been ingrained into our psyches by hypothesis testing and p-values. But, it’s a bad way to think about it; care does differ between the weekend and weekdays therefore it is quite plausible that care quality differs as well. I don’t think many people believe in what we’ll call strong weekendism, which might be described as there being no patient who would experience a different overall health outcome if they are admitted at the weekend or on a weekday. However, some people may take the weak weekendism position, which might state that no patient who was admitted on a weekend and who died would have survived had they been admitted on a weekday. However, only the strong weekendism position necessarily supports a conclusion that the 7-day NHS policy is unwarranted. Thus, the aforementioned open letter to Jeremy Hunt seems to take too strong a line. However I think both the strong and weak positions are too strong, the most plausible position in my view is that care quality is worse at the weekend. It’s just a question of how much.

Once the question is reframed about the magnitude of the effect rather than its existence, we can start debating whether a policy to remedy care quality differences is worth it. Contrary to Rashmi Patel’s claims above, their study may or may not support increasing clinical staff provision at the weekend. The evidence we have previously covered on this blog provides preliminary evidence that it is highly unlikely that the 7-day NHS policy will be cost-effective by any standard measure. This should be the focus of the debate, not whether a weekend effect exists or not.

Credits

Advertisements

The economics of a 7-day NHS

The recently delivered Queen’s speech set out the government’s plan for “a 7-day NHS”. This vision is a reaction to alarming statistics that mortality rates are increased by 11% for patients admitted to hospital on a Saturday, and 16% if admitted on a Sunday, compared to patients admitted during the week. In a recent paper, I (along with my co-authors) examine the evidence base being used to support this policy move in more detail, and estimate the economic consequences in terms of the potential costs and benefits. The paper focuses on emergency hospital admissions, as this is the area in which the majority of these deaths occur and has been the focus of much of the policy debate.

The evidence base for seven day services

The highly quoted figure of a 16% increase in the risk of mortality is in fact a relative risk, which we all know too well can be misleading. When interpreting risk statistics the key piece of information is the baseline level of risk; figures which are omitted from the case for seven-day services. The most recent figures from England put the elevated mortality risk experienced by patients admitted to hospital in an emergency during the weekend at 0.3 percentage points. Whilst by no means trivial, it is doubtful that this alternative interpretation of the statistics would have summoned quite the same passion for a reorganisation of the entire English healthcare system.

The classic confusion between correlation and causation is the next mistake made when interpreting the ‘weekend effect’ literature. The association between reduced staffing levels in hospitals at weekends and elevated mortality has been cited as the root of the problem, despite a lack of causal evidence to this effect. In spite of this absence of supportive evidence, making routine services available seven days a week has been declared as the solution to tackling the observed weekend effect. The crucial question then, is what are the likely costs and benefits of such service extensions?

“it’s about saving lives”

As economists we are familiar with the concept of opportunity cost, yet sadly it appears that politicians and policy makers have yet to grasp this key notion. Regardless of whether seven-day services are funded through a redistribution of current NHS budgets or an injection of new cash, this decision implicitly diverts potential resources away from patients admitted during the week. The average daily volume of patients admitted to hospital in an emergency is significantly higher on weekdays than during weekends. This means that staff would be diverted away from working at times of high patient volumes to times when there are fewer patients needing treatment. Yet these patients from whom resources are diverted away are never mentioned in arguments of fairness or equity. If, as the government suggest, staffing levels really are the key to reducing mortality, then the introduction of seven-day services may well narrow the gap between weekday and weekend mortality rates. However, it could easily do so by causing the weekday death rate to rise.

Potential benefits and costs of seven-day services

As healthcare policies such as seven-day services are funded from the same NHS budget as new treatments, they should be subject to the same cost-effectiveness evaluation as technologies seeking NICE approval. This requires rigorous evaluation of hard evidence, something seemingly neglected in favour of headline-hitting policy promises. In the paper we use the available evidence, albeit somewhat rudimentary, on the costs and benefits of introducing seven-day services in this setting to assess whether the policy change would likely pass a NICE assessment. We do so under the most optimistic assumption that this service change has the potential to completely eradicate the weekend effect.

Using methods described in detail in the paper, we estimate that reducing the mortality rate experienced by patients admitted in an emergency at the weekends to that observed during the week would result in an annual reduction of between 4,355 and 5,353 deaths occurring nationally (ceteris paribus, of course). This translates into a potential health gain of 29,727 – 36,539 QALYs per year if all of these deaths could be averted. Using the NICE threshold of £20,000 per QALY, the NHS should spend no more than £595m – £731m to achieve a health gain of this size.

Whilst the potential benefits of extending services appear large, they must be compared with the additional costs of doing so. Although caution was emphasised when producing the figures, the best available estimates of the costs of implementing seven-day services are those published by the NHS Seven Days a Week Forum. They estimate this to be 1.5% to 2% of total hospital income, equivalent to a 5% to 6% increase in the cost of emergency admissions. This translates to an annual cost of between £1.07bn and £1.43bn, exceeding our estimates of the maximum amount that the NHS should spend to eradicate the weekend effect by a factor of 1.5 to 2.4, or between £339m and £831m. To make matters worse, all of these calculations take place under the rather optimistic assumption that benefits to patients admitted at the weekend could be achieved without any detrimental effect on outcomes for those admitted during the week.

The way forward

Although alarming, the statistics on elevated weekend mortality are insufficient by themselves to justify a policy change towards extending normal hours of operation into the weekend. There is as yet no clear evidence: that seven-day working will, in isolation, reduce the weekend death rate; that lower weekend mortality rates can be achieved without increasing weekday death rates; or that such reorganisation is cost-effective.

A move towards a fully operational NHS service seven days a week has the potential to have impacts beyond reducing mortality, but these must be evidenced if the policy is to be supported. Mere suggestions that it may reduce factors such as readmission rates and hospital length of stay are not enough to justify a policy change, just as the verbal reassurance of a drug manufacturer that their product was able to cure cancer would not alone secure them NICE approval. Rigorous evidence and evaluation is needed in the policy sphere if we are truly to get the best use from our limited NHS resources. Evaluations of the implementation of seven-day services in the thirteen early adopters should be performed before national implementation is considered, just as any potential new treatment would be trialled before approval.

Disclaimer: The views and opinions expressed are those of the author and do not necessarily reflect those of the HS&DR programme, NIHR, NHS or the Department of Health.

Sofosbuvir: a fork in the road for NICE?

NICE recently completed their appraisal of the hepatitis C drug sofosbuvir. However, as has been reported in the media, NHS England will not be complying with the guidance within the normal time period.

The cost of a 24 week course of sofosbuvir is almost £70,000. Around 160,000 people are chronically infected with the hepatitis C virus in England, so that adds up to a fair chunk of the NHS budget. Yet the drug does appear to be cost-effective. ICERs differ for different patient groups, but for most scenarios the ICER is below £30,000 per QALY. In the NICE documentation, a number of reasons are listed for NHS England’s decision. But what they ultimately boil down to – it seems – is affordability.

The problem is that NICE doesn’t account for affordability in its guidance. One need only consider that the threshold has remained unchanged for over a decade to see that this is true. How to solve this problem really depends on what we believe the job of NICE should be. Should it be NICE’s job to consider what should and shouldn’t be purchased within the existing health budget? Or, rather, should it be NICE’s job simply to figure out what is ‘worth it’ to society, regardless of affordability? This isn’t the first time that an NHS organisation has appealed against a NICE decision in some way. Surely, it won’t be the last. These instances represent a failure in the system, not least on grounds of accountability for reasonableness. Here I’d like to suggest that NICE has 3 options for dealing with this problem; one easy, one hard and one harder.

The easy option

The simplest option involves the fewest changes to the NICE process. Indeed, it would involve doing pretty much what it does now, only with slightly different (and more transparent) reasoning. In this scenario NICE would explicitly ignore the problem of affordability. Its remit would cease to be the consideration of optimality on a national level and it would ignore the budget constraint. NICE’s remit would become figuring out which health technologies are ‘worth it’; i.e. would the public be willing to purchase a given technology with a given health benefit at a given cost. To some extent, therefore, NICE would become a threshold-setter. The threshold should be based on some definition of a social value of a QALY. This is the easy option for NICE as setting the threshold would be the only additional task to what they currently do. Its threshold might not change all that much, or may be a little higher.

However, even if NICE denies responsibility, clearly someone does need to take account of affordability. Given the events associated with sofosbuvir it seems that this could become the work of NHS England. NHS England could use a threshold based on the budget and current QALY-productivity in the NHS. One might expect NHS England to be in a better position to identify the local evidence necessary to determine appropriate thresholds, which would likely be much lower than NICE’s. It would also be responsible for disinvestment decisions. Given the nationwide remit of NHS England, this would still prevent postcode lotteries. The implication here, of course, is that NICE and NHS England might use different thresholds. Any number of decision rules could be used to determine the result for technologies falling between the two. Maybe this is where considerations for innovation or non-health-related equity concerns belong. It seems probable to me that NICE’s threshold would be higher than NHS England’s, in which case NICE would effectively be advising increases in the health budget. This is something that I quite like the sound of.

The hard option

Personally, I believe that NICE’s failure to justify their threshold(s) is quite a serious failing and undermines the enterprise. The hard option will involve them defining it properly, informed by current levels of QALY-productivity in the NHS. Thus properly adopting a position as a threshold-searcher, and doing the job prescribed to NHS England in the ‘easy option’. NICE guidance would therefore be informed by the current health budget and affordability, and therefore must include guidance on disinvestment. The first stage of this work has already been done. The disinvestment guidance would be the hard part. This argument has already been much discussed, and seems to be what many economists support.

I don’t find this argument entirely compelling, at least not as a solution to the affordability problem. To solve this issue NICE would need to regularly review the current threshold and revise it in light of current productivity and the prevailing health budget. It has no experience of doing this. I believe the task could be more effectively carried out by commissioning organisations (such as NHS England), who are in a better position to oversee the collection of the appropriate data and would have a public responsibility to do so. It might also be politically useful if decisions about affordability were made independently of decisions about value.

The harder option

The harder option is for there to be a paradigm shift in the way NICE – and health economics more generally – operates. It could involve programme budgeting and marginal analysis, or the Birch and Gafni approach. This might just be the best option, but it seems unlikely to happen nationally any time soon.

It’s possible that more cost-effective but unaffordable drugs are in the pipeline. Failure to address the affordability problem soon could seriously undermine NICE.

DOI: 10.6084/m9.figshare.1291123