The statistics underlying the arguments around the weekend effect are complicated. Despite over a hundred empirical studies on the topic, and an observed increase in the risk of mortality for weekend admissions in multiple countries, there is still no real consensus on what is going on. We have previously covered the arguments on this blog and suggested that the best explanation for the weekend effect is that healthier patients are less likely to be admitted to hospital at the weekend. Nevertheless, a little knowledge can be a dangerous thing (the motto of the Dunning-Kruger effect), and some people can be very confident about the interpretation of the statistics despite their complicated nature. For example, one consultant nephrologist wrote in a comment on a recent article that those who attribute the weekend effect to differences in admission are becoming ‘the climate change deniers of healthcare’ as they are not taking into account all the risk-adjusted analyses!
It may certainly be the case that there is a reduction in healthcare quality at the weekend. But it is also important for policy makers to understand that it is still possible to observe a weekend effect even with quite comprehensive mortality risk adjustment. The image below links to an app that simulates multiple weekend effect studies from a model where there is no weekend effect but potentially different chances of admission at the weekend and on weekdays. We are assuming that those who turn up to A&E but are not admitted and sent home are the healthiest patients. In the app, you can change the parameters: the proportion of attendances that are admitted on weekends and weekdays, the mortality rate among patients who are admitted, and, crucially, the amount of variation in patient mortality explained (a sort of “R-squared”) by our risk adjustment. It will display crude and adjusted odds ratios as well as a distribution of possible results from similar studies. (Be patient though, simulating lots of large studies seems to take a while on the server!).
As is evident, even with a very high proportion of variance explained, we can still get an odds ratio not equal to one and an observed weekend effect if the proportion of attendances who are admitted differs between weekend and weekday. And, with the very large sample sizes often used for these studies, these will likely appear “statistically significant“. Recent evidence from the UK has suggested that 27% of A&E attendances are admitted at the weekend compared to 30% on a weekday. Even when we can explain 90% of the variation in mortality, we can still get a ‘weekend effect’ with these small differences in propensity for admission. And, if there is any element of publication bias, or the ‘garden of forking paths’ [PDF] we will see lots of statistically significant weekend effect studies published.
When there is a misunderstanding about statistics, one often blames the audience for not understanding, but it is often the case that an idea has just not been explained well enough. I can’t judge whether little web apps will actually help explain concepts like this, but hopefully it’s a step in the right direction.
Credits
[…] produced a simple tool to demonstrate how weekend effect studies might estimate an increased risk of mortality associated with weekend […]
[…] at weekends relative to weekdays may be the foundation of the ‘weekend effect’ as we recently demonstrated. And yet all these different sources of variation produce a singular time series of numbers of […]