Free to choose?: A comment on Gaynor, Propper, and Seiler (2016)

Free to choose? Reform, choice, and consideration sets in the English National Health Service. M Gaynor, C Propper, and S Seiler. 2016. American Economic Review [RePEc] Forthcoming

The enhancement of patient choice about healthcare provider is a popular target for reform across many European countries, including the UK. In 2006, the government in the UK mandated that patients had to be given the choice of at least five providers when being referred for treatment. Prior to this time the decision lay principally with the referring clinician. The impact of this reform was previously examined in two papers: Gaynor, Moreno-Serra, and Propper (2012) and Cooper et al. (2011). The latter of these attracted some criticism particularly after it was used in support of the controversial Health and Social Care Act (2012). One key aspect of this criticism revolved around the use of mortality from acute myocardial infarction (AMI) as a quality indicator, despite AMI being an emergency condition over which patients have no choice about their treatment hospital. The former of those two papers expands the analysis to consider other outcomes such as all cause death.

In this new paper, examining the same 2006 reform, the authors this time examine coronary artery bypass graft surgery (CABG). CABG is an elective condition thus permitting patient choice. The analysis considers where patients chose to go and on what basis, the effect of choice on patient mortality, and the effect of competition on hospital market share. The authors develop a novel method to analyse consideration sets to compare choices made prior to and after the reform. One of the key findings is that patients respond to signals of quality – in this case hospital mortality rates. And this improved sorting of patients into hospitals with lower mortality rates. However, here the distinction between a quality signal and actual quality is blurred.

It stands to reason that a patient would prefer a hospital with lower apparent mortality rates. But, mortality rates, whether adjusted or unadjusted, have been shown to be poorly correlated with preventable mortality in the NHS. The mortality rates used in this paper are the estimated (OLS) coefficients from a model of in-hospital death regressed on dummy variables for each hospital, thus estimating the crude mortality rate. To address the potential mismatch between mortality rates and the causal effect of a hospital on patient mortality, Gaynor, Propper, and Seiler also use an instrumental variable (IV) estimator for the hospital dummy with patient distance to each hospital as the instrument. This follows the method of Gowrisankaran and Town (1999). Gaynor, Propper, and Seiler state that a Hausman test does not reject the hypothesis that the OLS and IV coefficients are different and so use the OLS crude mortality rate estimates in the primary analysis. Nevertheless they repeat the analysis and show that patient hospital of choice is also associated with the IV estimated mortality rate. But the question still remains as to whether these estimates can be relied upon to demonstrate that the reforms improved mortality risk in the CABG cohort.

Gowrisankaran and Town showed there was little correlation in their study between GLS and IV estimates of hospital quality (see the Figure). Hogan et al. (2015) showed that the association between standardised hospital mortality ratios (SMR) and the proportion of preventable deaths was very weak. And Girling et al. (2012) estimated that if 6% of hospital deaths are preventable then the predictive value of the SMR can be no greater than 9%. However, they suggest that this could rise to 30% if 15% of deaths were preventable. So it seems perhaps surprising that Gaynor, Propper, and Seiler find no evidence of a difference between their OLS and IV estimators. Now, for CABG, the proportion of preventable deaths may be very high, Guru et al. (2008) estimated it to be as high as 32%. But, they also find there to be no correlation between preventable deaths and mortality rates in hospitals. Taken altogether this might suggest a flaw in the analysis of Gaynor, Propper, and Seiler.

Scatterplot of the GLS and IV estimates of hospital quality. (c) Elsevier Science B.V.

When choosing between healthcare providers patients are provided with information about quality. This normally comes in the form of SMRs as we have previously discussed. Gaynor, Propper, and Seiler demonstrate that patients respond to this information. But, as we have argued, these signals are poor with respect to actual quality. Thus the consequences of patients sorting into hospitals in terms of actual deaths avoided is difficult to ascertain. A Hausman test suggests that the OLS and IV results are similar in this study and there is an association between patient choice and the IV estimated quality variable. But many arguments may run counter to these findings: the Hausman test could have low power, the IV estimator may be biased by the large number of moment restrictions, the instruments may not be conditionally independent of the hospitals, the common support between hospitals may not include the highest risk patients, and so forth. This paper successfully demonstrates how patients respond to information in making their choice between hospitals, but whether the reforms reduced mortality remains unanswered in my opinion.

Photo credit: Ramdlon (CC0)

Please leave this field empty

We now have a newsletter!

Sign up to receive updates about the blog and the wider health economics world.

Email Address *

Check your inbox or spam folder to confirm your subscription.