In 1976, Robert Lucas wrote a paper that articulated a common criticism of macroeconomic policy-making based on historical data. The essence of the critique was that, since the parameters of macroeconometric models were not structural, they were liable to change when other aspects of the system changed. A policy change could alter the parameters of the model; invalidating conclusions about the effects of that policy change. As one example, Lucas discussed the effect of income on consumption: a consumer, being aware of a policy change that will affect her income, will adjust her consumption with the expectation of the policy change. Thus, consumption decisions will change and will not necessarily reflect the historical relationship between income and consumption.
Lucas accepted that in the short run the estimated relationships may hold but showed that in the long run these estimated relationships were invalid. As a remedy he suggested that the focus should be on microfoundations; identifying the structural factors that determine individual decisions. But, how does this relate to hospital quality?
Estimating hospital quality is an important task for policy-makers although it is fraught with difficulty and controversy. Hospital quality may be examined in two different ways: as the effect of the hospital on the clinical outcomes of its patients (when compared to the average hospital), which we can call ‘outcomes quality’, or how well a hospital meets clinical guidelines and performs mandated tests or procedures, which we can refer to as ‘process quality’. When we conduct research into, for example, the effect of hospital volume on patient outcomes, what we are trying to uncover is how volume affects hospital outcomes quality. A policy maker wants to know how she can influence hospital quality by changing certain aspects of hospital organisation.
Recent evidence has suggested that increased competition between hospitals leads to an increase in outcomes quality, although only under fixed prices (for a recent, interesting blog on the topic including links to papers, see here). One paper showed that in areas where patients had greater choice, mortality rates and length of stay were, on average, less. This suggests that hospitals are improving in order to attract more patients. But, and this is where the Lucas critique enters, publishing information on quality will affect healthcare consumers’ decisions about which hospital to go to. Evidence is limited on the causes and correlates of hospital quality; if the higher quality hospital appears to be of high quality, its casemix, patient volume, and other variables may change. And these are the very variables that may cause the difference in hospital outcomes quality.
Studies of quality may take this change of casemix into account using appropriate controls in their analysis. In this case we may be confident that the results will hold in the short run. But, as individual preferences change, and healthcare consumers and hospitals adapt, the parameters of the model may change – they are not invariant to our policy.
Contributing to the problem, the commonly published hospital quality statistics may not be reliable. Even with casemix adjustment, the typically relied-upon measures, such as the standardised mortality ratio (SMR), may be inadequate. Mohammed et al. (2009) assessed the case mix adjustment commonly used to determine the SMR and found that, due to differences in admissions policies and coding, the effects of different variables in the casemix adjustment differed between hospitals, leading them to conclude:
“Claims that variations in hospital standardised mortality ratios from Dr Foster Unit reflect differences in quality of care are less than credible.”1
This also adds weight to the Lucas critique; these differing admissions policies will themselves be affected by a change in patient casemix that may occur due to increased competition. This would affect the validity of the casemix adjustment used in studies of hospital quality.
This is not a criticism of the validity of hospital quality studies, rather an emphasis on the importance of microfoundations. Unless we understand the factors that determine hospital quality and patient choice we cannot be sure of the long term effects of policies that, for example, affect hospital competition.
1Dr Foster Unit are one of the key providers of hospital quality stats in the UK
