# Category Archives: Health Statistics and Econometrics

data for econometric analysis; categorical data methods; count data; duration analysis; econometric evaluation by non-experimental methods; econometric evaluation with randomized experiments; econometrics in technology assess- ment; macro panels; models of health care costs; models for risk adjustment; panel data methods; productivity analysis; simulation methods and mixture models; spatial econometrics.

## The Lucas critique and hospital quality

In 1976, Robert Lucas wrote a paper that articulated a common criticism of macroeconomic policy-making based on historical data. The essence of the critique was that, since the parameters of macroeconometric models were not structural, they were liable to change when other aspects of the system changed. A policy change could alter the parameters of the model; invalidating conclusions about the effects of that policy change. As one example, Lucas discussed the effect of income on consumption: a consumer, being aware of a policy change that will affect her income, will adjust her consumption with the expectation of the policy change. Thus, consumption decisions will change and will not necessarily reflect the historical relationship between income and consumption.

Lucas accepted that in the short run the estimated relationships may hold but showed that in the long run these estimated relationships were invalid. As a remedy he suggested that the focus should be on microfoundations; identifying the structural factors that determine individual decisions. But, how does this relate to hospital quality?

Estimating hospital quality is an important task for policy-makers although it is fraught with difficulty and controversy. Hospital quality may be examined in two different ways: as the effect of the hospital on the clinical outcomes of its patients (when compared to the average hospital), which we can call ‘outcomes quality’, or how well a hospital meets clinical guidelines and performs mandated tests or procedures, which we can refer to as ‘process quality’. When we conduct research into, for example, the effect of hospital volume on patient outcomes, what we are trying to uncover is how volume affects hospital outcomes quality. A policy maker wants to know how she can influence hospital quality by changing certain aspects of hospital organisation.

Recent evidence has suggested that increased competition between hospitals leads to an increase in outcomes quality, although only under fixed prices (for a recent, interesting blog on the topic including links to papers, see here). One paper showed that in areas where patients had greater choice, mortality rates and length of stay were, on average, less. This suggests that hospitals are improving in order to attract more patients. But, and this is where the Lucas critique enters, publishing information on quality will affect healthcare consumers’ decisions about which hospital to go to. Evidence is limited on the causes and correlates of hospital quality; if the higher quality hospital appears to be of high quality, its casemix, patient volume, and other variables may change. And these are the very variables that may cause the difference in hospital outcomes quality.

Studies of quality may take this change of casemix into account using appropriate controls in their analysis. In this case we may be confident that the results will hold in the short run. But, as individual preferences change, and healthcare consumers and hospitals adapt, the parameters of the model may change – they are not invariant to our policy.

Contributing to the problem, the commonly published hospital quality statistics may not be reliable. Even with casemix adjustment, the typically relied-upon measures, such as the standardised mortality ratio (SMR), may be inadequate. Mohammed et al. (2009) assessed the case mix adjustment commonly used to determine the SMR and found that, due to differences in admissions policies and coding, the effects of different variables in the casemix adjustment differed between hospitals, leading them to conclude:

“Claims that variations in hospital standardised mortality ratios from Dr Foster Unit reflect differences in quality of care are less than credible.”1

This also adds weight to the Lucas critique; these differing admissions policies will themselves be affected by a change in patient casemix that may occur due to increased competition. This would affect the validity of the casemix adjustment used in studies of hospital quality.

This is not a criticism of the validity of hospital quality studies, rather an emphasis on the importance of microfoundations. Unless we understand the factors that determine hospital quality and patient choice we cannot be sure of the long term effects of policies that, for example, affect hospital competition.

1Dr Foster Unit are one of the key providers of hospital quality stats in the UK

## To ‘y’ or not to ‘y’: Dichotomous outcomes and endogenous regressors

Empirical research into health and health related outcomes is characterised by a predominance of binary outcomes. One of the most popular outcomes (in a statistical sense) is mortality, for example. This type of outcome warrants its own type of model that allows the outcome to be observed with probability p – the outcome is a draw from a Bernoulli distribution with this probability; is allowed to vary across individuals. One distinction arises between the economics literature and the medical literature in that the linear probability model (LPM) is fairly popular in the former whereas it is never seen in the latter. The LPM is simply OLS estimation of the binary outcome on the regressors and is popular due to its easily interpretable marginal effects (equal to the estimated coefficient) and ease with which other procedures such as instrumental variables estimation can be used. However, the LPM may predict probabilities outside of the zero to one range and may even be inconsistent in many cases (see here). The probit model is also popular in the economics literature but is likewise rarely seen in medical studies. It is the logit which is ubiquitous in these analyses. But, in the medical literature, studies don’t usually try to address endogeneity whereas in economics we often do. Usually instrumental variables are employed to tackle this problem, but how do we use them in logit models?

In a linear model the two stage least squares (2SLS) estimator can be used. The endogenous variables are regressed on all the exogenous variables including the instruments, then the predicted value of the endogenous variable is used in place of its actual value in a regression. But in a model where the outcome is a nonlinear function of the regressors, such as a logit, this method would be inconsistent. To see why, note that we are trying to estimate a model that assumes:

 $E(y|\textbf{x},\textbf{z},c)=m(\textbf{x}_1'\boldsymbol{\beta}+\textbf{z}'\gamma+c)$ (1)

Where x is assumed to be exogenous of which $x_1$ is a subset, c is unobserved and z is allowed to be correlated with c so that it is potentially endogenous. We model:

$\textbf{z}=\textbf{x}'\boldsymbol{\Pi}+\textbf{v}$

And,

$c=\textbf{v}'\boldsymbol{\rho}+e$

This issue is that if ρ≠0 then z is endogenous. For our estimates to be consistent we essentially require the conditional mean in (1) to be correctly specified. We have two options, we can estimate $\hat{\textbf{z}}$ and substitute it for z in (1) or we can eliminate c. In the latter case, assuming $E(e)=0$, we can rewrite (1) as:

 $E(y|\textbf{x},\textbf{z},\textbf{v})=m(\textbf{x}_1'\boldsymbol{\beta}+\textbf{z}'\gamma+\textbf{v}'\boldsymbol{\rho})$ (2)

But, we do not observe v, however, we can consistently estimate it as $\hat{\textbf{v}}=z-\textbf{x}'\hat{\boldsymbol{\Pi}}$ and include these values in our regression. This method is known as two stage residual inclusion (2SRI).

Our other method however is inconsistent; we can use estimates of $\hat{\textbf{z}}$ in place of z but this does not eliminate c since, even though $E(c)=0$, the expectation does not ‘pass through’ the nonlinear function m(.).

A further useful feature of this comes from the fact that exogeneity of z only happens when ρ=0. We can test this empirically when we estimate (2). This is equivalent to a Hausman test for the exogeneity of z.

The standard errors won’t be correct when estimating (2). Calculating the correct standard errors is not too difficult. But, often in health econometric applications, we want to adjust for clustering within hospitals or regions. To accommodate this into our standard errors makes the calculation considerably more difficult, if not intractable. In this case bootstrapping is the preferred solution.

I have noted in previous posts that endogeneity could be a serious issue in health econometrics, particularly when these types of studies are used to inform healthcare policy. Clearly there are methods for dealing with this, though having a suitable methodology is only the first hurdle. The next one is convincing non-economists why you are using it.

See more

2SRI in health econometrics is discussed in this paper. For a more thorough discussion see Wooldridge Chapter 12.

## The curse of endogeneity in the clinical literature

Endogeneity is everywhere. There is always a reason to assume that there is some endogeneity in a model; sometimes it can’t be totally eliminated and we must just reduce it to acceptable levels. Health economists produce research that often is relevant for both economics and clinical journals, but often the requirements of these two types of journal differ by quite a lot. One way they differ is that the clinical literature and biostatisticians don’t generally care about endogeneity, which is probably the exact opposite opinion of economics and econometricians. When it comes to making policy decisions, not being aware of the effects of endogeneity may have disastrous consequences. Here’s an example why.

Patient and procedure volume has been shown to be inversely correlated with clinical outcomes such as mortality. This suggests that big hospitals are good. But, what about causality? Is there any? And, if so, in which direction does it run?

The hypothesis that volume causes better outcomes is called ‘practice makes perfect’ (PMP). This could be due either to ‘learning by doing’ or ‘scale economies’. If PMP were the case then we could identify if a learning by doing mechanism was responsible either by looking at the effects of lagged volume, or by seeing if a clinician who had been at a high volume hospital ‘took’ his skills with him. The competing hypothesis is ‘selective referral’ (SR). Hospitals which have superior outcomes attract more patients which consequently boosts their volume.

In the case of a possibly simultaneous mechanism like this we resort to instrumental variables. A common instrument for volume in this case exploits the exogenous preference of individuals to go to their nearest hospital. The instrument could then be, at the patient level, the nearest hospital, or at the hospital level, the size of the catchment level.

In the clinical literature this issue of the direction of causality has often been ignored. The association of volume and positive clinical effects in some areas of medicine has led to calls for centralisation of healthcare services. This implicitly assumes that the PMP hypothesis is true, or at least plays a stronger role that SR. But what if the volume-outcome effect is driven more by SR than PMP? Then the sickest patients will all be sent to the new large hospitals which will not cause any effect to outcomes and may even have a negative effect by increasing burden on staff, inefficient use of resources, and making patients travel further among other things.

For many areas of medicine a causal link has been demonstrated between volume and outcome, and it may be shown that both PMP and SR play a role. But this is just a demonstration of the problems of ignoring endogeneity. Causal inference is demonstrated for many healthcare interventions through a randomised experiment – something which health economics could do with more of – but often it is unethical or impractical to perform such an experiment. If we do rely on observational research then economists and econometricians should be trying to communicate these issues.