Sam Watson’s journal round-up for 21st August 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Multidimensional performance assessment of public sector organisations using dominance criteria. Health Economics [RePEcPublished 18th August 2017

The empirical assessment of the performance or quality of public organisations such as health care providers is an interesting and oft-tackled problem. Despite the development of sophisticated methods in a large and growing literature, public bodies continue to use demonstrably inaccurate or misleading statistics such as the standardised mortality ratio (SMR). Apart from the issue that these statistics may not be very well correlated with underlying quality, organisations may improve on a given measure by sacrificing their performance on another outcome valued by different stakeholders. One example from a few years ago showed how hospital rankings based upon SMRs shifted significantly if one took into account readmission rates and their correlation with SMRs. This paper advances this thinking a step further by considering multiple outcomes potentially valued by stakeholders and using dominance criteria to compare hospitals. A hospital dominates another if it performs at least as well or better across all outcomes. Importantly, correlation between these measures is captured in a multilevel model. I am an advocate of this type of approach, that is, the use of multilevel models to combine information across multiple ‘dimensions’ of quality. Indeed, my only real criticism would be that it doesn’t go far enough! The multivariate normal model used in the paper assumes a linear relationship between outcomes in their conditional distributions. Similarly, an instrumental variable model is also used (using the now routine distance-to-health-facility instrumental variable) that also assumes a linear relationship between outcomes and ‘unobserved heterogeneity’. The complex behaviour of health care providers may well suggest these assumptions do not hold – for example, failing institutions may well show poor performance across the board, while other facilities are able to trade-off outcomes with one another. This would suggest a non-linear relationship. I’m also finding it hard to get my head around the IV model: in particular what the covariance matrix for the whole model is and if correlations are permitted in these models at multiple levels as well. Nevertheless, it’s an interesting take on the performance question, but my faith that decent methods like this will be used in practice continues to wane as organisations such as Dr Foster still dominate quality monitoring.

A simultaneous equation approach to estimating HIV prevalence with nonignorable missing responses. Journal of the American Statistical Association [RePEcPublished August 2017

Non-response is a problem encountered more often than not in survey based data collection. For many public health applications though, surveys are the primary way of determining the prevalence and distribution of disease, knowledge of which is required for effective public health policy. Methods such as multiple imputation can be used in the face of missing data, but this requires an assumption that the data are missing at random. For disease surveys this is unlikely to be true. For example, the stigma around HIV may make many people choose not to respond to an HIV survey, thus leading to a situation where data are missing not at random. This paper tackles the question of estimating HIV prevalence in the face of informative non-response. Most economists are familiar with the Heckman selection model, which is a way of correcting for sample selection bias. The Heckman model is typically estimated or viewed as a control function approach in which the residuals from a selection model are used in a model for the outcome of interest to control for unobserved heterogeneity. An alternative way of representing this model is as copula between a survey response variable and the response variable itself. This representation is more flexible and permits a variety of models for both selection and outcomes. This paper includes spatial effects (given the nature of disease transmission) not only in the selection and outcomes models, but also in the model for the mixing parameter between the two marginal distributions, which allows the degree of informative non-response to differ by location and be correlated over space. The instrumental variable used is the identity of the interviewer since different interviewers are expected to be more or less successful at collecting data independent of the status of the individual being interviewed.

Clustered multistate models with observation level random effects, mover–stayer effects and dynamic covariates: modelling transition intensities and sojourn times in a study of psoriatic arthritis. Journal of the Royal Statistical Society: Series C [ArXiv] Published 25th July 2017

Modelling the progression of disease accurately is important for economic evaluation. A delicate balance between bias and variance should be sought: a model too simple will be wrong for most people, a model too complex will be too uncertain. A huge range of models therefore exists from ‘simple’ decision trees to ‘complex’ patient-level simulations. A popular choice are multistate models, such as Markov models, which provide a convenient framework for examining the evolution of stochastic processes and systems. A common feature of such models is the Markov property, which is that the probability of moving to a given state is independent of what has happened previously. This can be relaxed by adding covariates to model transition properties that capture event history or other salient features. This paper provides a neat example of extending this approach further in the case of arthritis. The development of arthritic damage in a hand joint can be described by a multistate model, but there are obviously multiple joints in one hand. What is more, the outcomes in any one joint are not likely to be independent of one another. This paper describes a multilevel model of transition probabilities for multiple correlated processes along with other extensions like dynamic covariates and different mover-stayer probabilities.

Credits

Free to choose?: A comment on Gaynor, Propper, and Seiler (2016)

Free to choose? Reform, choice, and consideration sets in the English National Health Service. M Gaynor, C Propper, and S Seiler. 2016. American Economic Review [RePEcForthcoming

The enhancement of patient choice about healthcare provider is a popular target for reform across many European countries, including the UK. In 2006, the government in the UK mandated that patients had to be given the choice of at least five providers when being referred for treatment. Prior to this time the decision lay principally with the referring clinician. The impact of this reform was previously examined in two papers: Gaynor, Moreno-Serra, and Propper (2012) and Cooper et al. (2011). The latter of these attracted some criticism particularly after it was used in support of the controversial Health and Social Care Act (2012). One key aspect of this criticism revolved around the use of mortality from acute myocardial infarction (AMI) as a quality indicator, despite AMI being an emergency condition over which patients have no choice about their treatment hospital. The former of those two papers expands the analysis to consider other outcomes such as all cause death.

In this new paper, examining the same 2006 reform, the authors this time examine coronary artery bypass graft surgery (CABG). CABG is an elective condition thus permitting patient choice. The analysis considers where patients chose to go and on what basis, the effect of choice on patient mortality, and the effect of competition on hospital market share. The authors develop a novel method to analyse consideration sets to compare choices made prior to and after the reform. One of the key findings is that patients respond to signals of quality – in this case hospital mortality rates. And this improved sorting of patients into hospitals with lower mortality rates. However, here the distinction between a quality signal and actual quality is blurred.

It stands to reason that a patient would prefer a hospital with lower apparent mortality rates. But, mortality rates, whether adjusted or unadjusted, have been shown to be poorly correlated with preventable mortality in the NHS. The mortality rates used in this paper are the estimated (OLS) coefficients from a model of in-hospital death regressed on dummy variables for each hospital, thus estimating the crude mortality rate. To address the potential mismatch between mortality rates and the causal effect of a hospital on patient mortality, Gaynor, Propper, and Seiler also use an instrumental variable (IV) estimator for the hospital dummy with patient distance to each hospital as the instrument. This follows the method of Gowrisankaran and Town (1999). Gaynor, Propper, and Seiler state that a Hausman test does not reject the hypothesis that the OLS and IV coefficients are different and so use the OLS crude mortality rate estimates in the primary analysis. Nevertheless they repeat the analysis and show that patient hospital of choice is also associated with the IV estimated mortality rate. But the question still remains as to whether these estimates can be relied upon to demonstrate that the reforms improved mortality risk in the CABG cohort.

Gowrisankaran and Town showed there was little correlation in their study between GLS and IV estimates of hospital quality (see the Figure). Hogan et al. (2015) showed that the association between standardised hospital mortality ratios (SMR) and the proportion of preventable deaths was very weak. And Girling et al. (2012) estimated that if 6% of hospital deaths are preventable then the predictive value of the SMR can be no greater than 9%. However, they suggest that this could rise to 30% if 15% of deaths were preventable. So it seems perhaps surprising that Gaynor, Propper, and Seiler find no evidence of a difference between their OLS and IV estimators. Now, for CABG, the proportion of preventable deaths may be very high, Guru et al. (2008) estimated it to be as high as 32%. But, they also find there to be no correlation between preventable deaths and mortality rates in hospitals. Taken altogether this might suggest a flaw in the analysis of Gaynor, Propper, and Seiler.

Scatterplot of the GLS and IV estimates of hospital quality from separate years regression.

Scatterplot of the GLS and IV estimates of hospital quality. (c) Elsevier Science B.V.

When choosing between healthcare providers patients are provided with information about quality. This normally comes in the form of SMRs as we have previously discussed. Gaynor, Propper, and Seiler demonstrate that patients respond to this information. But, as we have argued, these signals are poor with respect to actual quality. Thus the consequences of patients sorting into hospitals in terms of actual deaths avoided is difficult to ascertain. A Hausman test suggests that the OLS and IV results are similar in this study and there is an association between patient choice and the IV estimated quality variable. But many arguments may run counter to these findings: the Hausman test could have low power, the IV estimator may be biased by the large number of moment restrictions, the instruments may not be conditionally independent of the hospitals, the common support between hospitals may not include the highest risk patients, and so forth. This paper successfully demonstrates how patients respond to information in making their choice between hospitals, but whether the reforms reduced mortality remains unanswered in my opinion.

Photo credit: Ramdlon (CC0)

Chris Sampson’s journal round-up for 16th May 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Higher mortality rates amongst emergency patients admitted to hospital at weekends reflect a lower probability of admission. Journal of Health Services Research & Policy Published 6th May 2016

The ‘weekend effect‘ is the hot topic in health policy in the UK right now. Whether or not it exists, and whether or not it can be corrected by steamrollering junior doctors’ contracts, has major implications for the NHS. In this study the authors used data on 12.7 million A&E attendances and 4.7 million emergency admissions in England in 2013-14. It’s possible to be admitted to hospital via A&E or directly from a community service. A&E is available 24/7, while community services are more limited at the weekend. The analyses mainly use logistic regressions with the usual case-mix adjustments to estimate the probability of death within 30 days. Weekend attendance at A&E was not associated with a significantly higher probability of death than attendance during the week. On Saturday or Sunday, there were 7% fewer admissions via A&E than on weekdays. The number of direct admissions via referral from community services was a whopping 61% lower at weekends. For both groups of people admitted, the mortality rate at the weekend was higher than on weekdays; we see the familiar weekend effect. The 7% difference in A&E admission rates could not be explained by the patient characteristics available in the data, suggesting that a higher admission threshold is used at weekends. There was no weekend effect associated with A&E attendances, which is perhaps what a lot of people have in mind when they think about this issue. Only those admitted at the weekend have a higher mortality rate, and in particular those referred from community services. The implication is that mortality rates hide the true story by combining the number of people dying (the numerator) with the number of people being admitted (the denominator). Increasing the number of doctors available at weekends might increase the number of people being admitted (at great cost) but with no reduction in the number of deaths. Patients who are admitted to hospital at the weekend are a different group of people, and different in a way that has not yet been adequately captured by risk-adjustment.

Ageing, justice and resource allocation. Journal of Medical Ethics [PhilPapers] [PubMedPublished 4th May 2016

People are living longer. This contributes to health care expenditure growth as people require more treatment to keep them alive. In this paper, the author argues that we should not focus only on the role of life-prolonging treatments but also on life-enhancing treatments. How people age and the ways in which the chances of becoming ill vary with age ought to be considered in resource allocation decisions. Social context is important in this respect; for example, the availability of public toilets may influence an older person’s willingness to engage in their usual activities. The arguments presented focus mainly on Norman Daniels’s prudential lifespan approach, which essentially considers whether or not a person would choose to purchase insurance for a particular health problem. We would expect an ageing population to insure more against the health problems of later life, and so proportionally greater resources ought to be allocated to older people. But the paper does not pursuade me that this requires any departure from current practice or thought. When Alan Williams described the fair innings approach to just allocations of resources in old age, he was expressly concerned with the quality of life. I’m not clear on what this paper adds, aside from further criticism of Harris’s view that life-extending treatment should always trump life-enhancing treatment. But I know of nobody who actually supports that view. Nevertheless, it’s an interesting discussion with which I hope health economists will engage.

An elicitation of utility for quality of life under prospect theory. Journal of Health Economics [RePEcPublished 2nd May 2016

Back in 1979, Kahneman and Tversky introduced prospect theory. Simply, this deviation from expected utility theory demonstrates that people value gains and losses from a given reference point differently, and that people’s decisions relate to probabilities in a nonlinear way. One of the key aspects of prospect theory is that it allows for loss aversion, which has been observed in the health context. We may therefore wish to develop methods for the estimation of QALYs that are based on prospect theory. This study demonstrates the limited validity of expected utility in estimating QALYs and shows how to estimate utility using prospect theory. A representative Dutch sample of 500 people was recruited for 2 experiments carried out online. Demographic and health state data were collected and participants were presented with possible gains and losses in quality of life within a 20%-100% interval associated with a specified reference point. Loss aversion was observed in both experiments, with evidence that responses were reference-dependent. Furthermore, there was risk aversion associated with both gains and losses. This undermines expected utility as a reasonable basis on which to estimate QALYs. Furthermore, the study found utility to be concave, such that a loss from 60% to 40% was perceived as smaller than a loss from 40% to 20%. This not only differs from the way in which we estimate QALYs, but also from the nature of prospect theory in the valuation of monetary outcomes. Expect to hear plenty more about PT-QALYs in the future.

Efficiency of health investment: education or intelligence? Health Economics [PubMedPublished 3rd May 2016

People with better education are healthier and live longer. But is this due to their education, or simply due to intelligence? It should go without saying that measuring intelligence, let alone separating it from the effects of education, is not straightforward. This study looks at whether education is associated with a higher efficiency of health investment. Health outcome is measured as survival and health investment as hospitalisation for a given condition. The authors then go on to consider the extent to which any benefit is due to intelligence. The data include 2570 Dutch individuals surveyed in 1952 in their final year of primary school and then followed up again in 1983 and 1993. The sample includes those people with hospitalisation records for 1995-2005 and mortality data for 1995-2011. A structural equation model is estimated to capture the impact of intelligence with the states ‘healthy’, ‘hospitalised’ and ‘dead’. Intelligence is modelled as a latent variable based on an IQ test and a vocabulary test at the age of 12. The analysis treats education choice as exogenous but controls for numerous socioeconomic and school-specific variables. People with higher education were less likely to die after a hospitalisation, though this relationship disappears once intelligence is accounted for. This suggests that the health investment advantage of the better educated is due to intelligence. There are plenty of limitations to the study in terms of the available data, but the findings nevertheless suggest that education per se might not be as beneficial to health as previous studies have shown.