Sam Watson’s journal round-up for 21st August 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Multidimensional performance assessment of public sector organisations using dominance criteria. Health Economics [RePEcPublished 18th August 2017

The empirical assessment of the performance or quality of public organisations such as health care providers is an interesting and oft-tackled problem. Despite the development of sophisticated methods in a large and growing literature, public bodies continue to use demonstrably inaccurate or misleading statistics such as the standardised mortality ratio (SMR). Apart from the issue that these statistics may not be very well correlated with underlying quality, organisations may improve on a given measure by sacrificing their performance on another outcome valued by different stakeholders. One example from a few years ago showed how hospital rankings based upon SMRs shifted significantly if one took into account readmission rates and their correlation with SMRs. This paper advances this thinking a step further by considering multiple outcomes potentially valued by stakeholders and using dominance criteria to compare hospitals. A hospital dominates another if it performs at least as well or better across all outcomes. Importantly, correlation between these measures is captured in a multilevel model. I am an advocate of this type of approach, that is, the use of multilevel models to combine information across multiple ‘dimensions’ of quality. Indeed, my only real criticism would be that it doesn’t go far enough! The multivariate normal model used in the paper assumes a linear relationship between outcomes in their conditional distributions. Similarly, an instrumental variable model is also used (using the now routine distance-to-health-facility instrumental variable) that also assumes a linear relationship between outcomes and ‘unobserved heterogeneity’. The complex behaviour of health care providers may well suggest these assumptions do not hold – for example, failing institutions may well show poor performance across the board, while other facilities are able to trade-off outcomes with one another. This would suggest a non-linear relationship. I’m also finding it hard to get my head around the IV model: in particular what the covariance matrix for the whole model is and if correlations are permitted in these models at multiple levels as well. Nevertheless, it’s an interesting take on the performance question, but my faith that decent methods like this will be used in practice continues to wane as organisations such as Dr Foster still dominate quality monitoring.

A simultaneous equation approach to estimating HIV prevalence with nonignorable missing responses. Journal of the American Statistical Association [RePEcPublished August 2017

Non-response is a problem encountered more often than not in survey based data collection. For many public health applications though, surveys are the primary way of determining the prevalence and distribution of disease, knowledge of which is required for effective public health policy. Methods such as multiple imputation can be used in the face of missing data, but this requires an assumption that the data are missing at random. For disease surveys this is unlikely to be true. For example, the stigma around HIV may make many people choose not to respond to an HIV survey, thus leading to a situation where data are missing not at random. This paper tackles the question of estimating HIV prevalence in the face of informative non-response. Most economists are familiar with the Heckman selection model, which is a way of correcting for sample selection bias. The Heckman model is typically estimated or viewed as a control function approach in which the residuals from a selection model are used in a model for the outcome of interest to control for unobserved heterogeneity. An alternative way of representing this model is as copula between a survey response variable and the response variable itself. This representation is more flexible and permits a variety of models for both selection and outcomes. This paper includes spatial effects (given the nature of disease transmission) not only in the selection and outcomes models, but also in the model for the mixing parameter between the two marginal distributions, which allows the degree of informative non-response to differ by location and be correlated over space. The instrumental variable used is the identity of the interviewer since different interviewers are expected to be more or less successful at collecting data independent of the status of the individual being interviewed.

Clustered multistate models with observation level random effects, mover–stayer effects and dynamic covariates: modelling transition intensities and sojourn times in a study of psoriatic arthritis. Journal of the Royal Statistical Society: Series C [ArXiv] Published 25th July 2017

Modelling the progression of disease accurately is important for economic evaluation. A delicate balance between bias and variance should be sought: a model too simple will be wrong for most people, a model too complex will be too uncertain. A huge range of models therefore exists from ‘simple’ decision trees to ‘complex’ patient-level simulations. A popular choice are multistate models, such as Markov models, which provide a convenient framework for examining the evolution of stochastic processes and systems. A common feature of such models is the Markov property, which is that the probability of moving to a given state is independent of what has happened previously. This can be relaxed by adding covariates to model transition properties that capture event history or other salient features. This paper provides a neat example of extending this approach further in the case of arthritis. The development of arthritic damage in a hand joint can be described by a multistate model, but there are obviously multiple joints in one hand. What is more, the outcomes in any one joint are not likely to be independent of one another. This paper describes a multilevel model of transition probabilities for multiple correlated processes along with other extensions like dynamic covariates and different mover-stayer probabilities.


Meeting round-up: International Society for Economics and Social Sciences of Animal Health inaugural meeting

Last week I attended a conference that was very different to any that I’ve attended before. It was the first meeting of a new society – the International Society for Economics and Social Sciences of Animal Health (ISESSAH). I and Prof Marilyn James wanted to get involved with ISESSAH from the get-go in order to start identifying opportunities for collaboration with animal health researchers. In particular, we see the potential for the application of cost-effectiveness analysis methods in the veterinary context. The proceedings of the conference suggested that this is not something that is currently being done.

So off to the Highlands we headed, happily arriving in Aviemore while the town was improbably celebrating being the hottest place in the UK. Aside from my lack of sunglasses and excess of thick jumpers, I did have some intellectual concerns. I was a little worried that there would be few points of commonality between me and the other delegates. A hands-in-the-air poll during the first keynote speech by Tim Carpenter suggested that a minority of people in the room identified primarily as economists. Most people identified as “animal health specialists” and I suspect that most of these people were principally interested in epidemiological questions relating to livestock animals.

Happily, my fears were not realised. The first talk, by Erwin Wauters, discussed the challenge of framing research questions and in particular identifying the context of the decision. This is something we figured out a while ago in health economics and now have the luxury of bickering about health service and societal perspectives for our analyses. But the overlap was striking, as Erwin discussed the proliferation of ‘cost of disease’ studies with limited interpretability. I wondered (aloud, as a question) what the unique challenges might be in defining the context (what we would call perspective) in animal health as opposed to human health. This turned out to be prudent, as numerous delegates approached me over the proceeding 48 hours to tell me what they thought the answer was (euthanasia/culling, market structure, data availability, amongst others).

The whole conference consisted of methods that were familiar. Don’t get me wrong, most (though not all) of the subject matter was alien to me. But that’s par for the course in applied health economics anyway. Many of the studies – and I mean this to be in no way a criticism of those presenting – would strike health economists as analytically rudimentary. There were lots of cost-benefit analyses, plenty of epidemiological models with costs attached (does that make it an economic model?) and a handful of econometric analyses. Some studies (aside from my own poster) were very familiar and referred explicitly to ideas from the health economics field. In particular, Paul Torgerson and colleagues presented a framework that incorporates animal disease burden with DALY estimation. A French group mused on the role of QALYs.

Something consistent across many of the empirical studies was that the decision problems were ill-defined. In the economic evaluation of (human) heath care, we attribute major importance to the adequate definition of the decision problem and the identification and definition of all relevant options for the decision maker. It is perhaps for this reason that – as Jonathan Rushton argued – economics in the animal health context is used more for advocacy than to achieve optimality. Or maybe the causality goes the other way.

There were also lots of sociological and other sub-disciplines of social science represented, with fertile opportunities for interdisciplinary research. I didn’t like the distinction that was made throughout the conference between economics and social science. Economics is a social science. It isn’t bigger or better or distinct. Economists don’t need any encouragement in distancing themselves from sociologists and other social scientists. All of the research (with no exaggeration, though to varying extents) could benefit from health economists’ input. Thanks to our subfield’s softer edges, health economists make for good social science all-rounders. But then I would say that.

There was a discussion of how the conference will operate in the future. As someone who worships at the church of HESG, my instinct was to advise copying it. But that wouldn’t be right in this case (except perhaps for the levy of a nominal membership fee). ISESSAH will need to focus on interdisciplinarity. Delegates had a palpable taste and even excitement for interdisciplinary research. My (previously unknown) Nottingham colleague Marnie Brennan described how she thought the society would do well to adopt a policy of infiltration, to force interdisciplinary engagement, by creating a presence for itself at other conferences. The 2017 meeting took place alongside that of the Society for Veterinary Epidemiology and Preventive Medicine (SVEPM). Hopefully, in the future, we’ll see collaboration with human health research and economics societies and, who knows, maybe even the health economists.

The health of people who live in slums and the trouble with estimating neighbourhood effects

Slums are a large and growing feature of urban areas in low and middle income countries. But, despite the ease with which you might picture what such informal settlements look like, there is no consensus about what exactly defines a slum. UN-Habitat defines ‘slum’ at the household level as a dwelling that lacks a number of important amenities such as improved water and sanitation while UNESCO defines it in terms of an urban area with certain features. In a new series of two papers, we discuss slum health and interventions designed to improve the health and welfare of people who live in slums. We make the argument that the UN-Habitat definition of slums is inadequate as slums exhibit unique neighbourhood effects.

Neighbourhood effects are those effects caused by the shared physical and social environment in which a person lives. Identifying and measuring such effects are important for public and health policy. In the context of slums, we argue that such neighbourhood effects determine the effectiveness of an intervention. For example, the benefits of provision of water and sanitation facilities are dependent on the already existing infrastructure, density of housing, and social structure of the area. The intervention may therefore be effective in some places but not in others, or require a certain level of input to reach a ‘tipping point’. However, estimation of the causal effect of a neighbourhood on population health and well-being can be difficult.

For certain outcomes causal neighbourhood effects are fairly easy to discern. Consider John Snow’s map of the 1854 outbreak of cholera below. The plot of cholera cases enabled John Snow to identify the infamous water pump from which cases of cholera were being contracted. In this instance, the causal neighbourhood effect of the shared water pump is clear, and the effects simple to measure. It’s not as if people contracted cholera and then decided to move closer to the water pump.


John Snow’s map of the 1854 cholera outbreak in London

A similar exercise can be conducted using survey data. The map below shows an estimate of the spatial distribution of cases of diarrhoea in the under 5s in Nairobi, Kenya, with notable slum areas marked by the letters A to E. There is clearly an strong correlation between slum areas and risk of diarrhoea. It would not be a strong assumption that there was a common cause of diarrhoea in slum areas rather than people who were more likely to get diarrhoea choose to move to those areas.*


Estimation of the risk of diarrhoea in the under 5s in Nairobi, Kenya. Disease risk is estimated by applying a spatial filter across a regular lattice grid and then estimating a binomial model to predict disease risk at each point. Red areas indicate higher risk and turquoise areas lower risk. Blue lines indicate areas with a >80% probability of having higher risk than the city average.

Inference about the effects of higher or lower wealth or socioeconomic status or more ephemeral characteristics of neighbourhoods on health and outcomes is more difficult. It is generally not possible to randomise people to neighbourhoods; individuals of lower socioeconomic status are more likely to move to poorer neighbourhoods. The exception is the Moving to Opportunity experiments in the US, which showed that better neighbourhoods improved adult health and improved the health and economic outcomes of their children.

J. Michael Oakes has a detailed discussion of the issues involved in the estimation of causal neighbourhood effects. He identifies four key problems. Firstly, due to social stratification between neighbourhoods, the “selection” equation that sorts individuals into neighbourhoods is likely to be nearly identical for all people in the same neighbourhood. Modelling selection therefore removes most of the variation between neighbourhoods. Secondly, even if neighbourhood effects were emergent properties of the interactions between individuals, such as the epidemiology of infection, they would still not be necessarily identifiable as the expression of those emergent properties is dependent on the neighbourhood level variables. Oakes likens it to trying to estimate the incidence by controlling for prevalence. Thirdly, neighbourhood level effects are not likely to be exchangeable, an assumption widely used in statistical inference. And fourthly, neighbourhood effects are not likely to be static. Arguably, quasi-experiemental methods such as instrumental variable or regression discontinuity designs and more sophisticated models may help solve these issues, but convincing applications can still remain elusive.

The points above contribute to the argument that the effectiveness of a community-level intervention, even when measured in a randomised trials, depends on neighbourhood effects. As we have discussed in other contexts, development of a causal theory is clearly required for the appropriate design of studies and interpretation of evidence. From a health and public policy standpoint innovations in methods of causal inference for neighbourhood effects can only be a good thing.


*Of course, there are other factors that may explain the correlation. For example, slum areas were more likely to be sampled in the rainy season. We also therefore examined childhood stunting, which showed the same pattern. See the paper for more detail.