The health of people who live in slums and the trouble with estimating neighbourhood effects

Slums are a large and growing feature of urban areas in low and middle income countries. But, despite the ease with which you might picture what such informal settlements look like, there is no consensus about what exactly defines a slum. UN-Habitat defines ‘slum’ at the household level as a dwelling that lacks a number of important amenities such as improved water and sanitation while UNESCO defines it in terms of an urban area with certain features. In a new series of two papers, we discuss slum health and interventions designed to improve the health and welfare of people who live in slums. We make the argument that the UN-Habitat definition of slums is inadequate as slums exhibit unique neighbourhood effects.

Neighbourhood effects are those effects caused by the shared physical and social environment in which a person lives. Identifying and measuring such effects are important for public and health policy. In the context of slums, we argue that such neighbourhood effects determine the effectiveness of an intervention. For example, the benefits of provision of water and sanitation facilities are dependent on the already existing infrastructure, density of housing, and social structure of the area. The intervention may therefore be effective in some places but not in others, or require a certain level of input to reach a ‘tipping point’. However, estimation of the causal effect of a neighbourhood on population health and well-being can be difficult.

For certain outcomes causal neighbourhood effects are fairly easy to discern. Consider John Snow’s map of the 1854 outbreak of cholera below. The plot of cholera cases enabled John Snow to identify the infamous water pump from which cases of cholera were being contracted. In this instance, the causal neighbourhood effect of the shared water pump is clear, and the effects simple to measure. It’s not as if people contracted cholera and then decided to move closer to the water pump.


John Snow’s map of the 1854 cholera outbreak in London

A similar exercise can be conducted using survey data. The map below shows an estimate of the spatial distribution of cases of diarrhoea in the under 5s in Nairobi, Kenya, with notable slum areas marked by the letters A to E. There is clearly an strong correlation between slum areas and risk of diarrhoea. It would not be a strong assumption that there was a common cause of diarrhoea in slum areas rather than people who were more likely to get diarrhoea choose to move to those areas.*


Estimation of the risk of diarrhoea in the under 5s in Nairobi, Kenya. Disease risk is estimated by applying a spatial filter across a regular lattice grid and then estimating a binomial model to predict disease risk at each point. Red areas indicate higher risk and turquoise areas lower risk. Blue lines indicate areas with a >80% probability of having higher risk than the city average.

Inference about the effects of higher or lower wealth or socioeconomic status or more ephemeral characteristics of neighbourhoods on health and outcomes is more difficult. It is generally not possible to randomise people to neighbourhoods; individuals of lower socioeconomic status are more likely to move to poorer neighbourhoods. The exception is the Moving to Opportunity experiments in the US, which showed that better neighbourhoods improved adult health and improved the health and economic outcomes of their children.

J. Michael Oakes has a detailed discussion of the issues involved in the estimation of causal neighbourhood effects. He identifies four key problems. Firstly, due to social stratification between neighbourhoods, the “selection” equation that sorts individuals into neighbourhoods is likely to be nearly identical for all people in the same neighbourhood. Modelling selection therefore removes most of the variation between neighbourhoods. Secondly, even if neighbourhood effects were emergent properties of the interactions between individuals, such as the epidemiology of infection, they would still not be necessarily identifiable as the expression of those emergent properties is dependent on the neighbourhood level variables. Oakes likens it to trying to estimate the incidence by controlling for prevalence. Thirdly, neighbourhood level effects are not likely to be exchangeable, an assumption widely used in statistical inference. And fourthly, neighbourhood effects are not likely to be static. Arguably, quasi-experiemental methods such as instrumental variable or regression discontinuity designs and more sophisticated models may help solve these issues, but convincing applications can still remain elusive.

The points above contribute to the argument that the effectiveness of a community-level intervention, even when measured in a randomised trials, depends on neighbourhood effects. As we have discussed in other contexts, development of a causal theory is clearly required for the appropriate design of studies and interpretation of evidence. From a health and public policy standpoint innovations in methods of causal inference for neighbourhood effects can only be a good thing.


*Of course, there are other factors that may explain the correlation. For example, slum areas were more likely to be sampled in the rainy season. We also therefore examined childhood stunting, which showed the same pattern. See the paper for more detail.

Sam Watson’s journal round-up for October 24th 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Mortality decrease according to socioeconomic groups during the economic crisis in Spain: a cohort study of 36 million people. The Lancet [PubMed] Published 13th October 2016

There is no shortage of studies examining the relationship between macroeconomic conditions and population health. Papers have come up on the journal round-up here, here, and here, and we previously discussed economic conditions and baby health. So what does this study add? Using data from the 2011 Spanish census on 36 million individuals, the study compares age-adjusted mortality rates for different socioeconomic groups before and after the economic crisis in Spain. The socioeconomic status of households was classified on the basis of household wealth, household floor space, and number of cars. The study compares the annual change in mortality rates for 2004-7 to the annual percentage change in the post-crisis period 2008-11. In essence the authors are looking for a structural break. The article reports that mortality rates declined faster post-crisis than before and that this effect was more pronounced in low socioeconomic status households. However, this conclusion is based on observed differences in estimated changes of rate: differences between the socioeconomic groups are not directly tested. The authors seem to fall foul of the problem that the difference between “significant” and “not significant” is not itself statistically significant. The plots in the paper illustrate strong differences in age-adjusted mortality rates by socioeconomic status, but a structural break in changes in rates is not so clearly evident. A large econometric literature has arisen around measuring structural breaks in macroeconomic series, many of these methods may have been of use. Indeed, there have been a number of sophisticated and careful analyses of the effect of macroeconomic conditions and health previous published, including the seminal study by Christopher Ruhm. Why this study landed in The Lancet therefore seems somewhat mysterious.

The ambiguous effect of GP competition: the case of hospital admissions. Health Economics [PubMedPublished 14th October 2016

Another mainstay of this blog: competition in healthcare. We’ve covered papers on this topic in previous journal round-ups here and here, and critically discussed a paper on the topic here. It seems to be one of those topics with important implications for healthcare policy but one which becomes less certain the more is known. Indeed, this paper recognises this in its title. The ambiguity to which it refers is the effect of GP competition on hospital admissions: if GPs retain more patients due to increased competition then admissions go down; if they recruit new patients due to increased competition then admissions go up. Typically studies in this area either compare outcomes before and after a pro-competitive policy change, or compare outcomes between areas with different densities (and hence competition) between GPs. This study adopts a variant of the latter approach using the number of open list practices in an area as their proxy for competition. They find that increased competition reduces inpatient attendances and increases outpatient attendances. I’ve often been skeptical of the use of GP density as a proxy for competition. Do people really compare GP practices before choosing them or do they just go to the nearest one? If a person is already registered at one practice, how often do they search around to choose another if care isn’t that bad? An observed effect of a change in GP density could be attributable to entry into or exit from the ‘market’ of differently performing providers, which may have little to do with competition, more the type of GP, GP age, and differences in medical training. Nevertheless, this article does present a well-considered analysis, the difficulty is in the interpretation in light of all the previous studies.

Modeling the economic burden of adult vaccine-preventable diseases in the United States. Health Affairs [PubMed] Published 12th October 2016

Andrew Wakefield, disbarred doctor and disgraced author of the fraudulent Lancet paper on MMR and autism, is currently promoting his new anti-vaccine film. His work and a cabal of conspiracy theorists have led many parents to refuse to get their children vaccinated. All this despite vaccines being one of the safest and most cost-effective of health interventions. This new paper seeks to determine the economic burden of vaccine-preventable diseases is in the US. The diseases considered include hepatitis A and B; measles, mumps, and rubella; and shingles (herpes zoster). Epidemiological models were developed in conjunction with experts; economic costs were assessed using both cost-of-illness and full income methodologies; and, parameters were specified on the basis of a literature review. Taking into account healthcare costs and productivity losses, the burden of the considered diseases was estimated at $9 billion annually. The authors also discuss taking into account social welfare losses using the value of a statistical life, however I think I may be misinterpreting the results when it states

The current-dollar value of statistical life calculated from each source was $5.9 billion from the FDA; $6.3 billion from the NHTSA; and $8.3 billion from the EPA. The full income value of death as a result of vaccine-preventable diseases is estimated to be $176 billion annually (plausibility range: $166 billion–$231 billion).

That seems way too large to me so I’m not sure what to make of that. Nevertheless, the study illustrates the potentially massive burden that vaccine-preventable diseases may present.


Placebos for all, or why the p-value should have no place in healthcare decision making

David Colquhoun, professor of pharmacology at UCL, has a new essay over at Aeon opining about the problems with p-values. A short while back, we also discussed p-value problems, and Colquhoun arrives at the same conclusions as us about the need to abandon ideas of ‘statistical significance’. Despite mentioning Bayes theorem, Colquhoun’s essay is firmly based in the frequentist statistical paradigm. He frames his discussion around the frequency of false positive and negative findings in repeated trials:

An example should make the idea more concrete. Imagine testing 1,000 different drugs, one at a time, to sort out which works and which doesn’t. You’d be lucky if 10 per cent of them were effective, so let’s proceed by assuming a prevalence or prior probability of 10 per cent. Say we observe a ‘just significant’ result, for example, a P = 0.047 in a single test, and declare that this is evidence that we have made a discovery. That claim will be wrong, not in 5 per cent of cases, as is commonly believed, but in 76 per cent of cases. That is disastrously high. Just as in screening tests, the reason for this large number of mistakes is that the number of false positives in the tests where there is no real effect outweighs the number of true positives that arise from the cases in which there is a real effect.

The argument is focused on the idea that the aim of medical research is to determine whether an effect exists or not and this determination is made through repeated testing. This idea, imbued by null hypothesis significance testing, supports the notion of the researcher as discoverer. By finding p<0.05 the researcher has ‘discovered’ an effect.

Perhaps I am being unfair on Colquhoun. His essay is a well written argument about how the p-value fails on its own terms. Nevertheless, in 1,000 clinical drug trials, as the quote above considers, why would we expect any of the drugs to not have an effect? If a drug has reached the stage of clinical trials, then it has been designed on the basis of biological and pharmacological theory and it has likely been tested in animals and early phase clinical trials. All of these things would suggest that the drug has some kind of physiological mechanism and should demonstrate an effect. Even placebos exhibit a physiological response, otherwise why would we need placebo controls in trials?

The evidence suggests that a placebo, on average, reduces the relative risk of a given outcome by around 5%. One would therefore need an exceptionally large sample size to find a statistically significant effect of a placebo versus no treatment with 80% power. If we were examining in-hospital mortality as our outcome with a baseline risk of, say, 3%, then we would need approximately 210,000 patients for a 5% significance level. But, let’s say we did do this trial and found p<0.05. Given this ‘finding’, should hospitals provide sugar pills at the door? Even if patients are aware that a drug is a placebo it can still have an effect. The cost of producing and distributing sugar pills is likely to be relatively tiny. In a hospital with 50,000 annual admissions, 75 deaths per year could be averted by our sugar pill, so even if it cost £1,000,000 per year to provide our sugar pill, it would result in a cost per death averted of approximately £13,000 – highly cost-effective in all likelihood.

I think most people would unequivocally reject provision of sugar pills in hospitals for the same reason many people reject other sham treatments like homeopathy. It is potentially cost-effective, but it is perhaps unethical or inequitable for a health service to invest in treatments whose effects are not medically founded. But as we delve deeper into the question of the sugar pill, it is clear that at no point does the p-value matter to our reasoning. Whether or not we conducted our placebo mega-trial, it is the magnitude of the effect, uncertainty, prior evidence and understanding of the treatment, cost-effectiveness, and ethics and equity that we must consider.

Colquhoun acknowledges that the scientific process relies on inductive enquiry and that p-values cannot satisfy this. Even if there were not the significant problems of false positives and negatives that he discusses, p-values have no useful role to play in making the substantive decisions the healthcare service has to take. Nevertheless, health technology appraisal bodies, such as NICE in England and Wales, often use p<0.05 as a heuristic to filter out ‘ineffective’ treatments. But, as the literature warning against the use and misuse of null hypothesis significance testing grows, the tide may yet turn.