Sam Watson’s journal round-up for 13th November 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Scaling for economists: lessons from the non-adherence problem in the medical literature. Journal of Economic Perspectives [RePEcPublished November 2017

It has often been said that development economics has been at the vanguard of the use of randomised trials within economics. Other areas of economics have slowly caught up; the internal validity, and causal interpretation, offered by experimental randomised studies can provide reliable estimates for the effects of particular interventions. Health economics though has perhaps an even longer history with randomised controlled trials (RCTs), and now economic evaluation is often expected alongside clinical trials. RCTs of physician incentives and payments, investment programmes in child health, or treatment provision in schools all feature as other examples. However, even experimental studies can suffer from the same biases in the data analysis process as observational studies. The multiple decisions made in the data analysis and publication stages of research can lead to over-inflated estimates. Beyond that, the experimental conditions of the trial may not pertain in the real world – the study may lack external validity. The medical literature has long recognised this issue, as many as 50% of patients don’t take the medicines prescribed to them by a doctor. As a result, there has been considerable effort to develop an understanding of, and interventions to remedy, the lack of transferability between RCTs and real-world outcomes. This article summarises this literature and develops lessons for economists, who are only just starting to deal with, what they term, ‘the scaling problem’. For example, there are many reasons people don’t respond to incentives as expected: there are psychological costs to switching; people are hyperbolic discounters and often prefer small short-term gains for larger long-term costs; and, people can often fail to understand the implications of sets of complex options. We have also previously discussed the importance of social preferences in decision making. The key point is that, as policy is becoming more and more informed by randomised studies, we need to be careful about over-optimism of effect sizes and start to understand adherence to different policies in the real world. Only then are recommendations reliable.

Estimating the opportunity costs of bed-days. Health Economics [PubMedPublished 6th November 2017

The health economic evaluation of health service delivery interventions is becoming an important issue in health economics. We’ve discussed on many occasions questions surrounding the implementation of seven-day health services in England and Wales, for example. Other service delivery interventions might include changes to staffing levels more generally, medical IT technology, or an incentive to improve hand washing. Key to the evaluation of these interventions is that they are all generally targeted at improving quality of care – that is, to reduce preventable harm. The vast majority of patients who experience some sort of preventable harm do not die but are likely to experience longer lengths of stay in hospital. Consider a person suffering from bed sores or a fall in hospital. Therefore, we need to be able to value those extra bed days to be able to say what the value of improving hospital quality is. Typically we use reference costs or average accounting costs for the opportunity cost of a bed-day, mainly for pragmatic reasons, but also on the assumption that this is equivalent to the value of the second-best alternative foregone. This requires the assumption that health care markets operate properly, which they almost certainly do not. This paper explores the different ways economists have thought about opportunity costs and applies them to the question of the opportunity cost of a hospital bed-day. This includes definitions such as “Net health benefit forgone for the second-best patient‐equivalents”, “Net monetary benefit forgone for the second-best treatment-equivalents”, and “Expenditure incurred + highest net revenue forgone.” The key takeaway is that there is wide variation in the estimated opportunity costs using all the different methods and that, given the assumptions underpinning the most widely used methodologies are unlikely to hold, we may be routinely under- or over-valuing the effects of different interventions.

Universal investment in infants and long-run health: evidence from Denmark’s 1937 Home Visiting Program. American Economic Journal: Applied Economics [RePEcPublished October 2017

We have covered a raft of studies that look at the effects of in-utero health on later life outcomes, the so-called fetal origins hypothesis. A smaller, though by no means small, literature has considered what impact improving infant and childhood health has on later life adult outcomes. While many of these studies consider programmes that occurred decades ago in the US or Europe, their findings are still relevant today as many countries are grappling with high infant and childhood mortality. For many low-income countries, programmes with community health workers – lay-community members provided with some basic public health training – involving home visits, education, and referral services are being widely adopted. This article looks at the later life impacts of an infant health programme, the Home Visiting Program, implemented in Denmark in the 1930s and 40s. The aim of the programme was to provide home visits to every newborn in each district to provide education on feeding and hygiene practices and to monitor infant progress. The programme was implemented in a trial based fashion with different districts adopting the programme at different times and some districts remaining as control districts, although selection into treatment and control was not random. Data were obtained about the health outcomes in the period 1980-2012 of people born 1935-49. In short, the analyses suggest that the programme improved adult longevity and health outcomes, although the effects are small. For example, they estimate the programme reduced hospitalisations by half a day between the age of 45 and 64, and 2 to 6 more people per 1,000 survived past 60 years of age. However, these effect sizes may be large enough to justify what may be a reasonably low-cost programme when scaled across the population.


Method of the month: Synthetic control

Once a month we discuss a particular research method that may be of interest to people working in health economics. We’ll consider widely used key methodologies, as well as more novel approaches. Our reviews are not designed to be comprehensive but provide an introduction to the method, its underlying principles, some applied examples, and where to find out more. If you’d like to write a post for this series, get in touch. This month’s method is synthetic control.


Health researchers are often interested in estimating the effect of a policy of change at the aggregate level. This might include a change in admissions policies at a particular hospital, or a new public health policy applied to a state or city. A common approach to inference in these settings is difference in differences (DiD) methods. Pre- and post-intervention outcomes in a treated unit are compared with outcomes in the same periods for a control unit. The aim is to estimate a counterfactual outcome for the treated unit in the post-intervention period. To do this, DiD assumes that the trend over time in the outcome is the same for both treated and control units.

It is often the case in practice that we have multiple possible control units and multiple time periods of data. To predict the post-intervention counterfactual outcomes, we can note that there are three sources of information: i) the outcomes in the treated unit prior to the intervention, ii) the behaviour of other time series predictive of that in the treated unit, including outcomes in similar but untreated units and exogenous predictors, and iii) prior knowledge of the effect of the intervention. The latter of these only really comes into play in Bayesian set-ups of this method. With longitudinal data we could just throw all this into a regression model and estimate the parameters. However, generally, this doesn’t allow for unobserved confounders to vary over time. The synthetic control method does.


Abadie, Diamond, and Haimueller motivate the synthetic control method using the following model:

y_{it} = \delta_t + \theta_t Z_i + \lambda_t \mu_i + \epsilon_{it}

where y_{it} is the outcome for unit i at time t, \delta_t are common time effects, Z_i are observed covariates with time-varying parameters \theta_t, \lambda_t are unobserved common factors with \mu_i as unobserved factor loadings, and \epsilon_{it} is an error term. Abadie et al show in this paper that one can derive a set of weights for the outcomes of control units that can be used to estimate the post-intervention counterfactual outcomes in the treated unit. The weights are estimated as those that would minimise the distance between the outcome and covariates in the treated unit and the weighted outcomes and covariates in the control units. Kreif et al (2016) extended this idea to multiple treated units.

Inference is difficult in this framework. So to produce confidence intervals, ‘placebo’ methods are proposed. The essence of this is to re-estimate the models, but using a non-intervention point in time as the intervention date to determine the frequency with which differences of a given order of magnitude are observed.

Brodersen et al take a different approach to motivating these models. They begin with a structural time-series model, which is a form of state-space model:

y_t = Z'_t \alpha_t + \epsilon_t

\alpha_{t+1} = T_t \alpha_t + R_t \eta_t

where in this case, y_t is the outcome at time t, \alpha_t is the state vector and Z_t is an output vector with \epsilon_t as an error term. The second equation is the state equation that governs the evolution of the state vector over time where T_t is a transition matrix, R_t is a diffusion matrix, and \eta_t is the system error.

From this setup, Brodersen et al expand the model to allow for control time series (e.g. Z_t = X'_t \beta), local linear time trends, seasonal components, and allowing for dynamic effects of covariates. In this sense the model is perhaps more flexible than that of Abadie et al. Not all of the large number of covariates may be necessary, so they propose a ‘slab and spike’ prior, which combines a point mass at zero with a weakly informative distribution over the non-zero values. This lets the data select the coefficients, as it were.

Inference in this framework is simpler than above. The posterior predictive distribution can be ‘simply’ estimated for the counterfactual time series to give posterior probabilities of differences of various magnitudes.



  • Synth Implements the method of Abadie et al.


  • Synth Implements the method of Abadie et al.
  • CausalImpact Implements the method of Brodersen et al.


Kreif et al (2016) estimate the effect of pay for performance schemes in hospitals in England and compare the synthetic control method to DiD. Pieters et al (2016) estimate the effects of democratic reform on under-five mortality. We previously covered this paper in a journal round-up and a subsequent post, for which we also used the Brodersen et al method described above. We recently featured a paper by Lépine et al (2017) in a discussion of user fees. The synthetic control method was used to estimate the impact that the removal of user fees had in various districts of Zambia on use of health care.


Sam Watson’s journal round-up for 30th October 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Conditional cash transfers: the case of Progresa/OportunidadesJournal of Economic Literature [RePEc] Published September 2017

The Progresa/Oportunidades programme was instigated in Mexico in 1995. The main innovation of the programme was a series of cash payments conditional on various human capital investments in children, such as regular school attendance and health check-ups. Beginning principally in rural areas, it expanded to urban areas in 2000-1. Excitingly for researchers, randomised implementation of the programme was built into its rollout, permitting evaluation of its effectiveness. Given it was the first such programme in a low- or middle-income country to do this, there has been a considerable amount of analysis and literature published on the topic. This article provides an in-depth review of this literature – incorporating over one hundred articles from economics and health journals. I’ll just focus on the health-related aspects of the review rather than education, labour market, or nutrition outcomes, but they’re also worth a look. The article provides a simple theoretical model about the effects of conditional cash transfers to start with and suggests that they have both a price effect, through reducing the shadow wage of time in activities other than those to which the payment is targeted, and an income effect, by increasing total income. The latter effect is ambiguous in its direction. For health, a large number of outcomes including child mortality and height, behavioural problems, obesity, and depression have all been assessed. For the most part  this has been through health modules applied to a subsample of people in surveys, which may limit the conclusions one can make for reasons such as attrition in the samples of treated and control households. Generally, the programme has demonstrated positive health effects (of varying magnitudes) in both the short and medium terms. Health care utilisation increased and with it there was a reduction in self-reported illness, behavioural problems, and obesity. However, positive effects are not reported universally. For example, one study reported an increase in child height in the short term, but in the medium term little change was reported in height-for-age z-scores in another study, which may suggest children catch-up in their growth. Nevertheless, it seems as though the programme succeeded in its aims, although there remains the question of its cost-benefit ratio and whether these ends could have been achieved more cost-effectively by other means. There is also the political question about the paternalism of the programme. While some political issues are covered, such as the perception of the programme as a vehicle for buying votes, and strategies for mitigating these issues, the issue of its acceptability to poor Mexicans is not well covered.

Health‐care quality and information failure: evidence from Nigeria. Health Economics [PubMedPublished 23rd October 2017

When we conceive of health care quality we often think of preventable harm to patients. Higher quality institutions make fewer errors such as incorrect diagnoses, mistakes with medication, or surgical gaffes. However, determining when an error has been made is difficult and quality is often poorly correlated with typical measures of performance like standardised mortality ratios. Evaluating quality is harder still in resource-poor settings where there are no routine data for evaluation and often an absence of patient records. Patients may also have less knowledge about what constitutes quality care. This may provide an environment for low-quality providers to remain in business as patients do not discriminate on the basis of quality. Patient satisfaction is another important aspect of quality, but not necessarily related to more ‘technical’ aspects of quality. For example, a patient may feel that they’ve not had to wait long and been treated respectfully even if they have been, unbeknownst to them, misdiagnosed and given the wrong medication. This article looks at data from Nigeria to examine whether measures of patient satisfaction are correlated with technical quality such as diagnostic accuracy and medicines availability. In brief, they report that there is little variation in patient satisfaction reports, which may be due to some reporting bias, and that diagnostic accuracy was correlated with satisfaction but other markers of quality were not. Importantly though, the measures of technical quality did little to explain the overall variation in patient satisfaction.

State intimate partner violence-related firearm laws and intimate partner homicide rates in the United States, 1991 to 2015. Annals of Internal Medicine [PubMedPublished 17th October 2017

Gun violence in the United States is a major health issue. Other major causes of death and injury attract significant financial investment and policy responses. However, the political nature of firearms in the US limit any such response. Indeed, a 1996 law passed by Congress forbade the CDC “to advocate or promote gun control”, which a succession of CDC directors has interpreted as meaning no federally funded research into gun violence at all. As such, for such a serious cause of death and disability, there is disproportionately little research. This article (not federally funded, of course) examines the impact of gun control legislation on inter-partner violence (IPV). Given the large proportion of inter-partner homicides (IPH) carried out with a gun, persons convicted of IPV felonies and, since 1996, misdemeanours are prohibited from possessing a firearm. However, there is variation in states about whether those convicted of an IPV crime have to surrender a weapon already in their possession. This article examines whether states that enacted ‘relinquishment’ laws that force IPV criminals to surrender their weapons reduced the rate of IPHs. They use state-level panel data and a negative binomial fixed effects model and find that relinquishment laws reduced the risk of IPHs by around 10% and firearm-related IPH by around 15%.