Rita Faria’s journal round-up for 28th January 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Appraising the value of evidence generation activities: an HIV modelling study. BMJ Global Health [PubMed] Published 7th December 2018

How much should we spend on implementing our health care strategy versus getting more information to devise a better strategy? Should we devolve budgets to regions or administer the budget centrally? These are difficult questions and this new paper by Beth Woods et al has a brilliant stab at answering them.

The paper looks at the HIV prevention and treatment policies in Zambia. It starts by finding the most cost-effective strategy and the corresponding budget in each region, given what is currently known about the prevalence of the infection, the effectiveness of interventions, etc. The idea is that the regions receive a cost-effective budget to implement a cost-effective strategy. The issue is that the cost-effective strategy and budget are devised according to what we currently know. In practice, regions might face a situation on the ground which is different from what was expected. Regions might not have enough budget to implement the strategy or might have some leftover.

What if we spend some of the budget to get more information to make a better decision? This paper considers the value of perfect information given the costs of research. Depending on the size of the budget and the cost of research, it may be worthwhile to divert some funds to get more information. But what if we had more flexibility in the budgetary policy? This paper tests 2 more budgetary options: a national hard budget but with the flexibility to transfer funds from under- to overspending regions, and a regional hard budget with a contingency fund.

The results are remarkable. The best budgetary policy is to have a national budget with the flexibility to reallocate funds across regions. This is a fascinating paper, with implications not only for prioritisation and budget setting in LMICs but also for high-income countries. For example, the 2012 Health and Social Care Act broke down PCTs into smaller CCGs and gave them hard budgets. Some CCGs went into deficit, and there are reports that some interventions have been cut back as a result. There are probably many reasons for the deficit, but this paper shows that hard regional budgets clearly have negative consequences.

Health economics methods for public health resource allocation: a qualitative interview study of decision makers from an English local authority. Health Economics, Policy and Law [PubMed] Published 11th January 2019

Our first paper looked at how to use cost-effectiveness to allocate resources between regions and across health care services and research. Emma Frew and Katie Breheny look at how decisions are actually made in practice, but this time in a local authority in England. Another change of the 2012 Health and Social Care Act was to move public health responsibilities from the NHS to local authorities. Local authorities are now given a ring-fenced budget to implement cost-effective interventions that best match their needs. How do they make decisions? Thanks to this paper, we’re about to find out.

This paper is an enjoyable read and quite an eye-opener. It was startling that health economics evidence was not much used in practice. But the barriers that were cited are not insurmountable. And the suggestions by the interviewees were really useful. There were suggestions about how economic evaluations should consider the local context to get a fair picture of the impact of the intervention to services and to the population, and to move beyond the trial into the real world. Equity was mentioned too, as well as broadening the outcomes beyond health. Fortunately, the health economics community is working on many of these issues.

Lastly, there was a clear message to make economic evidence accessible to lay audiences. This is a topic really close to my heart, and something I’d like to help improve. We have to make our work easy to understand and use. Otherwise, it may stay locked away in papers rather than do what we intended it for. Which is, at least in my view, to help inform decisions and to improve people’s lives.

I found this paper reassuring in that there is clearly a need for economic evidence and a desire to use it. Yes, there are some teething issues, but we’re working in the right direction. In sum, the future for health economics is bright!

Survival extrapolation in cancer immunotherapy: a validation-based case study. Value in Health Published 13th December 2018

Often, the cost-effectiveness of cancer drugs hangs in the method to extrapolate overall survival. This is because many cancer drugs receive their marketing authorisation before most patients in the trial have died. Extrapolation is tested extensively in the sensitivity analysis, and this is the subject of many discussions in NICE appraisal committees. Ultimately, at the point of making the decision, the correct method to extrapolate is a known unknown. Only in hindsight can we know for sure what the best choice was.

Ash Bullement and colleagues take advantage of hindsight to know the best method for extrapolation of a clinical trial of an immunotherapy drug. Survival after treatment with immunotherapy drugs is more difficult to predict because some patients can survive for a very long time, while others have much poorer outcomes. They fitted survival models to the 3-year data cut, which was available at the time of the NICE technology appraisal. Then they compared their predictions to the observed survival in the 5-year data cut and to long-term survival trends from registry data. They found that the piecewise model and a mixture-cure model had the best predictions at 5 years.

This is a relevant paper for those of us who work in the technology appraisal world. I have to admit that I can be sceptical of piecewise and mixture-cure models, but they definitely have a role in our toolbox for survival extrapolation. Ideally, we’d have a study like this for all the technology appraisals hanging on the survival extrapolation so that we can take learnings across cancers and classes of drugs. With time, we would get to know more about what works best for which condition or drug. Ultimately, we may be able to get to a stage where we can look at the extrapolation with less inherent uncertainty.


Are user fees a barrier to health care in poor countries?

The 1987 Bamako declaration promoted user or consultation fees for health care as a means to raise revenue and improve the quality of services. However, user fees may pose a barrier to access, and hence the key Sustainable Development Goal of Universal Health Coverage (UHC), for the global poor who typically have a high elasticity of demand for health services. The evidence has been mixed though on the impact of adding or removing user fees. A Cochrane review found that utilisation of services typically declined significantly with the introduction of fees and that quality was often found to improve with fees, but they also questioned the reliability of these studies due to a “high risk of bias”. Indeed, the evidence can be conflicting as to the effect of user fees on health service utilisation. Consider the following two studies from two similar countries: Malawi and Zambia.

The first looks at the effect the introduction and removal user fees had on health centre outpatient attendances, new diagnoses of malaria, and HIV diagnoses in a rural district of Malawi (which I should declare I authored!) Of 13 centres in Neno district, four were operated by the Christian Health Association of Malawi, of which one has always charged user fees and three introduced them in July 2013. The other centres were operated by the Ministry for Health and an NGO, Partners In Health, and did not charge fees. In July 2015, one centre removed user fees. These changes in charging status created a neat natural experiment. A plot of outpatient attendances shows what happened:

Figure 3

Even without modelling it is clear what happened – attendances dropped with the introduction of user fees and increased when they were removed. Similar changes were seen in new malaria and HIV diagnoses Further analysis also suggested patients weren’t moving between centres to avoid fees.

The second study, published this week, looks at a 2006 policy to remove user fees for publicly-funded health care facilities in rural districts across Zambia. The policy was instigated by the Zambian president as a step towards UHC, but was implemented haphazardly with funding not being completely in place and districts choosing to distribute the funding they received in different ways. Using data from a repeated cross-sectional health survey, the corresponding plot of the effects of the policy is:


Evident from this and reinforced by their synthetic control analysis, the policy did little to change the proportion of people seeking health care. The key impact of the policy was to reduce out of pocket expenditure as it seems people switched from using private providers to public providers. So why do the results of these studies, with seemingly similar ‘treatments’ in similar poor rural populations, differ so much?

In an earlier study of the Zambian policy it was found that outpatient attendances recorded in routine data – the same data used in the Malawi study above – there were large increases in use of public facilities when user fees were removed. The new study adds evidence though that this increase was a result of people switching from private to public providers. In Neno district, Malawi there are no private providers – only those in the study. Nevertheless, private providers also charge, so health care use in the face of fees was markedly higher in Zambia than Neno, Malawi. Perhaps there are relevant differences then in the populations under study.

Zambia, even in 2006, was much wealthier than Malawi in 2013. GDP per capita in comparable dollars was $1,030 in 2006 Zambia and $333 in 2013 Malawi. And Neno district is among the poorest in Malawi. The Malawi study population may be significantly poorer then than that in Zambia, and so have yet more elastic demand. Then again, Zambia is one of the most unequal countries in the world, its wealth generated from a boom in the copper price and other commodities. Its Gini coefficient is 57.5 as compared to Malawi’s 43.9. Thus, one may expect rural Zambians to perhaps be comparable to those in Malawi despite GDP differences. Unfortunately, there aren’t further statistics in the paper to compare the samples – and indeed no information on the relative prices of the user fees. And further, the Zambia paper does look at the poorest 50% of people separately and finds little difference in the treatment effect although there does appear to be large levels of heterogeneity in the estimated treatment effects between districts.

Differences in conclusions may also results from differences in data. For example, the Zambia study looked at changes in the reported use of formal health care among people who had an illness recently, whereas the Malawian study looked at outpatient attendances and diagnoses. Perhaps a difference could arise here such as reporting biases in the survey data.

It is not clear why results in the Zambian study differ from those in the Malawian one and indeed many others. It certainly shows the difficulty we have understanding the effect even small charges can have on access to care even as the quality of the evidence improves.


Does political reform really reduce child mortality?

Measuring causal effects is a tricky business. But, it’s necessary if we want to appropriately design effective policies and interventions. Many things are not amenable to manipulation in an experiment and so we rely upon a toolbox of statistical tools to try to identify the effect of interest. These methods are often ingenious, finding sophisticated ways of exploiting different types of variation, but they are essentially uninterpretable without an underlying causal theory. To illustrate this, let’s consider a paper that was featured a few weeks ago in the journal round-up: Effect of democratic reforms on child mortality: a synthetic control analysis.

A large number of countries have undergone democratic reform over the last 20 years. This article aimed to estimate how that reform has impacted upon child mortality. To do this a synthetic control method was used.

Synthetic control

The synthetic control method was formalised by Alberto Abadie, Alexis Diamond, and Jens Hainmueller in an article in the Journal of the American Statistical Association. It’s particularly useful in the situation where there is one area or cluster or country that has undergone a change, and multiple potential countries or clusters to act as controls that did not undergo the change. The eponymous synthetic control is a weighted average of the potential control sites where the weights have been chosen to best replicate the pre-intervention trend in the intervention site. The example given by Abadie and colleagues is estimation of the impact of tobacco control reform in California on tobacco consumption. The other US states are all potential controls. Bayesian synthetic control methods have also been established (by a team at Google), which we will make use of later.

The synthetic control method therefore seems appropriate to analyse the impact of democratic reform in a given country. Measurement of democratic reform was based on a change in the Polity2 index that rates the degree of autocracy/democracy in countries; a switch in the index from negative to positive (the index runs from -10 to 10) was considered ‘democratic reform’. Of the 60 countries identified as having reformed, 33 met the inclusion criteria, and 24 counterfactuals were able to be constructed. The primary outcome is the relative reduction in child mortality after ten years; the results from the 24 countries are shown in the histogram below (Figure 1). It would seem that, on average, democratic reform seems to reduce child mortality.


Figure 1. Histogram of results from the 24 included countries from Pieters et al.


Perhaps one of the factors that have limited research in the area of political economy and health is the complexity of the relationships between the various macro, micro, economic, and political effects. For example, on the basis of the evidence presented above, we would still not be able to say whether, for a given country, introducing democratic reform would have any impact on child mortality. Let’s consider a couple of examples to explore why.


Figure 2. Results from synthetic control analyses of the impact of democratic change on child mortality. Data from UN Inter-agency Group for Child Mortality Estimation.


Mozambique was engaged in a civil war between 1977 and 1992 as the communist Frelimo battled the anti-communist Renamo for control of the country. At the cessation of hostilities in 1993, wide sweeping reforms were enacted by Joaquim Chissano, and an election was held. We can consider 1993 as the year of democratic reform and conduct our own synthetic control analysis (using the aforementioned Bayesian approach). The results are shown in the figure above (Figure 2). Clearly, there is a significant reduction, but is this due to democratic reform or simply the end of civil war? A counterfactual approach is used as the theory of causation behind much statistical inference. Had an autocratic regime followed the civil war would there have still been declines in child mortality? I would conjecture that there would have been. Democratic reform in this case is either not a cause or a redundant cause.


Kenneth Kuanda was removed as the president of Zambia in 1991. Following this, reforms for multiparty democracy were enacted. Figure 2 above reports the estimated impact upon child mortality. A decline is clearly evident, however, this decline does not start until 2003, when the copper price tripled (Figure 3 below; copper constitutes 88% of Zambia’s exports) and GDP per capita almost doubled. Again, whether democratic reform can be inferred as a cause in this instance is questionable, especially when reduced to questions of counterfactuals.


Figure 3. Copper price per metric ton (US$). Data from IMF Cross Country Macroeconomic Statistics Database.

South Africa

The apartheid regime was ousted in 1994 following election of the ANC in South Africa. This regime change opened up the political institutions to the majority of South Africans who had previously been excluded. However, as Figure 2 above shows, this reform appeared to have little impact on child mortality. Indeed, South African healthcare still faces significant challenges, and large structural inequalities in access to quality healthcare persist even today. Taken together, this illustrates that democratic reform is not a sufficient condition for improvement of population health.


Finally, consider Zimbabwe, which became more autocratic in 1986 following a deal between Zanu and Zapu. If democratic reform improves child mortality then it seems reasonable that autocratic changes would worsen child mortality. Figure 2 above reports the synthetic control results for Zimbabwe. No change is evident.

What can we conclude?

Democratic reform is neither a sufficient nor necessary condition for improvements in child mortality. We cannot understand the evidence without an underlying theory. The study discussed here is a good data analysis; decent analyses in this area should be encouraged. But, the theory should come before the data.