Rita Faria’s journal round-up for 4th March 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Cheap and dirty: the effect of contracting out cleaning on efficiency and effectiveness. Public Administration Review Published 25th February 2019

Before I was a health economist, I used to be a pharmacist and worked for a well-known high street chain for some years. My impression was that the stores with in-house cleaners were cleaner, but I didn’t know if this was a true difference, my leftie bias or my small sample size of 2! This new study by Shimaa Elkomy, Graham Cookson and Simon Jones confirms my suspicions, albeit in the context of NHS hospitals, so I couldn’t resist to select it for my round-up.

They looked at how contracted-out services fare in terms of perceived cleanliness, costs and MRSA rate in NHS hospitals. MRSA is a type of hospital-associated infection that is affected by how clean a hospital is.

They found that contracted-out services are cheaper than in-house cleaning, but that perceived cleanliness is worse. Importantly, contracted-out services increase the MRSA rate. In other words, contracting-out cleaning services could harm patients’ health.

This is a fascinating paper that is well worth a read. One wonders if the cost of managing MRSA is more than offset by the savings of contracting-out services. Going a step further, are in-house services cost-effective given the impact on patients’ health and costs of managing infections?

What’s been the bang for the buck? Cost-effectiveness of health care spending across selected conditions in the US. Health Affairs [PubMed] Published 1st January 2019

Staying on the topic of value for money, this study by David Wamble and colleagues looks at the extent to which the increased spending in health care in the US has translated into better health outcomes over time.

It’s clearly reassuring that, for 6 out of the 7 conditions they looked at, health outcomes have improved in 2015 compared to 1996. After all, that’s the goal of investing in medical R&D, although it remains unclear how much of this difference can be attributed to health care versus other things that have happened at the same time that could have improved health outcomes.

I wasn’t sure about the inflation adjustment for the costs, so I’d be grateful for your thoughts via comments or Twitter. In my view, we would underestimate the costs if we used medical price inflation indices. This is because these indices reflect the specific increase in prices in health care, such as due to new drugs being priced high at launch. So I understand that the main results use the US Consumer Price Index, which means that this reflects the average increase in prices over time rather than the increase in health care.

However, patients may not have seen their income rise with inflation. This means that the cost of health care may represent a disproportionally greater share of people’s income. And that the inflation adjustment may downplay the impact of health care costs on people’s pockets.

This study caught my eye and it is quite thought-provoking. It’s a good addition to the literature on the cost-effectiveness of US health care. But I’d wager that the question remains: to what extent is today’s medical care better value for money that in the past?

The dos and don’ts of influencing policy: a systematic review of advice to academics. Palgrave Communications Published 19th February 2019

We all would like to see our research findings influence policy, but how to do this in practice? Well, look no further, as Kathryn Oliver and Paul Cairney reviewed the literature, summarised it in 8 key tips and thought through their implications.

To sum up, it’s not easy to influence policy; advice about how to influence policy is rarely based on empirical evidence, and there are a few risks to trying to become a mover-and-shaker in policy circles.

They discuss three dilemmas in policy engagement. Should academics try to influence policy? How should academics influence policy? What is the purpose of academics’ engagement in policy making?

I particularly enjoyed reading about the approaches to influence policy. Tools such as evidence synthesis and social media should make evidence more accessible, but their effectiveness is unclear. Another approach is to craft stories to create a compelling case for the policy change, which seems to me to be very close to marketing. The third approach is co-production, which they note can give rise to accusations of bias and can have some practical challenges in terms of intellectual property and keeping one’s independence.

I found this paper quite refreshing. It not only boiled down the advice circulating online about how to influence policy into its key messages but also thought through the practical challenges in its application. The impact agenda seems to be here to stay, at least in the UK. This paper is an excellent source of advice on the risks and benefits of trying to navigate the policy world.

Credits

Sam Watson’s journal round-up for 11th February 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Contest models highlight inherent inefficiencies of scientific funding competitions. PLoS Biology [PubMed] Published 2nd January 2019

If you work in research you will have no doubt thought to yourself at one point that you spend more time applying to do research than actually doing it. You can spend weeks working on (what you believe to be) a strong proposal only for it to fail against other strong bids. That time could have been spent collecting and analysing data. Indeed, the opportunity cost of writing extensive proposals can be very high. The question arises as to whether there is another method of allocating research funding that reduces this waste and inefficiency. This paper compares the proposal competition to a partial lottery. In this lottery system, proposals are short, and among those that meet some qualifying standard those that are funded are selected at random. This system has the benefit of not taking up too much time but has the cost of reducing the average scientific value of the winning proposals. The authors compare the two approaches using an economic model of contests, which takes into account factors like proposal strength, public benefits, benefits to the scientist like reputation and prestige, and scientific value. Ultimately they conclude that, when the number of awards is smaller than the number of proposals worthy of funding, the proposal competition is inescapably inefficient. It means that researchers have to invest heavily to get a good project funded, and even if it is good enough it may still not get funded. The stiffer the competition the more researchers have to work to win the award. And what little evidence there is suggests that the format of the application makes little difference to the amount of time spent by researchers on writing it. The lottery mechanism only requires the researcher to propose something that is good enough to get into the lottery. Far less time would therefore be devoted to writing it and more time spent on actual science. I’m all for it!

Preventability of early versus late hospital readmissions in a national cohort of general medicine patients. Annals of Internal Medicine [PubMed] Published 5th June 2018

Hospital quality is hard to judge. We’ve discussed on this blog before the pitfalls of using measures such as adjusted mortality differences for this purpose. Just because a hospital has higher than expected mortality does not mean those death could have been prevented with higher quality care. More thorough methods assess errors and preventable harm in care. Case note review studies have suggested as little as 5% of deaths might be preventable in England and Wales. Another paper we have covered previously suggests then that the predictive value of standardised mortality ratios for preventable deaths may be less than 10%.

Another commonly used metric is readmission rates. Poor care can mean patients have to return to the hospital. But again, the question remains as to how preventable these readmissions are. Indeed, there may also be substantial differences between those patients who are readmitted shortly after discharge and those for whom it may take a longer time. This article explores the preventability of early and late readmissions in ten hospitals in the US. It uses case note review and a number of reviewers to evaluate preventability. The headline figures are that 36% of early readmissions are considered preventable compared to 23% of late readmissions. Moreover, it was considered that the early readmissions were most likely to have been preventable at the hospital whereas for late readmissions, an outpatient clinic or the home would have had more impact. All in all, another paper which provides evidence to suggest crude, or even adjusted rates, are not good indicators of hospital quality.

Visualisation in Bayesian workflow. Journal of the Royal Statistical Society: Series A (Statistics in Society) [RePEc] Published 15th January 2019

This article stems from a broader programme of work from these authors on good “Bayesian workflow”. That is to say, if we’re taking a Bayesian approach to analysing data, what steps ought we to be taking to ensure our analyses are as robust and reliable as possible? I’ve been following this work for a while as this type of pragmatic advice is invaluable. I’ve often read empirical papers where the authors have chosen, say, a logistic regression model with covariates x, y, and z and reported the outcomes, but at no point ever justified why this particular model might be any good at all for these data or the research objective. The key steps of the workflow include, first, exploratory data analysis to help set up a model, and second, performing model checks before estimating model parameters. This latter step is important: one can generate data from a model and set of prior distributions, and if the data that this model generates looks nothing like what we would expect the real data to look like, then clearly the model is not very good. Following this, we should check whether our inference algorithm is doing its job, for example, are the MCMC chains converging? We can also conduct posterior predictive model checks. These have had their criticisms in the literature for using the same data to both estimate and check the model which could lead to the model generalising poorly to new data. Indeed in a recent paper of my own, posterior predictive checks showed poor fit of a model to my data and that a more complex alternative was better fitting. But other model fit statistics, which penalise numbers of parameters, led to the alternative conclusions. So the simpler model was preferred on the grounds that the more complex model was overfitting the data. So I would argue posterior predictive model checks are a sensible test to perform but must be interpreted carefully as one step among many. Finally, we can compare models using tools like cross-validation.

This article discusses the use of visualisation to aid in this workflow. They use the running example of building a model to estimate exposure to small particulate matter from air pollution across the world. Plots are produced for each of the steps and show just how bad some models can be and how we can refine our model step by step to arrive at a convincing analysis. I agree wholeheartedly with the authors when they write, “Visualization is probably the most important tool in an applied statistician’s toolbox and is an important complement to quantitative statistical procedures.”

Credits

 

Thesis Thursday: Firdaus Hafidz

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Firdaus Hafidz who has a PhD from the University of Leeds. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Assessing the efficiency of health facilities in Indonesia
Supervisors
Tim Ensor, Sand Tubeuf
Repository link
http://etheses.whiterose.ac.uk/id/eprint/21575

What are some of the key features of health and health care in Indonesia?

Indonesia is a diverse country, with more than 17 thousand islands and 500 districts. Thus, there is a wide discrepancy of health outcomes across Indonesia, which also reflects the country’s double burden of both communicable and emerging non-communicable diseases. Communicable diseases such as tuberculosis, diarrhoea and lower respiratory tract infections remain as significant issues in Indonesia, especially in remote areas. At the same time, non-communicable diseases are becoming a major public health problem, especially in urban areas.

Total healthcare expenditure per capita grew rapidly, but in certain outcomes, such as maternal mortality rate, Indonesia performs less well than other low- and middle-income countries. Health facilities represent the largest share of healthcare expenditures, but utilisation is still considered low in both hospitals and primary healthcare facilities. Given the scarcity of public healthcare resources, out-of-pocket expenditure remains considerably higher than the global average.

To reduce financial barriers, the Government of Indonesia introduced health insurance in 1968. Between 2011 and 2014, there were three major insurance schemes: 1) Jamkesmas – poor scheme; 2) Jamsostek – formal sector workers scheme; and 3) Askes – civil servant scheme. In 2014, the three schemes were combined into a single-entity National Health Insurance scheme.

What methods can be used to measure the efficiency of health care in low and middle-income countries?

We reviewed measurements of efficiency in empirical analyses conducted in low- and middle-income countries. Methods, including techniques, variables, and efficiency indicators were summarised. There was no consensus on the most appropriate technique to measure efficiency, though most existing studies have relied on ratio analysis and data envelopment analysis because it is simple, easy to compute, low-cost and can be performed on small samples. The physical inputs included the type of capital (e.g. the number of beds and size of health facilities) and the type of labour (e.g. the number of medical and non-medical staff). Most of the published literature used health services as outputs (e.g. the number of outpatient visits, admission and inpatient days). However, because of poor data availability, fewer studies used case-mix and quality indicators to adjust outputs. So most of the studies in the literature review assumed that there was no difference in the severity and effectiveness of healthcare services. Despite the complexity of the techniques, researchers are responsible to provide interpretable results to the policymakers to guide their decisions for a better health policy on efficiency. Adopting appropriate methods that have been used globally would be beneficial to benchmark empirical studies.

Were you able to identify important sources of inefficiency in Indonesia?

We used several measurement techniques including frontier analysis and ratio analysis. We explored contextual variables to assess factors determining efficiency. The range of potential models produced help policymakers in the decision-making process according to their priority and allow some control over the contextual variables. The results revealed that the efficiency of primary care facilities can be explained by population health insurance coverage, especially through the insurance scheme for the poor. Geographical factors, such as the main islands (Java or Bali), better access to health facility, and location in an urban area also have a strong impact on efficiency. At the hospitals, the results highlighted higher efficiency levels in larger hospitals; they were more likely to present in deprived areas with low levels of education; and they were located on Java or Bali. Greater health insurance coverage also had a positive and significant influence on efficiency.

How could policymakers improve the efficiency of health care in Indonesia or other similar settings?

I think there are several ideas. First, we need to have a careful tariff adjustment as we found an association between low unit costs and high efficiency scores. Case base group tariffs need to account for efficiency scores to prevent unnecessary incentives for the providers, exacerbating inefficiency in the health system.

Secondly, we need flexibility in employment contracts, particularly for the less productive civil servant worker so the less productive worker could be reallocated. We also need a better remuneration policy to attract skilled labour and improve health facilities efficiency.

From the demand side, reducing physical barriers by improving infrastructure could increase efficiency in the rural health care facilities through higher utilisation of care. Facilities with very low utilisation rates still incur a fixed cost and thus create inefficiency. Through the same argument we also need to reduce financial barriers using incentives programmes and health insurance, thus patients who are economically disadvantaged can access healthcare services.

How would you like to see other researchers build on your work?

Data quality is crucial in secondary data analysis research, and it was quite a challenge in an Indonesian setting. Meticulous data management is needed to mitigate data errors such as inconsistency, outliers and missing values.

As this study used a 2011 cross-sectional dataset, replicating this study using a more recent and even longitudinal data would highlight changes in efficiency due to policy changes or interventions. Particularly interesting is the effect of the 2014 implementation of Indonesian national health insurance.

My study has some limitations and thus warrants further investigation. The stochastic frontier analysis failed to identify any inefficiency at hospitals when outpatient visits were included. The statistical errors of the frontier function cannot be distinguished from the inefficiency effect of the model. It might be related to the volume and heterogeneity of outpatient services which swamps the total volume of services and masks any inefficiency.