Sam Watson’s journal round-up for 10th September 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Probabilistic sensitivity analysis in cost-effectiveness models: determining model convergence in cohort models. PharmacoEconomics [PubMed] Published 27th July 2018

Probabilistic sensitivity analysis (PSA) is rightfully a required component of economic evaluations. Deterministic sensitivity analyses are generally biased; averaging the outputs of a model based on a choice of values from a complex joint distribution is not likely to be a good reflection of the true model mean. PSA involves repeatedly sampling parameters from their respective distributions and analysing the resulting model outputs. But how many times should you do this? Most times, an arbitrary number is selected that seems “big enough”, say 1,000 or 10,000. But these simulations themselves exhibit variance; so-called Monte Carlo error. This paper discusses making the choice of the number of simulations more formal by assessing the “convergence” of simulation output.

In the same way as sample sizes are chosen for trials, the number of simulations should provide an adequate level of precision, anything more wastes resources without improving inferences. For example, if the statistic of interest is the net monetary benefit, then we would want the confidence interval (CI) to exclude zero as this should be a sufficient level of certainty for an investment decision. The paper, therefore, proposed conducting a number of simulations, examining the CI for when it is ‘narrow enough’, and conducting further simulations if it is not. However, I see a problem with this proposal: the variance of a statistic from a sequence of simulations itself has variance. The stopping points at which we might check CI are themselves arbitrary: additional simulations can increase the width of the CI as well as reduce them. Consider the following set of simulations from a simple ratio of random variables ICER = gamma(1,0.01)/normal(0.01,0.01):ciwidthThe “stopping rule” therefore proposed doesn’t necessarily indicate “convergence” as a few more simulations could lead to a wider, as well as narrower, CI. The heuristic approach is undoubtedly an improvement on the current way things are usually done, but I think there is scope here for a more rigorous method of assessing convergence in PSA.

Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. The Lancet [PubMed] Published 5th September 2018

Richard Horton, the oracular editor-in-chief of the Lancet, tweeted last week:

There is certainly an argument that academic journals are good forums to make advocacy arguments. Who better to interpret the analyses presented in these journals than the authors and audiences themselves? But, without a strict editorial bulkhead between analysis and opinion, we run the risk that the articles and their content are influenced or dictated by the political whims of editors rather than scientific merit. Unfortunately, I think this article is evidence of that.

No-one debates that improving health care quality will improve patient outcomes and experience. It is in the very definition of ‘quality’. This paper aims to estimate the numbers of deaths each year due to ‘poor quality’ in low- and middle-income countries (LMICs). The trouble with this is two-fold: given the number of unknown quantities required to get a handle on this figure, the definition of quality notwithstanding, the uncertainty around this figure should be incredibly high (see below); and, attributing these deaths in a causal way to a nebulous definition of ‘quality’ is tenuous at best. The approach of the article is, in essence, to assume that the differences in fatality rates of treatable conditions between LMICs and the best performing health systems on Earth, among people who attend health services, are entirely caused by ‘poor quality’. This definition of quality would therefore seem to encompass low resourcing, poor supply of human resources, a lack of access to medicines, as well as everything else that’s different in health systems. Then, to get to this figure, the authors have multiple sources of uncertainty including:

  • Using a range of proxies for health care utilisation;
  • Using global burden of disease epidemiology estimates, which have associated uncertainty;
  • A number of data slicing decisions, such as truncating case fatality rates;
  • Estimating utilisation rates based on a predictive model;
  • Estimating the case-fatality rate for non-users of health services based on other estimated statistics.

Despite this, the authors claim to estimate a 95% uncertainty interval with a width of only 300,000 people, with a mean estimate of 5.0 million, due to ‘poor quality’. This seems highly implausible, and yet it is claimed to be a causal effect of an undefined ‘poor quality’. The timing of this article coincides with the Lancet Commission on care quality in LMICs and, one suspects, had it not been for the advocacy angle on care quality, it would not have been published in this journal.

Embedding as a pitfall for survey‐based welfare indicators: evidence from an experiment. Journal of the Royal Statistical Society: Series A Published 4th September 2018

Health economists will be well aware of the various measures used to evaluate welfare and well-being. Surveys are typically used that are comprised of questions relating to a number of different dimensions. These could include emotional and social well-being or physical functioning. Similar types of surveys are also used to collect population preferences over states of the world or policy options, for example, Kahneman and Knetsch conducted a survey of WTP for different environmental policies. These surveys can exhibit what is called an ’embedding effect’, which Kahneman and Knetsch described as when the value of a good varies “depending on whether the good is assessed on its own or embedded as part of a more inclusive package.” That is to say that the way people value single dimensional attributes or qualities can be distorted when they’re embedded as part of a multi-dimensional choice. This article reports the results of an experiment involving students who were asked to weight the relative importance of different dimensions of the Better Life Index, including jobs, housing, and income. The randomised treatment was whether they rated ‘jobs’ as a single category, or were presented with individual dimensions, such as the unemployment rate and job security. The experiment shows strong evidence of embedding – the overall weighting substantially differed by treatment. This, the authors conclude, means that the Better Life Index fails to accurately capture preferences and is subject to manipulation should a researcher be so inclined – if you want evidence to say your policy is the most important, just change the way the dimensions are presented.

Credits

Alastair Canaway’s journal round-up for 28th May 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Information, education, and health behaviours: evidence from the MMR vaccine autism controversy. Health Economics [PubMed] Published 2nd May 2018

In 1998, Andrew Wakefield published (in the Lancet) his infamous and later retracted research purportedly linking the measles-mumps-rubella (MMR) vaccine and autism. Despite the thorough debunking and exposure of academic skulduggery, a noxious cloud of misinformation remained in the public mind, particularly in the US. This study examined several facets of the MMR fake news including: what impact did this have on vaccine uptake in the US (both MMR and other vaccines); how did state level variation in media coverage impact uptake; and what role did education play in subsequent decisions about whether to vaccinate or not. This study harnessed the National Immunization Survey from 1995 to 2006 to answer these questions. This is a yearly dataset of over 200,000 children aged between 19 to 35 months with detailed information on not just immunisation, but also maternal education, income and other sociodemographics. The NewsLibrary database was used to identify stories published in national and state media relating to vaccines and autism. Various regression methods were implemented to examine these data. The paper found that, unsurprisingly, for the year following the Wakefield publication the MMR vaccine take-up declined by between 1.1%-1.5% (notably less than 3% in the UK), likewise this fall in take-up spilled over into other vaccines take-up. The most interesting finding related to education: MMR take-up for children of college-educated mothers declined significantly compared to those without a degree. This can be explained by the education gradient where more-educated individuals absorb and respond to health information more quickly. However, in the US, this continued for many years beyond 2003 despite proliferation of research refuting the autism-MMR link. This contrasts to the UK where educational link closed soon after the findings were refuted, that is, in the UK, the educated responded to the new information refuting the MMR-Autism link. In the US, despite the research being debunked, MMR uptake was lower in the children of those with higher levels of education for many more years. The author speculates that this contrast to the UK may be a result of the media influencing parents’ decisions. Whilst the media buzz in the UK peaked in 2002, it had largely subsided by 2003. In the US however, the media attention was constant, if not increasing till 2006, and so this may have been the reason the link remained within the US. So, we have Andrew Wakefield and arguably fearmongering media to blame for causing a long-term reduction in MMR take-up in the US. Overall, an interesting study leaning on multiple datasets that could be of interest for those working with big data.

Can social care needs and well-being be explained by the EQ-5D? Analysis of the Health Survey for England. Value in Health Published 23rd May 2018

There is increasing discussion about integrating health and social care to provide a more integrated approach to fulfilling health and social care needs. This creates challenges for health economists and decision makers when allocating resources, particularly when comparing benefits from different sectors. NICE itself recognises that the EQ-5D may be inappropriate in some situations. With the likes of ASCOT, ICECAP and WEMWBS frequenting the health economics world this isn’t an unknown issue. To better understand the relationship between health and social care measures, this EuroQol Foundation funded study examined the relationship between social care needs as measured by the Barthel Index, well-being measured using WEMWBS and also the GGH-12, and the EQ-5D as the measure of health. Data was obtained through the Health Survey for England (HSE) and contained 3354 individuals aged over 65 years. Unsurprisingly the authors found that higher health and wellbeing scores were associated with an increased probability of no social care needs. Those who are healthier or at higher levels of wellbeing are less likely to need social care. Of all the instruments, it was the self-care and the pain/discomfort dimensions of the EQ-5D that were most strongly associated with the need for social care. No GHQ-12 dimensions were statistically significant, and for the WEMWBS only the ‘been feeling useful’ and ‘had energy to spare’ were statistically significantly associated with social care need. The authors also investigated various other associations between the measures with many unsurprising findings e.g. EQ-5D anxiety/depression dimension was negatively associated with wellbeing as measured using the GHQ-12. Although the findings are favourable for the EQ-5D in terms of it capturing to some extent social care needs, there is clearly still a gap whereby some outcomes are not necessarily captured. Considering this, the authors suggest that it might be appropriate to strap on an extra dimension to the EQ-5D (known as a ‘bolt on’) to better capture important ‘other’ dimensions, for example, to capture dignity or any other important social care outcomes. Of course, a significant limitation with this paper relates to the measures available in the data. Measures such as ASCOT and ICECAP have been developed and operationalised for economic evaluation with social care in mind, and a comparison against these would have been more informative.

The health benefits of a targeted cash transfer: the UK Winter Fuel Payment. Health Economics [PubMed] [RePEc] Published 9th May 2018

In the UK, each winter is accompanied by an increase in mortality, often known as ‘excess winter mortality’ (EWM). To combat this, the UK introduced the Winter Fuel Payment (WFP), the purpose of the WFP is an unconditional cash transfer to households containing an older person (those most vulnerable to EWM) above the female state pension age with the intent for this to used to help the elderly deal with the cost of keeping their dwelling warm. The purpose of this paper was to examine whether the WFP policy has improved the health of elderly people. The authors use the Health Surveys for England (HSE), the Scottish health Survey (SHeS) and the English Longitudinal Study of Ageing (ELSA) and employ a regression discontinuity design to estimate causal effects of the WFP. To measure impact (benefit) they focus on circulatory and respiratory illness as measured by: self-reports of chest infection, nurse measured hypertension, and two blood biomarkers for infection and inflammation. The authors found that for those living in a household receiving the payment there was a 6% point reduction (p<0.01) in the incidence of high levels of serum fibrinogen (biomarker) which are considered to be a marker of current infection and are associated with chronic pulmonary disease. For the other health outcomes, although positive, the estimated effects were less robust and not statistically significant. The authors investigated the impact of increasing the age of eligibility for the WFP (in line with the increase of women’s pension age). Their findings suggest there may be some health cost associated with the increase in age of eligibility for WFP. To surmise, the paper highlights that there may be some health benefits from the receipt of the WFP. What it doesn’t however consider is opportunity cost. With WFP costing about £2 billion per year, as a health economist, I can’t help but wonder if the money could have been better spent through other avenues.

Credits

 

Thesis Thursday: James Oswald

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr James Oswald who has a PhD from the University of Sheffield. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Essays on well-being and mental health: determinants and consequences
Supervisors
Sarah Brown, Jenny Roberts, Bert Van Landeghem
Repository link
http://etheses.whiterose.ac.uk/18915/

What measures of health did you use in your research and how did these complement broader measures of well-being?

I didn’t use any measures of physical health. I used a few measures of subjective well-being (SWB) and mental health which vary across the chapters. In Chapter 2 I used life satisfaction and the Rutter Malaise Inventory. Life satisfaction is a global retrospective judgement of one’s life and is a measure of evaluative well-being (see Dolan and Metcalfe, 2012). The Rutter Malaise Inventory, a measure of affective well-being, is an index that is composed of 9 items that measure the respondent’s symptoms of psychological distress or depression. The measure of the mental health problems of adolescents that is used in Chapter 3 is the Strengths and Difficulties Questionnaire (SDQ). The SDQ is made up of four 5-item subscales: emotional problems, peer relationship problems, conduct problems, and hyperactivity/ inattention problems. Chapter 3 utilises the General Health Questionnaire (GHQ) as a measure of the mental health of parents. The GHQ is a screening instrument that was initially developed to diagnose psychiatric disorders. Chapter 4 utilises one measure of subjective well-being, which relates to the number of days of poor self-reported mental health (stress, depression, and problems with emotions) in the past 30 days.

Did your research result in any novel findings regarding the social determinants of mental well-being?

The findings of Chapter 2 suggested that bullying victimisation at age 11 has a large, adverse effect on SWB as an adult. Childhood bullying remains prevalent – recent estimates suggest that approximately 20-30% of children are bullied by other children. The evidence provided in Chapter 3 indicated that greater externalising problems of adolescents are positively associated with the likelihood that they engage in antisocial behaviour. Chapter 4 indicated two important findings. Firstly, Hurricane Katrina had a negative effect on the SWB of individuals living in the states that were directly affected by the disaster. Secondly, the analysis suggested that the Indian Ocean tsunami and the Haiti earthquake increased the SWB of Americans living closest to the affected areas.

How can natural disasters affect mental health?

My thesis presents evidence to suggest that their impact depends upon whether you live in the disaster area. I explored the role of geography by exploring the effects of three disasters – hurricane Katrina in 2005 (USA), Indian Ocean tsunami in 2004 (East Asia), and the Haiti earthquake in 2010. Firstly, Hurricane Katrina had a negative effect on the SWB of individuals living in the states that were directly affected by the disaster. As a result, the findings suggest that government intervention in the aftermath of disasters is needed to help mitigate the adverse effects of natural disasters on the SWB of people who live in the directly affected areas. For example, appropriate mental health services and counselling could be offered to people suffering unhappiness or distress. Secondly, the analysis suggested that the Indian Ocean tsunami and the Haiti earthquake increased the SWB of Americans living closest to the affected areas. This surprising finding may be explained by the interdependence of utility functions. Following the disasters, Americans were exposed to widespread coverage of the disasters via social and traditional media sources. Because of the media coverage, they may have thought about the catastrophic repercussions of the disasters for the victims. Consequently, Americans who lived closest to the affected areas may have compared themselves to the disaster victims, leading them to feel thankful that the disaster did not affect them, thus increasing their SWB.

The empirical results support the case that the utility functions of strangers may be interdependent, rather than independent, an assumption generally made in economics. Furthermore, the findings indicated no evidence that the effects of the disasters were more pronounced for individuals of the same ethnicity as the disaster victims. The results therefore suggest that geographical proximity to the affected areas, rather than sharing similar characteristics with the disaster victims, may determine the effects of natural disasters on SWB outside of the areas that were directly affected by the disasters. This issue is discussed in greater detail in Chapter 4 of my thesis.

How did you go about identifying some of the consequences of mental health problems?

My thesis uses a range of econometric methods to explore the determinants and consequences of mental health and subjective well-being. In Chapter 2 – for childhood bullying and adult subjective well-being – I used a range of methods including random effects ordered probit models, Hausman tests, and Heckman models. Chapter 3 investigates how the mental health of adolescents affects their participation in antisocial behaviour. The analysis uses random effects probit, multivariate probit, and conditional logit models. Chapter 4 investigates the effects of three natural disasters on subjective well-being in the USA. The chapter uses difference-in-differences methodology with a count data model called a zero-inflated negative binomial model.

Are there any policy recommendations that you would make in light of your research?

Chapter 2 suggests that being bullied as a child adversely affects subjective well-being as an adult. My analysis supports the case that preventing children bullying in schools may have a positive effect on the SWB of a large percentage of the adult population. Chapter 3 indicated that greater externalising problems of adolescents are positively associated with the likelihood that they engage in antisocial behaviour. Previous research has suggested that adolescents who commit antisocial behaviour have an increased probability of committing crime as adults. Consequently, the findings suggest that mental health interventions to target the externalising problems of adolescents may reduce future crime. The findings also suggest that the money spent on the “Troubled Families” programme may be spent more cost-effectively in reducing antisocial behaviour by expanding access to mental health interventions for adolescents, such as via the Improving Access to Psychological Therapies programme.

The findings of Chapter 4 suggest that government intervention in the aftermath of disasters is needed to help mitigate the adverse effects of natural disasters on the SWB of people who live in the directly affected areas. For example, appropriate mental health services and counselling could be offered to people suffering unhappiness or distress because of natural disasters.