Sam Watson’s journal round-up for 10th April 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Expertise versus Bias in Evaluation: Evidence from the NIH. American Economic Journal: Applied Economics. Published April 2017.

As an academic’s career progresses, she learns two things: patience and learning to deal with rejection. Getting a paper accepted by a top journal is hard. Obtaining funding for what seems like a good idea similarly so. We sometimes convince ourselves that the system is rigged, or at least biased. Research funding bodies may make poor decisions. This paper considers this question in great deal. While reviewers may have an informational advantage that allows them to assess quality, they may also be biased towards projects in their own domain of expertise. More funding for health economics blogs! To assess this, this paper examines 100,000 applications to the US National Institutes of Health. The proximity of the reviewer to the subject area of the application is judged by the number of times the reviewer has cited the work of the applicant. Quality is judged by the number of publications and citations the research produces – an attempt is made to adapt this to judge unfunded work. The principle finding is that reviewers are both more informed and more biased about work in their own field. Each additional permanent reviewer in a applicant’s area is estimated to increase the chance of funding by 2.2 percent, an equivalent effect to increasing quality by one quarter standard deviation. These effects seem small, as the author notes, and what strikes me is how little variation these measures in explain in funding decisions. Perhaps I will find some solace in the fact that there is quite a lot of apparent randomness in what gets funded. Nevertheless, the author suggests that the findings suggest that by trying to reduce bias by using impartial reviewers, the ability to judge quality will also decline.

Long-term effects of youth unemployment on mental health: does an economic crisis make a difference? Journal of Epidemiology and Community Health. [PubMedPublished April 2017.

Unemployment is related to mental health issues. The effect is appears to be particularly acute among young people for whom the transition to adult life can be difficult. Indeed, at this vulnerable period young people also transition from youth to adult mental health services, which breaks their continuity of care. Many become lost in the system. Services in many areas are being redesigned in light of this. This paper asks if the effect of unemployment on youth mental health is different depending on the economic conditions. Do period of high unemployment nationally exacerbate the effects of becoming unemployed? Surprisingly, the paper concludes, no, there is no difference. I say ‘surprisingly’ since I cannot recall finding a paper in this area or one that has featured on this blog with a negative finding. The analyses seem careful, and the authors concentrate on the magnitude of the effects, rather than statistical significance. Large sample sizes are required for adequate power to test a hypothesis on an interaction; this study does have a large sample size. The interactive effect is likely to be very small, not necessarily non-existent. But in comparison with the large effects of unemployment on youth mental health in general, the effect of economic conditions is of little importance. Nevertheless, Simpson’s paradox may rear its head here: during times of high unemployment, the cohort of the unemployed will be different. If those who only become unemployed during economic downturns have lower risk of mental health issues, then this may attenuate the estimated effect of unemployment on mental health. This issue is not addressed unfortunately, but I don’t want that to detract from a sensible use of statistics.

The Distortionary Effects of Incentives in Government: Evidence from China’s ‘Death Ceiling’ Program. American Economic Journal: Applied Economics. [RePEcPublished April 2017.

Targets and incentives to achieve those targets can distort the actions of agents. This is especially true of difficult to observe outcomes. People may be more inclined to manipulate the data than to actually achieve the target. Gaming and other similar behaviours have been noted in health services, for example. This article examines a policy in China designed to reduce the high rates of accidental deaths. In 2004 the State Administration of Work Safety announced that provinces would have to reduce their rate of accidental deaths by 2.5% per year. The provinces were set a so-called ‘death ceiling’. In 2012, the policy was declared a success; accidental death rates had come down by 45% since 2005. But further examination of the data, which were made publicly available in the state newspaper the People’s Daily, suggests this may not be the case. First of all, there was a sharp discontinuity of accidental deaths right below the death ceiling. This discontinuity was not consistent with a continuous variable. Provinces had much discretion about how to achieve the reductions. Those that used significant incentives for local officials were more likely to be successful. The authors also consider why, if the data were manipulated, deaths weren’t made to look significantly below the death ceiling rather than just below the death ceiling. They speculate that this would have the effect of making next year’s death ceiling even lower and more difficult to achieve. This paper provides a nice narrative that adds to our understanding of the perverse effects of incentives. For health services this is important. For many of the difficult to observe outcomes, like patient health, merely incentivising doctors and hospitals to improve may have little actual benefit.

 

 

Credits

Advertisements

Meeting round-up: International Society for Economics and Social Sciences of Animal Health inaugural meeting

Last week I attended a conference that was very different to any that I’ve attended before. It was the first meeting of a new society – the International Society for Economics and Social Sciences of Animal Health (ISESSAH). I and Prof Marilyn James wanted to get involved with ISESSAH from the get-go in order to start identifying opportunities for collaboration with animal health researchers. In particular, we see the potential for the application of cost-effectiveness analysis methods in the veterinary context. The proceedings of the conference suggested that this is not something that is currently being done.

So off to the Highlands we headed, happily arriving in Aviemore while the town was improbably celebrating being the hottest place in the UK. Aside from my lack of sunglasses and excess of thick jumpers, I did have some intellectual concerns. I was a little worried that there would be few points of commonality between me and the other delegates. A hands-in-the-air poll during the first keynote speech by Tim Carpenter suggested that a minority of people in the room identified primarily as economists. Most people identified as “animal health specialists” and I suspect that most of these people were principally interested in epidemiological questions relating to livestock animals.

Happily, my fears were not realised. The first talk, by Erwin Wauters, discussed the challenge of framing research questions and in particular identifying the context of the decision. This is something we figured out a while ago in health economics and now have the luxury of bickering about health service and societal perspectives for our analyses. But the overlap was striking, as Erwin discussed the proliferation of ‘cost of disease’ studies with limited interpretability. I wondered (aloud, as a question) what the unique challenges might be in defining the context (what we would call perspective) in animal health as opposed to human health. This turned out to be prudent, as numerous delegates approached me over the proceeding 48 hours to tell me what they thought the answer was (euthanasia/culling, market structure, data availability, amongst others).

The whole conference consisted of methods that were familiar. Don’t get me wrong, most (though not all) of the subject matter was alien to me. But that’s par for the course in applied health economics anyway. Many of the studies – and I mean this to be in no way a criticism of those presenting – would strike health economists as analytically rudimentary. There were lots of cost-benefit analyses, plenty of epidemiological models with costs attached (does that make it an economic model?) and a handful of econometric analyses. Some studies (aside from my own poster) were very familiar and referred explicitly to ideas from the health economics field. In particular, Paul Torgerson and colleagues presented a framework that incorporates animal disease burden with DALY estimation. A French group mused on the role of QALYs.

Something consistent across many of the empirical studies was that the decision problems were ill-defined. In the economic evaluation of (human) heath care, we attribute major importance to the adequate definition of the decision problem and the identification and definition of all relevant options for the decision maker. It is perhaps for this reason that – as Jonathan Rushton argued – economics in the animal health context is used more for advocacy than to achieve optimality. Or maybe the causality goes the other way.

There were also lots of sociological and other sub-disciplines of social science represented, with fertile opportunities for interdisciplinary research. I didn’t like the distinction that was made throughout the conference between economics and social science. Economics is a social science. It isn’t bigger or better or distinct. Economists don’t need any encouragement in distancing themselves from sociologists and other social scientists. All of the research (with no exaggeration, though to varying extents) could benefit from health economists’ input. Thanks to our subfield’s softer edges, health economists make for good social science all-rounders. But then I would say that.

There was a discussion of how the conference will operate in the future. As someone who worships at the church of HESG, my instinct was to advise copying it. But that wouldn’t be right in this case (except perhaps for the levy of a nominal membership fee). ISESSAH will need to focus on interdisciplinarity. Delegates had a palpable taste and even excitement for interdisciplinary research. My (previously unknown) Nottingham colleague Marnie Brennan described how she thought the society would do well to adopt a policy of infiltration, to force interdisciplinary engagement, by creating a presence for itself at other conferences. The 2017 meeting took place alongside that of the Society for Veterinary Epidemiology and Preventive Medicine (SVEPM). Hopefully, in the future, we’ll see collaboration with human health research and economics societies and, who knows, maybe even the health economists.

Chris Sampson’s journal round-up for 3rd April 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Return on investment of public health interventions: a systematic review. Journal of Epidemiology & Community Health [PubMed] Published 29th March 2017

Cost-effectiveness analysis in the context of public health is tricky. Often the health benefits are small at the individual level and the returns to investment might be cross-sectoral. Lots of smart people believe that spending on public health is low in proportion to other health spending. Here we have a systematic review of studies reporting cost-benefit ratios (CBR) or return on investment (ROI) estimates for public health interventions. The stated aim of the paper is to demonstrate the false economy associated with cuts to public health spending. 52 titles were included from a search that identified 2957. The inclusion and exclusion criteria are not very clear, with some studies rejected on the basis of ‘poor generalisability to the UK’. There’s a bit too much subjectivity sneaking around in the methods for my liking.  Results for CBR and ROI estimates are presented according to local or national level and grouped by ‘specialism’. From all studies, the median CBR was 8.3 and the median ROI was 14.3. As we might have suspected, public health interventions are cost-saving in a big way. National health protection and legislative interventions offered the greatest return on investment. While there is wide variation in the results, all specialism groupings showed a positive return on average. I don’t doubt the truth of the study’s message – that cuts to public health spending are foolish. But the review doesn’t really demonstrate what the authors want it to demonstrate. We don’t know what (if any) disinvestment is taking place with respect to the interventions identified in the review. The results presented in the study represent a useful reference point for discussion and further analysis, but they aren’t a sufficient basis for supporting general increases in public health spending. That said, the study adds to an already resounding call and may help bring more attention to the issue.

Acceptable health and priority weighting: discussing a reference-level approach using sufficientarian reasoning. Social Science & Medicine Published 27th March 2017

In some ways, the moral principle of sufficiency is very attractive. It acknowledges a desire for redistribution from the haves to the have-nots and may also make for a more manageable goal than all-out maximisation. It may also be particularly useful in specific situations, such as evaluating health care for the elderly, for whom ‘full health’ is never achievable and not a meaningful reference point. This paper presents a discussion of the normative issues at play, drawing insights from the distributive justice literature. We’re reminded of the fair innings argument as a familiar sufficientarian flavoured allocation principle. The sufficientarian approach is outlined in contrast to egalitarianism and prioritarianism. Strict sufficientarian value weighting is not a good idea. If we suppose a socially ‘acceptable’ health state value of 0.7, such an approach would – for example – value an improvement from 0.69 to 0.71 for one person as infinitely more valuable than an improvement from 0.2 to 0.6 for the whole population. The authors go on to outline some more relaxed sufficiency weightings, whereby improvements below the threshold are attributed a value greater than 0 (though still less than those achieving sufficiency). The sufficientarian approach alone is (forgive me) an insufficient framework for the allocation of health care resources and cannot represent the kind of societal preferences that have been observed in the literature. Thus, hybrids are proposed. In particular, a sufficientarian-prioritarian weighting function is presented and the authors suggest that this may be a useful basis for priority setting. One can imagine a very weak form of the sufficientarian approach that corresponds to a prioritarian weighting function that is (perhaps) concave below the threshold and convex above it. Still, we have the major problem of identifying a level of acceptable health that is not arbitrary. The real question you need to ask yourself is this: do you really want health economists to start arguing about another threshold?

Emotions and scope effects in the monetary valuation of health. The European Journal of Health Economics [PubMed] Published 24th March 2017

It seems obvious that emotions could affect the value people attach to goods and services, but little research has been conducted with respect to willingness to pay for health services. This study considers the relationship between a person’s self-reported fear of being operated on and their willingness to pay for risk-reducing drug-eluting stents. A sample of 1479 people in Spain made a series of choices between bare-metal stents at no cost and drug-eluting stents with some out-of-pocket cost, alongside a set of sociodemographic questions and a fear of surgery Likert scale. Each respondent provided 8 responses with 4 different risk reductions and 2 different willingness to pay ‘bids’. The authors outline what they call a ‘cognitive-emotional random utility model’ including an ’emotional shift effect’. Four different models are presented to demonstrate the predictive value of the emotion levels interacting with the risk reduction levels. The sample was split roughly in half according to whether people reported high emotion (8, 9 or 10 on the fear Likert) or low emotion (<8). People who reported more fear of being operated on were willing to pay more for risk reductions, which is the obvious result. More interesting is that the high emotion group exhibited a lower sensitivity to scope – that is, there wasn’t much difference in their valuation of the alternative magnitudes of risk reduction. This constitutes a problem for willingness to pay estimates in this group as it may prevent the elicitation of meaningful values, and it is perhaps another reason why we usually go for collective approaches to health state valuation. The authors conclude that emotional response is a bias that needs to be corrected. I don’t buy this interpretation and would tend to the view that the bias that needs correcting here is that of the economist. Emotions may be a justifiable reflection of personality traits that ought to determine preferences, at least at the individual level. But I do agree with the authors that this is an interesting field for further research if only to understand possible sources of heterogeneity in health state valuation.

Credits