Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.
Expertise versus Bias in Evaluation: Evidence from the NIH. American Economic Journal: Applied Economics. Published April 2017.
As an academic’s career progresses, she learns two things: patience and learning to deal with rejection. Getting a paper accepted by a top journal is hard. Obtaining funding for what seems like a good idea similarly so. We sometimes convince ourselves that the system is rigged, or at least biased. Research funding bodies may make poor decisions. This paper considers this question in great deal. While reviewers may have an informational advantage that allows them to assess quality, they may also be biased towards projects in their own domain of expertise. More funding for health economics blogs! To assess this, this paper examines 100,000 applications to the US National Institutes of Health. The proximity of the reviewer to the subject area of the application is judged by the number of times the reviewer has cited the work of the applicant. Quality is judged by the number of publications and citations the research produces – an attempt is made to adapt this to judge unfunded work. The principle finding is that reviewers are both more informed and more biased about work in their own field. Each additional permanent reviewer in a applicant’s area is estimated to increase the chance of funding by 2.2 percent, an equivalent effect to increasing quality by one quarter standard deviation. These effects seem small, as the author notes, and what strikes me is how little variation these measures in explain in funding decisions. Perhaps I will find some solace in the fact that there is quite a lot of apparent randomness in what gets funded. Nevertheless, the author suggests that the findings suggest that by trying to reduce bias by using impartial reviewers, the ability to judge quality will also decline.
Long-term effects of youth unemployment on mental health: does an economic crisis make a difference? Journal of Epidemiology and Community Health. [PubMed] Published April 2017.
Unemployment is related to mental health issues. The effect is appears to be particularly acute among young people for whom the transition to adult life can be difficult. Indeed, at this vulnerable period young people also transition from youth to adult mental health services, which breaks their continuity of care. Many become lost in the system. Services in many areas are being redesigned in light of this. This paper asks if the effect of unemployment on youth mental health is different depending on the economic conditions. Do period of high unemployment nationally exacerbate the effects of becoming unemployed? Surprisingly, the paper concludes, no, there is no difference. I say ‘surprisingly’ since I cannot recall finding a paper in this area or one that has featured on this blog with a negative finding. The analyses seem careful, and the authors concentrate on the magnitude of the effects, rather than statistical significance. Large sample sizes are required for adequate power to test a hypothesis on an interaction; this study does have a large sample size. The interactive effect is likely to be very small, not necessarily non-existent. But in comparison with the large effects of unemployment on youth mental health in general, the effect of economic conditions is of little importance. Nevertheless, Simpson’s paradox may rear its head here: during times of high unemployment, the cohort of the unemployed will be different. If those who only become unemployed during economic downturns have lower risk of mental health issues, then this may attenuate the estimated effect of unemployment on mental health. This issue is not addressed unfortunately, but I don’t want that to detract from a sensible use of statistics.
The Distortionary Effects of Incentives in Government: Evidence from China’s ‘Death Ceiling’ Program. American Economic Journal: Applied Economics. [RePEc] Published April 2017.
Targets and incentives to achieve those targets can distort the actions of agents. This is especially true of difficult to observe outcomes. People may be more inclined to manipulate the data than to actually achieve the target. Gaming and other similar behaviours have been noted in health services, for example. This article examines a policy in China designed to reduce the high rates of accidental deaths. In 2004 the State Administration of Work Safety announced that provinces would have to reduce their rate of accidental deaths by 2.5% per year. The provinces were set a so-called ‘death ceiling’. In 2012, the policy was declared a success; accidental death rates had come down by 45% since 2005. But further examination of the data, which were made publicly available in the state newspaper the People’s Daily, suggests this may not be the case. First of all, there was a sharp discontinuity of accidental deaths right below the death ceiling. This discontinuity was not consistent with a continuous variable. Provinces had much discretion about how to achieve the reductions. Those that used significant incentives for local officials were more likely to be successful. The authors also consider why, if the data were manipulated, deaths weren’t made to look significantly below the death ceiling rather than just below the death ceiling. They speculate that this would have the effect of making next year’s death ceiling even lower and more difficult to achieve. This paper provides a nice narrative that adds to our understanding of the perverse effects of incentives. For health services this is important. For many of the difficult to observe outcomes, like patient health, merely incentivising doctors and hospitals to improve may have little actual benefit.
Credits
