Thesis Thursday: Thomas Allen

On the third Thursday of every month we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Thomas Allen who graduated with a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

The impact of provider incentives on professionals and patients
Matt Sutton, William Whittaker
Repository link

Let’s dive straight in: what was the most important or overarching finding of your research?

My thesis focused on a large financial incentive scheme for UK GPs. So the thesis is a collection of UK studies, but I think the main findings can be generalised reasonably well.

Two of these studies actually looked at how the non-financial incentives of the scheme affected GPs, namely reputation and peer effects. I found reputation became more important, compared to revenue, a few years into the scheme. My explanation for this: reputation matters once you can observe performance benchmarks.

As for peer effects, the focus was on how practices react to their peer groups getting larger, this was caused by mergers in PCTs (groups of practices). You might expect peer effects to shrink when the group gets larger and this is what I found. Practice performance is also pulled down by poor peers more than it is pulled up by good peers. An analogy to merging a good classroom with a bad classroom is helpful to imagine.

There is quite a lot of variation (at GP level) in the amount of income that was linked to performance, 10-30% in most cases, so the third study exploits this variation. The size of this exposure to performance pay does affect GPs working lives – their job satisfaction, working hours, intentions to quit etc.

The final study was pretty novel as it linked patient reported quality with practice reported quality. It seemed to be the case that as practices improved on the incentivised areas of quality (e.g. blood pressure test) they got worse on the non-incentivised areas (communication).

What were the main methodologies that you used and which researchers’ work did your study most depend on?

It was a quantitative thesis so various regression methods were used. I’ll admit there was nothing particularly special or new with the methods used, they were standard methods but I think they were applied in interesting ways. For example, two studies linked existing datasets in new ways so I could answer questions which would have otherwise been impossible, probably. One method used which is not so common was the continuous difference in differences from the job satisfaction chapter. It’s been used before by David Card and Carol Propper. It can be used when you have a continuous treatment variable, instead of the typical treatment vs control situation. Everyone is treated but there is some exogenous factor deciding the amount of treatment.

I’m not sure there is one researcher that my study most depended on. The four different empirical chapters were influenced by slightly different literatures. Two big influences were systematic reviews of financial incentives (Scott et al. 2011) and of the scheme which I studied (Steel & Willems 2010). Both helped to identify areas where I could add to the existing literature.

What was the most surprising thing that you discovered; was there anything odd or unexpected?

Lots of theories would suggest an effect of pay for performance on job satisfaction and working lives. For example, large financial incentives should crowd out internal motivation and so reduce job satisfaction. Pay for performance appeals more to risk seeking individuals; those who are risk averse should feel uncomfortable as more income is linked to performance. Pay for performance can often result in wage dispersion, where incomes differ because some individuals perform better, this is usually linked to lower job satisfaction. A section of Chapter 6 is dedicated to these theories but I found no effect of pay for performance on GPs’ job satisfaction or working lives. Even specific areas you would expect to be affected weren’t, like satisfaction with choice of working methods or levels of autonomy.

This was certainly an unexpected result but I think still very interesting. I was able to publish this quite recently in Social Science & Medicine.

What was the biggest challenge that you encountered during your PhD, and did it change the direction of your research?

I started to answer this saying I didn’t have any big challenges but then a few came to me. I guess looking back they don’t seem as significant as they were at the time.

In the first few weeks I realised one of the studies from the PhD proposal couldn’t be done – basically I wanted to use PROMs to analyse a policy but had glossed over the difference between hip fractures and hip replacements, which seems very obvious now. I had to think of Plan B.

Plan B turned into Plan C around the end of my second year. I was going to try linking three datasets to measure the impact of pay for performance using administrative data, patient data and GP data. Imagine a Venn diagram of the overlapping samples from these three datasets. In the end the sample covered by all three was too small.

I’m pleased with how the thesis turned out, these challenges ended up improving the finished product.

Have you any words of wisdom for any researchers who might be embarking on a similar programme of research?

On this research area… The incentive scheme I focused on, the QOF, has been around for 12 years. If you have a new research question maybe someone else already tried it and it doesn’t work. Review the literature well and talk to those who have done work on the scheme. My internal examiner was a GP. She gave some great insight which would have been helpful at the start of the PhD not the end! So if you can, talk with those affected by the incentive or policy you are evaluating – it might not work in the way described in policy documents.

On PhDs generally… Choose your supervisors wisely – they are more than just a boss/manager, so try and find someone you think you can work with, not for. If you can, have a professor and a less senior person. Matt and Will were a great combo. In the end you might find you are sick of the PhD topic, so make sure you at least start off liking it. Don’t just pick it because it is the only one going. Try and do some extra work: teaching, collaborate with others, blogs. But make sure you gain from it in some way. Plan your time well at the start. You won’t stick to it, but at least you’ll know how far you are behind.


Paul Mitchell’s journal round-up for 26th December 2016

Every Monday (even if it’s Boxing Day here in the UK) our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Out-migration and attrition of physicians and dentists before and after EU accession (2003 and 2011): the case of Hungary. European Journal of Health Economics [PubMedPublished 2nd December 2016

Medical staff migration is an important cross-national policy issue given the international shortage of supply of doctors to meet healthcare demand. This study uses a large administrative survey collected in Hungary from 2004-2011 and focuses on the trends of medical doctors (GPs, specialists, dentists) since Hungary joined the EU in 2004 and the introduction of full freedom of movement between Hungary with Austria and Germany in 2011. The author conducted a time-to-event analysis with monthly collection of data on a person’s occupation used as a guide for outward-migration. A competing-risks model was used to also consider medical doctors exiting the profession, becoming inactive or dying. From the 18,266 medical doctors found in this sample over the nine year period, 12% migrated, 17% exited the profession and 14% became inactive. A five-fold increase in migration was seen when the restrictions on freedom of movement between Hungary and Austria/Germany were lifted, a worrying sign of brain drain from Hungary. For those who stayed but exited the profession, relative income is argued to have been a contributory factor, with incomes increasing by on average 40% in their new line of work (although this does not account for the “thank you money” received by doctors in Hungary for healthcare access). Generous maternity leave was argued to play a key role in absence from employment. A recognised limitation in this study is the inability to conduct robust analysis on the migration patterns of new medical graduates who are likely to be more prone to migration than their established colleagues (estimated to be 40% of all medical graduates in Hungary between 2007-2010 who migrated, before restrictions on freedom of movement between Austria and Germany were lifted). Nonetheless, the study still manages to shine a light on the external (competing against countries with larger economies) but also the internal (“attrition and feminisation of workforce”) challenges to national doctor staffing policy.

Does the proportion of pay linked to performance affect the job satisfaction of general practitioners? Social Science & Medicine [PubMedPublished 24th November 2016

The impact of pay for performance (P4P) on healthcare practice has been subject to much debate surrounding the pros and cons of incentives for medical staff to achieve specific goals. This study focuses on the impact that the introduction of the Quality and Outcomes Framework (QOF) for GPs in the UK in 2004 had on their subsequent job satisfaction. Job satisfaction for GPs is argued to be an important topic area due to it having an important role in retaining GPs and the quality of care they provide to their patients. Using linked data from the the GP Worklife Survey and the QOF, that rewards GPs performance based on clinical, organisation, additional services and patient experience indicators, across three time points (2004, 2005 and 2008), the authors model the relationship between P4P exposure (i.e. the proportion of income related to performance) and job satisfaction. Using a continuous difference-in-difference model with a random effects regression, the authors find that P4P exposure has no significant effect on job satisfaction after 1 and 4 years following the introduction of the QOF P4P system. The introduction of the QOF did lead to a large increase in GP life satisfaction; this is likely to be due to the large increase in average income for GPs following the introduction of QOF. The authors argue that their findings suggest GP job satisfaction is unlikely to be affected by changes in P4P exposure, so long as the final income the GP receives remains constant. Given the generous increases on GP final income from the initial QOF, it remains to be seen how generalisable these results would be to P4P systems that did not lead to such large increases in staff income.

Country-level cost-effectiveness thresholds: initial estimates and the need for further research. Value in Health [PubMed] Published 14th December 2016

National thresholds used to determine if a health intervention is cost-effective have been under scrutiny in the UK in recent years. It has been argued on the grounds of healthcare opportunity costs that the NICE £20,000-30,000 per QALY gained threshold is too high, with an estimate of £13,000 per QALY gain proposed instead. Until now, less attention has been paid to international cost-effectiveness thresholds recommended by the WHO, who have argued for a threshold between one and three times the GDP of a country. This study provides preliminary estimates of cost-effectiveness thresholds across a number of countries with varying levels of national income. Using estimates from the recent £13,000 per QALY gain threshold study in England, a ratio between the supply-side threshold with the consumption value of health was estimated and used as a basis to calculate other national thresholds. The authors use a range of income elasticity estimates for the value placed on a statistical life to take account of uncertainty around these values. The results suggest that even the lower end of the WHO recommended threshold range of 1x national GDP is likely to be an overestimate in most countries. It would appear something closer to 50% of GDP may be a better estimate, albeit with a great amount of uncertainty and variation between high and low income countries. The importance of these estimates according to the authors is that the application of the current WHO thresholds could lead to policies that reduce instead of increase population health. However, the threshold estimates from this study rely on a number of assumptions based on UK data that may not provide an accurate estimate when setting cost-effectiveness thresholds at an international level.


Alastair Canaway’s journal round-up for 31st October 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Ethical hurdles in the prioritization of oncology care. Applied Health Economics and Health Policy [PubMedPublished 21st October 2016

Recently between health economists, there has been significant scrutiny and disquiet directed towards the Cancer Drugs Fund with Professor Karl Claxton describing it as “an appalling, unfair use of NHS resources”. With the latest reorganization of the Cancer Drugs Fund in mind, this article examining the ethical issues surrounding prioritisation of cancer care was of particular interest. As all health economists will tell you, there is an opportunity cost with any allocation of scarce resources. Likewise, with prioritisation of specific disease groups, there may be equity issues with specific patients’ lives essentially being valued more greatly than those suffering other conditions. This article conducts a systematic review of the oncology literature to examine the ethical issues surrounding inequity in healthcare. The review found that public and political attention often focuses on ‘availability’ of pharmacological treatment in addition to factors that lead to good outcomes. The public and political focus on availability can have perverse consequences as highlighted by the Cancer Drugs Fund: resources are diverted towards availability and away from other more cost-effective areas, and in turn this may have had a detrimental effect on care for non-cancer patients. Additionally, by approving high cost, less cost-effective agents, strain will be placed upon health budgets and causing problems for existing cost-effectiveness thresholds. If prioritisation for cancer drugs is to be pursued then the authors suggest that the question of how to fund new therapies equitably will need to be addressed. Although the above issues will not be new to most, the paper is still worth reading as it: i) gives an overview of the different prioritisation frameworks used across Europe, ii) provides several suggestions for how, if prioritization is to be pursued, it can be done in a fairer manner rather than simply overriding typical HTA decision processes, iii) considers the potential legal consequences of prioritisation and iv) the impact of prioritisation on the sustainability of healthcare funding.

Doctor-patient differences in risk and time preferences: a field experiment. Journal of Health Economics Published 19th October 2016

The patient-doctor agency interaction, and associated issues due to asymmetrical information is something that was discussed often during my health economics MSc, but rarely during my day to day work. Despite being very familiar with supplier induced demand, differences in risk and time preferences in the patient-doctor dyad wasn’t something I’d considered in recent times. Upon reading, immediately, it is clear that if risk and time preferences do differ, then what is seen as the optimal treatment for the patient may be very different to that of the doctor. This may lead to poorer adherence to treatments and worse outcomes. This paper sought to investigate whether patients and their doctors had similar time and risk preferences using a framed field experiment with 300 patients and 67 doctors in Athens, Greece in a natural clinical setting. The authors claim to be the first to attempt this, and have three main findings: i) there were significant time preference differences between the patients and doctors – doctors discounted future health gains and financial outcomes less heavily than patients; ii) there were no significant differences in risk preferences for health with both doctors and patients being mildly risk averse; iii) there were however risk preference differences for financial impacts with doctors being more risk averse than patients. The implication of this paper is that there is potential for improvements in doctor-patient communication for treatments, and as agents for patients, doctors should attempts to gauge their patient’s preferences and attitudes before recommending treatment. For those who heavily discount the future it may be preferable to provide care that increases the short term benefits.

Hospital productivity growth in the English NHS 2008/09 to 2013/14 [PDF]. Centre for Health Economics Research Paper [RePEcPublished 21st October 2016

Although this is technically a ‘journal round-up’, this week I’ve chosen to include the latest CHE report as I think it is something which may be of wider interest to the AHEBlog community. Given limited resources, there is an unerring call for both productivity and efficiency gains within the NHS. The CHE report examines the extent to which NHS hospitals have improved productivity: have they made better use of their resources by increasing the number of patients they treat and the services they deliver for the same or fewer inputs. To assess productivity, the report uses established methods: Total Factor Productivity (TFP) which is the ratio of all outputs to all inputs. Growth in TFP is seen as being key to improving patient care with limited resources. The primary report finding was that TFP growth at the trust level exhibits ‘extraordinary volatility’. For example one year there maybe TFP growth followed by negative growth the next year, and then positive growth. The authors assert that much of the TFP growth measured is in fact implausible, and much of the changes are driven largely by nominal effects alongside some real changes. These nominal effects may be data entry errors or changes in accounting practices and data recording processes which results in changes to the timing of the recording of outputs and inputs. This is an important finding for research assessing productivity growth within the NHS. The TFP approach is an established methodology, yet as this research demonstrates, such methods do not provide credible measures of productivity at the hospital level. If hospital level productivity growth is to be measured credibly, then a new methodology will be required.