On the third Thursday of every month we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Thomas Allen who graduated with a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.
Title
The impact of provider incentives on professionals and patients
Supervisors
Matt Sutton, William Whittaker
Repository link
https://www.escholar.manchester.ac.uk/item/?pid=uk-ac-man-scw:296844
Let’s dive straight in: what was the most important or overarching finding of your research?
My thesis focused on a large financial incentive scheme for UK GPs. So the thesis is a collection of UK studies, but I think the main findings can be generalised reasonably well.
Two of these studies actually looked at how the non-financial incentives of the scheme affected GPs, namely reputation and peer effects. I found reputation became more important, compared to revenue, a few years into the scheme. My explanation for this: reputation matters once you can observe performance benchmarks.
As for peer effects, the focus was on how practices react to their peer groups getting larger, this was caused by mergers in PCTs (groups of practices). You might expect peer effects to shrink when the group gets larger and this is what I found. Practice performance is also pulled down by poor peers more than it is pulled up by good peers. An analogy to merging a good classroom with a bad classroom is helpful to imagine.
There is quite a lot of variation (at GP level) in the amount of income that was linked to performance, 10-30% in most cases, so the third study exploits this variation. The size of this exposure to performance pay does affect GPs working lives – their job satisfaction, working hours, intentions to quit etc.
The final study was pretty novel as it linked patient reported quality with practice reported quality. It seemed to be the case that as practices improved on the incentivised areas of quality (e.g. blood pressure test) they got worse on the non-incentivised areas (communication).
What were the main methodologies that you used and which researchers’ work did your study most depend on?
It was a quantitative thesis so various regression methods were used. I’ll admit there was nothing particularly special or new with the methods used, they were standard methods but I think they were applied in interesting ways. For example, two studies linked existing datasets in new ways so I could answer questions which would have otherwise been impossible, probably. One method used which is not so common was the continuous difference in differences from the job satisfaction chapter. It’s been used before by David Card and Carol Propper. It can be used when you have a continuous treatment variable, instead of the typical treatment vs control situation. Everyone is treated but there is some exogenous factor deciding the amount of treatment.
I’m not sure there is one researcher that my study most depended on. The four different empirical chapters were influenced by slightly different literatures. Two big influences were systematic reviews of financial incentives (Scott et al. 2011) and of the scheme which I studied (Steel & Willems 2010). Both helped to identify areas where I could add to the existing literature.
What was the most surprising thing that you discovered; was there anything odd or unexpected?
Lots of theories would suggest an effect of pay for performance on job satisfaction and working lives. For example, large financial incentives should crowd out internal motivation and so reduce job satisfaction. Pay for performance appeals more to risk seeking individuals; those who are risk averse should feel uncomfortable as more income is linked to performance. Pay for performance can often result in wage dispersion, where incomes differ because some individuals perform better, this is usually linked to lower job satisfaction. A section of Chapter 6 is dedicated to these theories but I found no effect of pay for performance on GPs’ job satisfaction or working lives. Even specific areas you would expect to be affected weren’t, like satisfaction with choice of working methods or levels of autonomy.
This was certainly an unexpected result but I think still very interesting. I was able to publish this quite recently in Social Science & Medicine.
What was the biggest challenge that you encountered during your PhD, and did it change the direction of your research?
I started to answer this saying I didn’t have any big challenges but then a few came to me. I guess looking back they don’t seem as significant as they were at the time.
In the first few weeks I realised one of the studies from the PhD proposal couldn’t be done – basically I wanted to use PROMs to analyse a policy but had glossed over the difference between hip fractures and hip replacements, which seems very obvious now. I had to think of Plan B.
Plan B turned into Plan C around the end of my second year. I was going to try linking three datasets to measure the impact of pay for performance using administrative data, patient data and GP data. Imagine a Venn diagram of the overlapping samples from these three datasets. In the end the sample covered by all three was too small.
I’m pleased with how the thesis turned out, these challenges ended up improving the finished product.
Have you any words of wisdom for any researchers who might be embarking on a similar programme of research?
On this research area… The incentive scheme I focused on, the QOF, has been around for 12 years. If you have a new research question maybe someone else already tried it and it doesn’t work. Review the literature well and talk to those who have done work on the scheme. My internal examiner was a GP. She gave some great insight which would have been helpful at the start of the PhD not the end! So if you can, talk with those affected by the incentive or policy you are evaluating – it might not work in the way described in policy documents.
On PhDs generally… Choose your supervisors wisely – they are more than just a boss/manager, so try and find someone you think you can work with, not for. If you can, have a professor and a less senior person. Matt and Will were a great combo. In the end you might find you are sick of the PhD topic, so make sure you at least start off liking it. Don’t just pick it because it is the only one going. Try and do some extra work: teaching, collaborate with others, blogs. But make sure you gain from it in some way. Plan your time well at the start. You won’t stick to it, but at least you’ll know how far you are behind.
