Thesis Thursday: Raymond Oppong

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Raymond Oppong who graduated with a PhD from the University of Birmingham. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Economic analysis alongside multinational studies
Supervisors
Sue Jowett, Tracy Roberts
Repository link
http://etheses.bham.ac.uk/7288/

What attracted you to studying economic evaluation in the context of multinational studies?

One of the first projects that I was involved in when I started work as a health economist was the Genomics to combat Resistance against Antibiotics in Community-acquired lower respiratory tract infections (LRTI) in Europe (GRACE) project. This was an EU-funded study aimed at integrating and coordinating the activities of physicians and scientists from institutions in 14 European countries to combat antibiotic resistance in community-acquired lower respiratory tract infections.

My first task on this project was to undertake a multinational costing study to estimate the costs of treating acute cough/LRTI in Europe. I faced quite a number of challenges including the lack of unit cost data across countries. Conducting a full economic evaluation alongside the interventional studies in GRACE also brought up a number of issues with respect to methods of analysis of multinational trials which needed to be resolved. The desire to understand and resolve some of these issues led me to undertake the PhD to investigate the implications of conducting economic evaluations alongside multinational studies.

Your thesis includes some case studies from a large multinational project. What were the main findings of your empirical work?

I used three main case studies for my empirical work. The first was an observational study aimed at describing the current presentation, investigation, treatment and outcomes of community-acquired lower respiratory tract infections and analysing the determinants of antibiotic use in Europe. The other 2 were RCTs. The first was aimed at studying the effectiveness of antibiotic therapy (amoxicillin) in community-acquired lower respiratory tract infections, whilst the second was aimed at assessing training interventions to improve antibiotic prescribing behaviour by general practitioners. The observational study was used to explore issues relating to costing and outcomes in multinational studies whilst the RCTs explored the various analytical approaches (pooled and split) to economic evaluation alongside multinational studies.

The results from the observational study revealed large variations in costs across Europe and showed that contacting researchers in individual countries was the most effective way of obtaining unit costs. Results from both RCTs showed that the choice of whether to pool or split data had an impact on the cost-effectiveness of the interventions.

What were the key analytical methods used in your analysis?

The overall aim of the thesis was to study the implications of conducting economic analysis alongside multinational studies. Specific objectives include: i) documenting challenges associated with economic evaluations alongside multinational studies, ii) exploring various approaches to obtaining and estimating unit costs, iii) exploring the impact of using different tariffs to value EQ-5D health state descriptions, iv) comparing methods that have been used to conduct economic evaluation alongside multinational studies and v) making recommendations to guide the design and conduct of future economic evaluations carried out alongside multinational studies.

A number of approaches were used to achieve each of the objectives. A systematic review of the literature identified challenges associated with economic evaluations alongside multinational studies. A four-stage approach to obtaining unit costs was assessed. The UK, European and country-specific EQ-5D value sets were compared to determine which is the most appropriate to use in the context of multinational studies. Four analytical approaches – fully pooled one country costing, fully pooled multicountry costing, fully split one country costing and fully split multicountry costing – were compared in terms of resource use, costs, health outcomes and cost-effectiveness. Finally, based on the findings of the study, a set of recommendations were developed.

You completed your PhD part-time while working as a researcher. Did you find this a help or a hindrance to your studies?

I must say that it was both a help and a hindrance. Working in a research environment was really helpful. There was a lot of support from supervisors and colleagues which kept me motivated. I might have not gotten this support if I was not working in a research/academic environment. However, even though some time during the week was allocated to the PhD, I had to completely put it on hold for long periods of time in order to deal with the pressures of work/research. Consequently, I always had to struggle to find my bearings when I got back to the PhD. I also spent most weekends working on the PhD especially when I was nearing submission.

On the whole, it should be noted that a part-time PhD requires a lot of time management skills. I personally had to go on time management courses which were really helpful.

What advice would you give to a health economist conducting an economic evaluation alongside a multinational study?

For a health economist conducting an economic evaluation alongside a multinational trial, it is important to plan ahead and understand the challenges that are associated with economic evaluations alongside multinational studies. A lot of the problems such as those related to the identification of unit costs can be avoided by ensuring adequate measures are put in place at the design stage of the study. An understanding of the various health systems of the countries involved in the study is important in order to make a judgement about the differences and similarities in resource use across countries. Decision makers are interested in results that can be applied to their jurisdiction; therefore it is important to adopt transparent methods e.g. state the countries that participated in the study, state the sources of unit costs and make it clear whether data from all countries (pooling) or from a subset (splitting) were used. To ensure that the results of the study are generalisable to a number of countries it may be advisable to present country-specific results and probably conduct the analysis from different perspectives.

Advertisements

Thesis Thursday: Thomas Allen

On the third Thursday of every month we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Thomas Allen who graduated with a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
The impact of provider incentives on professionals and patients
Supervisors
Matt Sutton, William Whittaker
Repository link
https://www.escholar.manchester.ac.uk/item/?pid=uk-ac-man-scw:296844

Let’s dive straight in: what was the most important or overarching finding of your research?

My thesis focused on a large financial incentive scheme for UK GPs. So the thesis is a collection of UK studies, but I think the main findings can be generalised reasonably well.

Two of these studies actually looked at how the non-financial incentives of the scheme affected GPs, namely reputation and peer effects. I found reputation became more important, compared to revenue, a few years into the scheme. My explanation for this: reputation matters once you can observe performance benchmarks.

As for peer effects, the focus was on how practices react to their peer groups getting larger, this was caused by mergers in PCTs (groups of practices). You might expect peer effects to shrink when the group gets larger and this is what I found. Practice performance is also pulled down by poor peers more than it is pulled up by good peers. An analogy to merging a good classroom with a bad classroom is helpful to imagine.

There is quite a lot of variation (at GP level) in the amount of income that was linked to performance, 10-30% in most cases, so the third study exploits this variation. The size of this exposure to performance pay does affect GPs working lives – their job satisfaction, working hours, intentions to quit etc.

The final study was pretty novel as it linked patient reported quality with practice reported quality. It seemed to be the case that as practices improved on the incentivised areas of quality (e.g. blood pressure test) they got worse on the non-incentivised areas (communication).

What were the main methodologies that you used and which researchers’ work did your study most depend on?

It was a quantitative thesis so various regression methods were used. I’ll admit there was nothing particularly special or new with the methods used, they were standard methods but I think they were applied in interesting ways. For example, two studies linked existing datasets in new ways so I could answer questions which would have otherwise been impossible, probably. One method used which is not so common was the continuous difference in differences from the job satisfaction chapter. It’s been used before by David Card and Carol Propper. It can be used when you have a continuous treatment variable, instead of the typical treatment vs control situation. Everyone is treated but there is some exogenous factor deciding the amount of treatment.

I’m not sure there is one researcher that my study most depended on. The four different empirical chapters were influenced by slightly different literatures. Two big influences were systematic reviews of financial incentives (Scott et al. 2011) and of the scheme which I studied (Steel & Willems 2010). Both helped to identify areas where I could add to the existing literature.

What was the most surprising thing that you discovered; was there anything odd or unexpected?

Lots of theories would suggest an effect of pay for performance on job satisfaction and working lives. For example, large financial incentives should crowd out internal motivation and so reduce job satisfaction. Pay for performance appeals more to risk seeking individuals; those who are risk averse should feel uncomfortable as more income is linked to performance. Pay for performance can often result in wage dispersion, where incomes differ because some individuals perform better, this is usually linked to lower job satisfaction. A section of Chapter 6 is dedicated to these theories but I found no effect of pay for performance on GPs’ job satisfaction or working lives. Even specific areas you would expect to be affected weren’t, like satisfaction with choice of working methods or levels of autonomy.

This was certainly an unexpected result but I think still very interesting. I was able to publish this quite recently in Social Science & Medicine.

What was the biggest challenge that you encountered during your PhD, and did it change the direction of your research?

I started to answer this saying I didn’t have any big challenges but then a few came to me. I guess looking back they don’t seem as significant as they were at the time.

In the first few weeks I realised one of the studies from the PhD proposal couldn’t be done – basically I wanted to use PROMs to analyse a policy but had glossed over the difference between hip fractures and hip replacements, which seems very obvious now. I had to think of Plan B.

Plan B turned into Plan C around the end of my second year. I was going to try linking three datasets to measure the impact of pay for performance using administrative data, patient data and GP data. Imagine a Venn diagram of the overlapping samples from these three datasets. In the end the sample covered by all three was too small.

I’m pleased with how the thesis turned out, these challenges ended up improving the finished product.

Have you any words of wisdom for any researchers who might be embarking on a similar programme of research?

On this research area… The incentive scheme I focused on, the QOF, has been around for 12 years. If you have a new research question maybe someone else already tried it and it doesn’t work. Review the literature well and talk to those who have done work on the scheme. My internal examiner was a GP. She gave some great insight which would have been helpful at the start of the PhD not the end! So if you can, talk with those affected by the incentive or policy you are evaluating – it might not work in the way described in policy documents.

On PhDs generally… Choose your supervisors wisely – they are more than just a boss/manager, so try and find someone you think you can work with, not for. If you can, have a professor and a less senior person. Matt and Will were a great combo. In the end you might find you are sick of the PhD topic, so make sure you at least start off liking it. Don’t just pick it because it is the only one going. Try and do some extra work: teaching, collaborate with others, blogs. But make sure you gain from it in some way. Plan your time well at the start. You won’t stick to it, but at least you’ll know how far you are behind.

Paul Mitchell’s journal round-up for 26th December 2016

Every Monday (even if it’s Boxing Day here in the UK) our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Out-migration and attrition of physicians and dentists before and after EU accession (2003 and 2011): the case of Hungary. European Journal of Health Economics [PubMedPublished 2nd December 2016

Medical staff migration is an important cross-national policy issue given the international shortage of supply of doctors to meet healthcare demand. This study uses a large administrative survey collected in Hungary from 2004-2011 and focuses on the trends of medical doctors (GPs, specialists, dentists) since Hungary joined the EU in 2004 and the introduction of full freedom of movement between Hungary with Austria and Germany in 2011. The author conducted a time-to-event analysis with monthly collection of data on a person’s occupation used as a guide for outward-migration. A competing-risks model was used to also consider medical doctors exiting the profession, becoming inactive or dying. From the 18,266 medical doctors found in this sample over the nine year period, 12% migrated, 17% exited the profession and 14% became inactive. A five-fold increase in migration was seen when the restrictions on freedom of movement between Hungary and Austria/Germany were lifted, a worrying sign of brain drain from Hungary. For those who stayed but exited the profession, relative income is argued to have been a contributory factor, with incomes increasing by on average 40% in their new line of work (although this does not account for the “thank you money” received by doctors in Hungary for healthcare access). Generous maternity leave was argued to play a key role in absence from employment. A recognised limitation in this study is the inability to conduct robust analysis on the migration patterns of new medical graduates who are likely to be more prone to migration than their established colleagues (estimated to be 40% of all medical graduates in Hungary between 2007-2010 who migrated, before restrictions on freedom of movement between Austria and Germany were lifted). Nonetheless, the study still manages to shine a light on the external (competing against countries with larger economies) but also the internal (“attrition and feminisation of workforce”) challenges to national doctor staffing policy.

Does the proportion of pay linked to performance affect the job satisfaction of general practitioners? Social Science & Medicine [PubMedPublished 24th November 2016

The impact of pay for performance (P4P) on healthcare practice has been subject to much debate surrounding the pros and cons of incentives for medical staff to achieve specific goals. This study focuses on the impact that the introduction of the Quality and Outcomes Framework (QOF) for GPs in the UK in 2004 had on their subsequent job satisfaction. Job satisfaction for GPs is argued to be an important topic area due to it having an important role in retaining GPs and the quality of care they provide to their patients. Using linked data from the the GP Worklife Survey and the QOF, that rewards GPs performance based on clinical, organisation, additional services and patient experience indicators, across three time points (2004, 2005 and 2008), the authors model the relationship between P4P exposure (i.e. the proportion of income related to performance) and job satisfaction. Using a continuous difference-in-difference model with a random effects regression, the authors find that P4P exposure has no significant effect on job satisfaction after 1 and 4 years following the introduction of the QOF P4P system. The introduction of the QOF did lead to a large increase in GP life satisfaction; this is likely to be due to the large increase in average income for GPs following the introduction of QOF. The authors argue that their findings suggest GP job satisfaction is unlikely to be affected by changes in P4P exposure, so long as the final income the GP receives remains constant. Given the generous increases on GP final income from the initial QOF, it remains to be seen how generalisable these results would be to P4P systems that did not lead to such large increases in staff income.

Country-level cost-effectiveness thresholds: initial estimates and the need for further research. Value in Health [PubMed] Published 14th December 2016

National thresholds used to determine if a health intervention is cost-effective have been under scrutiny in the UK in recent years. It has been argued on the grounds of healthcare opportunity costs that the NICE £20,000-30,000 per QALY gained threshold is too high, with an estimate of £13,000 per QALY gain proposed instead. Until now, less attention has been paid to international cost-effectiveness thresholds recommended by the WHO, who have argued for a threshold between one and three times the GDP of a country. This study provides preliminary estimates of cost-effectiveness thresholds across a number of countries with varying levels of national income. Using estimates from the recent £13,000 per QALY gain threshold study in England, a ratio between the supply-side threshold with the consumption value of health was estimated and used as a basis to calculate other national thresholds. The authors use a range of income elasticity estimates for the value placed on a statistical life to take account of uncertainty around these values. The results suggest that even the lower end of the WHO recommended threshold range of 1x national GDP is likely to be an overestimate in most countries. It would appear something closer to 50% of GDP may be a better estimate, albeit with a great amount of uncertainty and variation between high and low income countries. The importance of these estimates according to the authors is that the application of the current WHO thresholds could lead to policies that reduce instead of increase population health. However, the threshold estimates from this study rely on a number of assumptions based on UK data that may not provide an accurate estimate when setting cost-effectiveness thresholds at an international level.

Credits