On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Martin Eden who has a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.
Title
Evaluating the economic benefits of complex interventions: accounting for non-health effects
Supervisors
Katherine Payne, Graeme Black, Gavin Daker-White
Repository link
https://www.research.manchester.ac.uk/portal/en/theses/evaluating-the-economic-benefits-of-complex-interventions-accounting-for-nonhealth-effects(ee0d6399-2b92-4a27-bac7-14e5f3ac8992).html
What types of complex interventions did you consider in your research?
I was specifically interested in the provision of information from genomic tests. Before I started my PhD, I was working in the Manchester Centre for Genomic Medicine on projects involving inherited eye conditions. New tests had been developed within the Centre which allowed families to get diagnostic and prognostic information about their eye conditions which wasn’t possible just a few years ago. Often there was no treatment available to these families. I started to understand, though, how useful it was for them to finally be able to get this diagnostic information. I also saw how, for an increasing number of conditions, genomic test information could form part of a complex precision medicine intervention with the potential to improve health.
Katherine, my line manager (and soon-to-be PhD supervisor), encouraged me to apply for an NIHR Doctoral Research Fellowship so that we could start to look at some issues around the economic evaluation of complex interventions which produce outcomes beyond health. For my mixed-methods PhD, I used hypothetical scenarios involving genomic tests for different conditions where a range of health and non-health outcomes could be realised.
How did you identify the relevant types of non-health effects?
There was already a lot of published qualitative work about the provision of clinical genetics services for many types of familial conditions. In these studies, people had recounted their experiences of seeking and receiving diagnostic information. Some studies had elicited views on genetic testing from those with no prior experience of inherited conditions. I synthesised evidence from these published qualitative papers to identify relevant types of non-health effects.
The specific approach I used for the qualitative evidence synthesis is called meta-ethnography. It involves translating findings from different studies into one another and can be used to generate new theories beyond those put forward in the original papers. It allowed me to define a novel taxonomy of non-health value deriving from genomic-based diagnostic information.
I used in-depth interviews involving people with and without previous experience of genetic conditions to check that the taxonomy made sense and that it included all relevant types of value.
How did you develop your approach to preference elicitation?
I began by using face-to-face interviews in which I asked people to do a choice-based task based on hypothetical scenarios designed around the taxonomy of non-health value. This helped me to construct a discrete choice experiment using health and non-health attributes. I was lucky being based in the Manchester Centre for Health Economics where we have a lot of expertise in running DCEs; my colleagues Stuart Wright and Caroline Vass were really helpful in this respect.
A key challenge was to provide DCE participants with sufficient background information on genomics and about how we measure health status in economic evaluation so that they could make meaningful choices. I produced a five-minute animated training video with narration and tested it, along with the DCE, using think-aloud interviews. Once I was happy with everything, I piloted the survey using a larger online sample, including tasks to help validate DCE answers. I looked at how people had completed tasks, how long they had spent watching the training video and how quickly they finished the survey. It all seemed to work well and so it was now ready to be completed by 1000 people, representative of the UK population.
To what extent are people willing to trade away health for non-health outcomes?
On average, the trade-offs were pretty big. For the three non-health attributes in the DCE, I calculated ‘willingness-to-trade health’ values. The amount of health that people would give up for gains in each of the non-health outcomes was comparable to published estimates of the magnitude of change in EQ-5D scores which would force people to seek medical attention.
People would trade most health – around 0.13 units on the EQ-5D tariff scale – if the genomic test information improved their ability to make important life decisions. Health would also be given up if diagnostic information could be of use to other people like close family members. If the information could reduce uncertainty – by providing a name for their condition, for example – people would be willing to trade a substantial amount of health for this intrinsic value of knowing.
Do different people value different attributes?
Yes, the qualitative work I did leading up to the DCE had already indicated that individual-specific situations, attitudes, and other unseen factors might influence how test information would be valued. For example, some people – like myself – would be very reluctant to have a genomic test under any circumstances. Whereas for others, any additional information about their health would always be sought no matter what.
To try to get an understanding of this preference heterogeneity, I did some sub-group analyses stratifying the DCE data by factors that might have influenced responses, like whether people had children or if they had previously had a genetic test. Those who had actually undergone testing were especially willing to give up health for non-health value in the hypothetical scenario.
I also used latent class analysis to explore the influence of unseen variables on choice behaviour. I was able to identify three classes of respondents, which included a sub-group who, in general, did not really value testing at all. Interestingly, the largest class of respondents placed more value on improved non-health outcomes over health gains despite being more likely to have below-average self-reported EQ-5D scores.
What do your findings imply for the allocation of resources in genomic testing?
Resource allocation decisions should be made in light of the fact that non-health outcomes are important in this context. This research addresses key policy recommendations relating to how the true value of complex interventions with a genomic-based diagnostic component will be underestimated using current decision-making frameworks.
I have been able to quantify the relative importance of non-health outcomes from genomic testing as valued by users and funders of the NHS. This has provided the basis for subsequent work to look at how the evaluative space used in cost-effectiveness analysis could be refined to take into account consequences which currently go unmeasured.
