Thesis Thursday: Matthew Quaife

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Matthew Quaife who has a PhD from the London School of Hygiene and Tropical Medicine. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Using stated preferences to estimate the impact and cost-effectiveness of new HIV prevention products in South Africa
Supervisors
Fern Terris-Prestholt, Peter Vickerman
Repository link
http://researchonline.lshtm.ac.uk/4646708

Stated preferences for what?

Our main study looked at preferences for new HIV prevention products in South Africa – estimating the uptake and cost-effectiveness of multi-purpose prevention products, which protect against HIV, pregnancy and STIs. You’ll notice that condoms do this, so why even bother? Condom use needs both partners to agree (for the duration of a given activity) and, whilst female partners tend to prefer condom-protected sex, there is lots of evidence that male partners – who also have greater bargaining power in many contexts – do not.

Oral pre-exposure prophylaxis (PrEP), microbicide gels, and vaginal rings are new products which prevent HIV infection. More importantly, they are female-initiated and can generally be used without a male partner’s knowledge. But trials and demonstration projects among women at high risk of HIV in sub-Saharan Africa have shown low levels of uptake and adherence. We used a DCE to inform the development of attractive and usable profiles for these products, and also estimate how much additional demand – and therefore protection – would be gained from adding contraceptive or STI-protective attributes.

We also elicited the stated preferences of female sex workers for client risk, condom use, and payments for sex. Sex workers can earn more for risky unprotected sex, and we used a repeated DCE to predict risk compensation (i.e. how much condom use would change) if they were to use HIV prevention products.

What did you find most influenced people’s preferences in your research?

Unsurprisingly for products, HIV protection was most important to people, followed by STI and then pregnancy protection. But digging below these averages with a latent class analysis, we found some interesting variation within female respondents: over a third were not concerned with HIV protection at all, instead strongly caring about pregnancy and STI protection. Worryingly, these were more likely to be respondents from high-incidence adolescent and sex worker groups. The remainder of the sample overwhelmingly chose based on HIV protection.

In the second sex worker DCE, we found that using a new HIV prevention product made condoms become less important and price more important. We predict that the price premium for unprotected sex would reduce by two thirds, and the amount of condomless sex would double. This is an interesting labour market/economic finding, but – if true – also has real public health implications. Since economic changes mean sex workers move from multi-purpose condoms to single-purpose products which need high levels of adherence, we thought this would be interesting to model.

How did you use information about people’s preferences to inform estimates of cost-effectiveness?

In two ways. First, we used simple uptake predictions from DCEs to parameterise an HIV transmission model, allowing for condom substitution uptake to vary by condom users and non-users (it was double in the latter). We were also able to model the potential uptake of multipurpose products which don’t exist yet – e.g. a pill protecting from HIV and pregnancy. We predict that this combination, in particular, would double uptake among high-risk young women.

Second, we predicted risk compensation among sex workers who chose new products instead of condoms. We were also able to calculate the price elasticity of supply of unprotected sex, which we built into a dynamic transmission model as a determinant of behaviour.

Can discrete choice experiments accurately predict the kinds of behaviours that you were looking at?

To be honest, when I started the PhD I was really sceptical – and I still am to an extent. But two things make me think DCEs can be useful in predicting behaviours.

First is the data. We published a meta-analysis of how well DCEs predict real-world health choices at an individual level. We only found six studies with individual-level data, but these showed DCEs predict with an 88% sensitivity but just a 34% specificity. If a DCE says you’ll do something, you more than likely will – which is important for modelling heterogeneity in uptake. We desperately need more studies following up DCE participants making real-world choices.

Second is the lack of alternative inputs. Where products are new and potential users are inexperienced, modellers pick an uptake number/range and hope for the best. Where we don’t know efficacy, we may assume that uptake and efficacy are linearly related – but they may not be (e.g. if proportionately more people use a 95% effective product than a 45% effective one). Instead, we might assume uptake and efficacy are independent, but that might sound even less realistic. I think that DCEs can tell us something about these behaviours that are useful for the parameters and structures of models, even if they are not perfect predictors.

Your tread the waters of infectious disease modelling in your research – was the incorporation of economic factors a challenge?

It was pretty tricky, though not as challenging as building the simple dynamic transmission model as a first exposure to R. In general, behaviours are pretty crudely modelled in transmission models, largely due to assumptions like random mixing and other population-level dynamics. We made a simple mechanistic model of sex work based on the supply elasticities estimated in the DCE, and ran a few scenarios, each time estimating the impact of prevention products. We simulated the price of unprotected sex falling and quantity rising as above, but also overlaid a few behavioural rules (e.g. Camerer’s constant income hypothesis) to simulate behavioural responses to a fall in overall income. Finally, we thought about competition between product users and non-users, and how much the latter may be affected by the market behaviours of the former. Look out for the paper at Bristol HESG!

How would you like to see research build on your work to improve HIV prevention?

I did a public engagement event last year based on one statistic: if you are a 16-year old girl living in Durban, you have an 80% lifetime risk of acquiring HIV. I find it unbelievable that, in 2018, when millions have been spent on HIV prevention and we have a range of interventions that can prevent HIV, incidence among some groups is still so dramatically and persistently high.

I think research has a really important role in understanding how people want to protect themselves from HIV, STIs, and pregnancy. In addition to highlighting the populations where interventions will be most cost-effective, we show that variation in preferences drives impact. I hope we can keep banging the drum to make attractive and effective options available to those at high risk.

Paul Mitchell’s journal round-up for 25th December 2017

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Consensus-based cross-European recommendations for the identification, measurement and valuation of costs in health economic evaluations: a European Delphi study. European Journal of Health Economics [PubMedPublished 19th December 2017

The primary aim of this study was to develop guidelines for costing in economic evaluation studies conducted across more than one European country. The starting point of the societal perspective as the benchmark for costing was not entirely obvious from the abstract, where this broadest approach to costing is not recommended uniformly across all European countries. Recommendations following this starting point looked at the identification, measurement and valuation of resource use, discount rate and discounting of future costs. A three-step Delphi study was used to gain consensus on what should be included in an economic evaluation from a societal perspective, based initially on findings from a review of costing methodologies adopted across European country-specific guidelines. Consensus required at least two thirds (67%) agreement across those participating in the Delphi study at all 3 stages. Where no agreement was reached after the three stages, a panel of four of the co-authors made a final decision on what should be recommended. In total, 26 of the 110 invited to participate completed at least one Delphi round, with all Delphi rounds having at least 16 participants. It remains unclear to me if 16 for a Delphi round is sufficient to reach a European wide consensus on costing methodologies. There were a number of key areas where no consensus was reached (e.g. including costs unrelated to the intervention, measurement of resource use and absenteeism, and valuation of opportunity costs of patient time and informal care), so the four-strong author panel had a leading role on some of the main recommendations. Notwithstanding the limitations associated with the reference perspective taken and sample for the Delphi study and panel, the paper provides a useful illustration of the different approaches to costing across European countries. It also provides a good coverage of costing issues that need to be explained in detail in economic evaluations to allow for clear understanding of methods used and the underpinning rationale for those decisions where a choice is required on the costing methodology applied.

A (five-)level playing field for mental health conditions?: exploratory analysis of EQ-5D-5L derived utility values. Quality of Life Research [PubMedPublished 16th December 2017

The UK health economics community has been reeling from the decision made earlier this year by UK guidelines developer, the National Institute for Health and Care Excellence (NICE), who recommended to not adopt the new population values developed for the EQ-5D-5L version when calculating QALYs and instead rely on a crosswalk of the values developed over 20 years ago for the 3 level EQ-5D version. This paper provides a timely comparison of how these two value sets perform for the EQ-5D-5L descriptive system in patient groups with mental health conditions, groups often thought to be disadvantaged by the physical health functioning focus of the EQ-5D descriptive system. Using baseline data from three trials, the authors find that the new utility values produce a higher mean EQ-5D score of 0.08 compared to the old crosswalk values, with a 0.225 difference for those reporting extreme problems with the anxiety/depression dimension on EQ-5D. Although, the authors of this study highlight using these new values would increase cost per QALY results in this sample using scenario analysis, when improvements are in the depression/anxiety category only, such improvements are relatively better than across the whole EQ-5D-5L descriptive system due to the relative additional value placed on the anxiety/depression dimension in the new values. This paper makes for interesting reading and one that NICE should take into consideration when reviewing their decision on this issue next year. Although I would disagree with the authors when they state that this study would be a primary reason for revising the NICE cost-effectiveness threshold (more compelling arguments for this elsewhere in my view), it does clearly highlight the influence of the choice of descriptive system and the values used in the outcomes produced for economic analysis such as QALYs, even when the two descriptive systems in question (EQ-5D-3L and EQ-5D-5L) are roughly the same.

What characteristics of nursing homes are most valued by customers? A discrete choice experiment with residents and family members. Value in Health Published 1st December 2017

Our final paper for review in 2017 looks at the characteristics that are of most importance to individuals and their family members when it comes to nursing home provision. The authors conducted a valuation exercise using a discrete choice experiment (DCE) to calculate the relative importance of the attributes contained on the Consumer Choice Index-Six Dimension (CCI-6D), a measure developed to assess the quality of nursing home care across 3 levels on six domains: 1. level of time care staff spent with residents; 2. homeliness of shared spaces; 3. homeliness of room setup; 4. access to outside and garden; 5. frequency of meaningful activities; and 6. flexibility with care routines. Those who lived in a nursing home for at least a year with low levels of cognitive impairment completed the DCE themselves, whereas family members were asked to proxy for their close relative with more severe cognitive impairment. 126 residents and 416 family member proxies completed the DCE comparisons of nursing homes with different qualities in these six areas. The results of the DCE show differences in preferences across the two groups. Although similar importance is placed on some dimensions across both groups (i.e. “homeliness of room set up” ranked highly, whereas “frequency of meaningful activities” ranked lower), residents value access to outside and garden four times as much as the family proxies do (second most important dimension for residents, lowest for family proxies), family members value level of time care staff spent with residents twice as much as residents themselves (most important attribute for family proxies, third most important for residents). Although residents in both groups may have important differences in characteristics that might explain some of this difference, it is probably a good time of year to remember family preferences may be inconsistent with individuals within them, so make sure to take account of this variation when preparing those Christmas dinners.

Happy holidays all.

Credits

Thesis Thursday: Caroline Vass

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Caroline Vass who has a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Using discrete choice experiments to value benefits and risks in primary care
Supervisors
Katherine Payne, Stephen Campbell, Daniel Rigby
Repository link
https://www.escholar.manchester.ac.uk/uk-ac-man-scw:295629

Are there particular challenges associated with asking people to trade-off risks in a discrete choice experiment?

The challenge of communicating risk in general, not just in DCEs, was one of the things which drew me to the PhD. I’d heard a TED talk discussing a study which asked people’s understanding of weather forecasts. Although most people think they understand a simple statement like “there’s a 30% chance of rain tomorrow”, few people correctly interpreted that as meaning it will rain 30% of the days like tomorrow. Most interpret it to mean there will be rain 30% of the time or in 30% of the area.

My first ever publication was reviewing the risk communication literature, which confirmed our suspicions; even highly educated samples don’t always interpret information as we expect. Therefore, testing if the communication of risk mattered when making trade-offs in a DCE seemed a pretty important topic and formed the overarching research question of my PhD.

Most of your study used data relating to breast cancer screening. What made this a good context in which to explore your research questions?

All women are invited to participate in breast screening (either from a GP referral or at 47-50 years old) in the UK. This makes every woman a potential consumer and a potential ‘patient’. I conducted a lot of qualitative research to ensure the survey text was easily interpretable, and having a disease which many people had heard of made this easier and allowed us to focus on the risk communication formats. My supervisor Prof. Katherine Payne had also been working on a large evaluation of stratified screening which made contacting experts, patients and charities easier.

There are also national screening participation figures so we were able to test if the DCE had any real-world predictive value. Luckily, our estimates weren’t too far off the published uptake rates for the UK!

How did you come to use eye-tracking as a research method, and were there any difficulties in employing a method not widely used in our field?

I have to credit my supervisor Prof. Dan Rigby with planting the seed and introducing me to the method. I did a bit of reading into what psychologists thought you could measure using eye-movements and thought it was worth further investigation. I literally found people publishing with the technology at our institution and knocked on doors until someone would let me use it! If the University of Manchester didn’t already have the equipment, it would have been much more challenging to collect these data.

I then discovered the joys of lab-based work which I think many health economists, fortunately, don’t encounter in their PhDs. The shared bench, people messing with your experiment set-up, restricted lab time which needs to be booked weeks in advance etc. I’m sure it will all be worth it… when the paper is finally published.

What are the key messages from your research in terms of how we ought to be designing DCEs in this context?

I had a bit of a null-result on the risk communication formats, where I found it didn’t affect preferences. I think looking back that might have been with the types of numbers I was presenting (5%, 10%, 20% are easier to understand) and maybe people have a lot of knowledge about the risks of breast screening. It certainly warrants further research to see if my finding holds in other settings. There is a lot of support for visual risk communication formats like icon arrays in other literatures and their addition didn’t seem to do any harm.

Some of the most interesting results came from the think-aloud interviews I conducted with female members of the public. Although I originally wanted to focus on their interpretation of the risk attributes, people started verbalising all sorts of interesting behaviour and strategies. Some of it aligned with economic concepts I hadn’t thought of such as feelings of regret associated with opting-out and discounting both the costs and health benefits of later screens in the programme. But there were also some glaring violations, like ignoring certain attributes, associating cost with quality, using other people’s budget constraints to make choices, and trying to game the survey with protest responses. So perhaps people designing DCEs for benefit-risk trade-offs specifically or in healthcare more generally should be aware that respondents can and do adopt simplifying heuristics. Is this evidence of the benefits of qualitative research in this context? I make that argument here.

Your thesis describes a wealth of research methods and findings, but is there anything that you wish you could have done that you weren’t able to do?

Achieved a larger sample size for my eye-tracking study!