Thesis Thursday: Anna Heath

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Anna Heath who has a PhD from the University College London. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Bayesian computations for value of information measures using Gaussian processes, INLA and Moment Matching
Supervisors
Gianluca Baio, Ioanna Manolopoulou
Repository link
http://discovery.ucl.ac.uk/id/eprint/10050229

Why are new methods needed for value of information analysis?

Value of Information (VoI) has been around for a really long time – it was first mentioned in a book published in 1959! More recently, it has been suggested that VoI methods can be used in health economics to direct and design future research strategies. There are several different concepts in VoI analysis and each of these can be used to answer different questions. The VoI measure with the most potential calculates the economic benefit of collecting additional data to inform a health economic model (known as the EVSI). The EVSI can be compared with the cost of collecting data and allow us to make sure that our clinical research is “cost-effective”.

The problem is that, mathematically, VoI measures are almost impossible to calculate, so we have to use simulation. Traditionally, these simulation methods have been very slow (in my PhD, one example took over 300 days to compute 10 VoI measures) so we need simulation methods that speed up the computation significantly before VoI can be used for decisions about research design and funding.

Do current EVPPI and EVSI estimation methods give different results?

For most examples, the current estimation methods give similar results but the computational time to obtain these results differs significantly. Since starting my PhD, different estimation methods for the EVPPI and the EVSI have been published. The difference between these methods are the assumptions and the ease of use. The results seem to be pretty stable for all the different methods, which is good!

The EVPPI determines which model parameters have the biggest impact on the cost-effectiveness of the different treatments. This is used to direct possible avenues of future research, i.e. we should focus on gaining more information about parameters with a large impact on cost-effectiveness. The EVPPI is calculated based only on simulations of the model parameters so the number of methods for EVPPI calculation is quite small. To calculate the EVSI, you need to consider how to collect additional data, through a clinical trial, observational study etc, so there is a wider range of available methods.

How does the Gaussian process you develop improve EVPPI estimation?

Before my PhD started, Mark Strong and colleagues at the University of Sheffield developed a method to calculate the EVPPI based on flexible regression. This method is accurate but when you want to calculate the value of a group of model parameters, the computational time increases significantly. A Gaussian process is a method for very flexible regression but could be slow when trying to calculate the EVPPI for a group of parameters. The method we developed adapted the Gaussian process to speed up computation when calculating the EVPPI for a group of parameters. The size of the group of parameters does not really make a difference to the computation for this method, so we allowed for fast EVPPI computation in nearly all practical examples!

What is moment matching, and how can it be used to estimate EVSI?

Moments define the shape of a distribution – the first moment is the mean, the second the variance, the third is the skewness and so on. To estimate the EVSI, we need to estimate a distribution with some specific properties. We can show that this distribution is similar to the distribution of the net benefit from a probabilistic sensitivity analysis. Moment matching is a fancy way of saying that we estimate the EVSI by changing the distribution of the net benefit so it has the same variance as the distribution needed to estimate the EVSI. This significantly decreases the computation time for the EVSI because traditionally we would estimate the distribution for the EVSI using a large number of simulations (I’ve used 10 billion simulations for one estimate).

The really cool thing about this method is that we extended it to use the EVSI to find the trial design and sample size that gives the maximum value for money from research investment resources. The computation time for this analysis was around 5 minutes whereas the traditional method took over 300 days!

Do jobbing health economists need to be experts in value of information analysis to use your BCEA and EVSI software?

The BCEA software uses the costs and effects calculated from a probabilistic health economic model alongside the probabilistic analysis for the model parameters to give standard graphics and summaries. It is based in R and can be used to calculate the EVPPI without being an expert in VoI methods and analysis. All you need is to decide which model parameters you are interested in valuing. We’ve put together a Web interface, BCEAweb, which allows you to use BCEA without using R.

The EVSI software requires a model that incorporates how the data from the future study will be analysed. This can be complicated to design although I’m currently putting together a library of standard examples. Once you’ve designed the study, the software calculates the EVSI without any input from the user, so you don’t need to be an expert in the calculation methods. The software also provides graphics to display the EVSI results and includes text to help interpret the graphical results. An example of the graphical output can be seen here.

Thesis Thursday: Cheryl Jones

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Cheryl Jones who has a PhD from the University of Manchester. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
The economics of presenteeism in the context of rheumatoid arthritis, ankylosing spondylitis and psoriatic arthritis
Supervisors
Katherine Payne, Suzanne Verstappen, Brenda Gannon
Repository link
https://www.research.manchester.ac.uk/portal/en/theses/the-economics-of-presenteeism-in-the-context-of-rheumatoid-arthritis-ankylosing-spondylitis-and-psoriatic-arthritis%288215e79a-925e-4664-9a3c-3fd42d643528%29.html

What attracted you to studying health-related presenteeism?

I was attracted to study presenteeism because it gave me a chance to address both normative and positive issues. Presenteeism, a concept related to productivity, is a controversial topic in the economic evaluation of healthcare technologies and is currently excluded from health economic evaluations, following the recommendation made by the NICE reference case. The reasons why productivity is excluded from economic evaluations are important and valid, however, there are some circumstances where excluding productivity is difficult to defend. Presenteeism offered an opportunity for me to explore and question the social value judgements that underpin economic evaluation methods with respect to productivity. In terms of positive issues related to presenteeism, research into the development of methods that can be used to measure and value presenteeism was (and still is) limited. This provided an opportunity to think creatively about the types of methods we could use, both quantitative and qualitative, to address and further methods for quantifying presenteeism.

Are existing tools adequate for measuring and valuing presenteeism in inflammatory arthritic conditions?

That is the question! Research into methods that can be used to quantify presenteeism is still in its infancy. Presenteeism is difficult to measure accurately because there are a lack of objective measures that can be used, for example, the number of cars assembled per day. As a consequence, many methods rely on self-report surveys, which tend to suffer from bias, such as reporting or recall bias. Methods that have been used to value presenteeism have largely focused on valuing presenteeism as a cost using the human capital approach (HCA: volume of presenteeism multiplied by a monetary factor). The monetary factor typically used to convert the volume of presenteeism into a cost value is wages. Valuing productivity using wages risks taking account of discriminatory factors that are associated with wages, such as age. There are also economic arguments that question whether the value of the wage truly reflects the value of productivity. My PhD focused on developing a method that values presenteeism as a non-monetary benefit, thereby avoiding the need to value it as a cost using wages. Overall, methods to measure and value presenteeism still have some way to go before a ‘gold standard’ can be established, however, there are many experts from many disciplines who are working to improve these methods.

Why was it important to conduct qualitative interviews as part of your research?

The quantitative component of my PhD was to develop an algorithm, using mapping methods, that links presenteeism with health status and capability measures. A study by Connolly et al. recommend conducting qualitative interviews to provide some evidence of face/content validity to establish whether a quantitative link between two measures (or concepts) is feasible and potentially valid. The qualitative study I conducted was designed to understand the extent to which the EQ-5D-5L, SF6D and ICECAP-C were able to capture those aspects of rheumatic conditions that negatively impact presenteeism. The results suggested that all three measures were able to capture those important aspects of rheumatic conditions that affect presenteeism; however, the results indicated that the SF6D would most likely be the most appropriate measure. The results from the quantitative mapping study identified the SF6D as the most suitable outcome measure able to predict presenteeism in working populations with rheumatic conditions. The advantage of the qualitative results was that it provided some evidence that explained why the SF6D was the more suitable measure rather than relying on speculation.

Is it feasible to predict presenteeism using outcome measures within economic evaluation?

I developed an algorithm that links presenteeism, measured using the Work Activity Productivity Impairment (WPAI) questionnaire, with health and capability. Health status was measured using the EQ-5D-5L and SF6D, and capability was measured using the ICECAP-A. The SF6D was identified as the most suitable measure to predict presenteeism in a population of employees with rheumatoid arthritis or ankylosing spondylitis. The results indicate that it is possible to predict presenteeism using generic outcome measures; however, the results have yet to be externally validated. The qualitative interviews provided evidence as to why the SF6D was the better predictor for presenteeism and the result gave rise to questions about the suitability of outcome measures given a specific population. The results indicate that it is potentially feasible to predict presenteeism using outcome measures.

What would be your key recommendation to a researcher hoping to capture the impact of an intervention on presenteeism?

Due to the lack of a ‘gold standard’ method for capturing the impact of presenteeism, I would recommend that the researcher reports and justifies their selection of the following:

  1. Provide a rationale that explains why presenteeism is an important factor that needs to be considered in the analysis.
  2. Explain how and why presenteeism will be captured and included in the analysis; as a cost, monetary benefit, or non-monetary benefit.
  3. Justify the methods used to measure and value presenteeism. It is important that the research clearly reports why specific tools, such as presenteeism surveys, have been selected for use.

Because there is no ‘gold standard’ method for measuring and valuing presenteeism and guidelines do not exist to inform the reporting of methods used to quantify presenteeism, it is important that the researcher reports and justifies their selection of methods used in their analysis.

Thesis Thursday: Angela Devine

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Angela Devine who has a PhD from The Open University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
The economics of vivax malaria treatment
Supervisors
Yoel Lubell, Ric Price, Ricardo Aguas, Shunmay Yeung
Repository link
https://thesiscommons.org/zsc6x/

What is vivax malaria and what are some of the key challenges that it presents for health economists?

One infectious bite from a mosquito carrying vivax malaria can lead to multiple episodes of malaria due to dormant liver parasites called hypnozoites. We can’t tell the difference between these relapse infections and new infections, which means that it’s challenging to model. Unlike falciparum malaria, which frequently results in severe outcomes and deaths, vivax malaria doesn’t often result in direct mortality. Instead, it likely causes indirect mortality through the malnutrition and anaemia that are caused by repeated malaria episodes. Unfortunately, the evidence of this is limited.

To prevent future relapses, patients need to be given a drug to treat the liver parasites (radical cure) in addition to treating the blood stage treatment. The only drug that is currently licensed for radical cure, primaquine, can cause potentially life-threatening haemolysis in individuals who have a genetic disorder called glucose-6-phosphate-dehydrogenase (G6PD) deficiency. While some countries are so concerned about haemolytic events that primaquine isn’t used at all, other settings prescribe primaquine to everyone. The evidence on the risk of primaquine-induced haemolysis and death is sparse, and expert opinion on this matter is fiercely divided.

How did you go about collecting the data needed for your study?

Not much has been done previously on vivax malaria costs, which meant that a lot of my work involved generating cost data. I started by analysing some fairly old data that my supervisors had from a study on treatment-seeking behaviour in Papua, Indonesia. The cost of illness study indicated that household costs were similar for both vivax and falciparum malaria in 2006. I also collected provider and patient-level cost data alongside a multi-country clinical trial on vivax malaria treatment. I wasn’t able to travel to the some of the study sites (e.g. Afghanistan) to collect the provider costs, so I had to create worksheets for local trial staff to fill out. It was an iterative process, particularly with the first site in Indonesia, but it got faster and easier to do each time. I’m very thankful that the local teams were enthusiastic about this work and patient with my many questions and requests.

What is the economic burden of vivax malaria and who bears the cost?

A lot of vivax malaria episodes occur in remote areas where access to care is limited. The highest incidence of the disease is in children, particularly those under the age of five. This often means that someone will need to take time off from usual activities, such as farming, attending school, or household chores, to care for the sick. I estimated the global economic burden to be US$330 million. These estimates don’t include mortality, malnutrition or anaemia. Since we know that repeated episodes can have a profound impact on a household’s income, I included productivity losses for those who were ill and their carers. We also know that malaria causes educational losses, so I included these productivity costs for children as well as adults to try to capture some of those losses. In total, productivity losses accounted for US$263 million, nearly 80% of the total costs. Since many who are affected by this disease aren’t paid for their work, I used one GDP per capita per day for every day lost to illness or caretaking. Other methods of valuing these losses would have a substantial impact on the total costs. While there’s a considerable amount of uncertainty around some of the numbers I used and assumptions that I made, my hope is that by identifying the issues, we will be able to generate the data needed for better estimates in the future.

What methods did you use to evaluate the cost-effectiveness of new treatment strategies?

Asia-Pacific malaria control programs stated that the cost of G6PD screening was an obstacle to its widespread use. My research addressed those concerns through a decision tree model in R that weighed up the costs, risks and benefits of screening using newly developed G6PD rapid diagnostic tests (RDTs) before prescribing primaquine. I wanted to make this work as relevant to policymakers as possible, so I did two separate comparisons. First I compared this strategy to not using primaquine, then I compared it to prescribing primaquine to everyone without screening. While this strayed from typical economic evaluation methods, it seemed unlikely that a setting where primaquine isn’t prescribed due to fear of haemolysis would switch to prescribing primaquine to everyone without screening, or that a setting where primaquine is prescribed to everyone would stop using it altogether.

As G6PD deficiency is X-linked, the risk of haemolysis varies by gender, so results need to be stratified by gender. The prevalence and severity of G6PD deficiency and the latency period and number of relapses for vivax malaria varies geographically. While I wanted to have more than one setting to explore how these might impact the results, four comparisons was already a lot of information to present. Instead, I used R-shiny with my model to create an interactive website where people can see how changes in the baseline model parameters impact the results. My goal was to provide a tool that policymakers could use to help make decisions about treatment strategies in their settings. This also provides an opportunity to explore the impact of parameter values that may be seen as contentious.

What are some of the issues you encountered in working with policymakers to ensure that cost-effective treatments become more widely used?

One issue is that patients, especially those who can afford to do so, seek treatment in the private sector, which is harder to control. Encouragingly, the follow-up survey in Papua, Indonesia indicated that changing treatment policy in the public sector also had an impact on how private sector providers diagnosed and treated malaria. As someone keen to influence policy, I benefited a lot from meetings with malaria control program officials from the Asia Pacific. These provided insights on the challenges that countries are facing. For example, the work I did on G6PD screening was aimed at addressing the cost issue that kept coming up in these meetings. Unfortunately, I’m not aware of settings where they have begun to routinely use G6PD RDTs. There are additional barriers, like getting the tests licensed so that malaria control programs can purchase them with a subsidy from The Global Fund. Another issue that I hadn’t fully appreciated before beginning my PhD is that funding for diseases like malaria is often siloed for specific purposes by the various donors. This can make it more challenging to ensure that countries are getting the best possible value for the money that is spent. There’s also been a lot of debate recently about what willingness to pay threshold should be used in poorly resourced settings. This is a debate that we need to have, but it also makes it more challenging to decide which treatments should be considered to be cost-effective.