On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Anna Heath who has a PhD from the University College London. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.
Title
Bayesian computations for value of information measures using Gaussian processes, INLA and Moment Matching
Supervisors
Gianluca Baio, Ioanna Manolopoulou
Repository link
http://discovery.ucl.ac.uk/id/eprint/10050229
Why are new methods needed for value of information analysis?
Value of Information (VoI) has been around for a really long time – it was first mentioned in a book published in 1959! More recently, it has been suggested that VoI methods can be used in health economics to direct and design future research strategies. There are several different concepts in VoI analysis and each of these can be used to answer different questions. The VoI measure with the most potential calculates the economic benefit of collecting additional data to inform a health economic model (known as the EVSI). The EVSI can be compared with the cost of collecting data and allow us to make sure that our clinical research is “cost-effective”.
The problem is that, mathematically, VoI measures are almost impossible to calculate, so we have to use simulation. Traditionally, these simulation methods have been very slow (in my PhD, one example took over 300 days to compute 10 VoI measures) so we need simulation methods that speed up the computation significantly before VoI can be used for decisions about research design and funding.
Do current EVPPI and EVSI estimation methods give different results?
For most examples, the current estimation methods give similar results but the computational time to obtain these results differs significantly. Since starting my PhD, different estimation methods for the EVPPI and the EVSI have been published. The difference between these methods are the assumptions and the ease of use. The results seem to be pretty stable for all the different methods, which is good!
The EVPPI determines which model parameters have the biggest impact on the cost-effectiveness of the different treatments. This is used to direct possible avenues of future research, i.e. we should focus on gaining more information about parameters with a large impact on cost-effectiveness. The EVPPI is calculated based only on simulations of the model parameters so the number of methods for EVPPI calculation is quite small. To calculate the EVSI, you need to consider how to collect additional data, through a clinical trial, observational study etc, so there is a wider range of available methods.
How does the Gaussian process you develop improve EVPPI estimation?
Before my PhD started, Mark Strong and colleagues at the University of Sheffield developed a method to calculate the EVPPI based on flexible regression. This method is accurate but when you want to calculate the value of a group of model parameters, the computational time increases significantly. A Gaussian process is a method for very flexible regression but could be slow when trying to calculate the EVPPI for a group of parameters. The method we developed adapted the Gaussian process to speed up computation when calculating the EVPPI for a group of parameters. The size of the group of parameters does not really make a difference to the computation for this method, so we allowed for fast EVPPI computation in nearly all practical examples!
What is moment matching, and how can it be used to estimate EVSI?
Moments define the shape of a distribution – the first moment is the mean, the second the variance, the third is the skewness and so on. To estimate the EVSI, we need to estimate a distribution with some specific properties. We can show that this distribution is similar to the distribution of the net benefit from a probabilistic sensitivity analysis. Moment matching is a fancy way of saying that we estimate the EVSI by changing the distribution of the net benefit so it has the same variance as the distribution needed to estimate the EVSI. This significantly decreases the computation time for the EVSI because traditionally we would estimate the distribution for the EVSI using a large number of simulations (I’ve used 10 billion simulations for one estimate).
The really cool thing about this method is that we extended it to use the EVSI to find the trial design and sample size that gives the maximum value for money from research investment resources. The computation time for this analysis was around 5 minutes whereas the traditional method took over 300 days!
Do jobbing health economists need to be experts in value of information analysis to use your BCEA and EVSI software?
The BCEA software uses the costs and effects calculated from a probabilistic health economic model alongside the probabilistic analysis for the model parameters to give standard graphics and summaries. It is based in R and can be used to calculate the EVPPI without being an expert in VoI methods and analysis. All you need is to decide which model parameters you are interested in valuing. We’ve put together a Web interface, BCEAweb, which allows you to use BCEA without using R.
The EVSI software requires a model that incorporates how the data from the future study will be analysed. This can be complicated to design although I’m currently putting together a library of standard examples. Once you’ve designed the study, the software calculates the EVSI without any input from the user, so you don’t need to be an expert in the calculation methods. The software also provides graphics to display the EVSI results and includes text to help interpret the graphical results. An example of the graphical output can be seen here.
