Thesis Thursday: Koh Jun Ong

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Koh Jun Ong who has a PhD from the University of Groningen. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Economic aspects of public health programmes for infectious disease control: studies on human immunodeficiency virus & human papillomavirus
Supervisors
Maarten Postma, Mark Jit
Repository link
http://hdl.handle.net/11370/0edbcfae-2a0c-4103-9722-fb8086d75cff

Which public health programmes did you consider in your research?

Three public health programmes were considered in the thesis: 1) HIV Pre-Exposure Prophylaxis (PrEP), 2) Human Papillomavirus (HPV) vaccination, and 3) HIV screening to reduce undiagnosed infections in the population.

The first two of the three involved primary infectious disease prevention among men who have sex with men (MSM), and both of these programmes were to be delivered via sexual health clinics in England (commonly known as genitourinary medicine, GUM, clinics).

The third public health infectious disease control programme involved secondary prevention of onward HIV transmission in the general population by encouraging routine HIV screening to reduce undiagnosed HIV, with a view of earlier diagnosis leading to antiretroviral treatment initiation, which will stop HIV transmission with viral suppression.

Was it necessary to develop complex mathematical models?

It depends on the policy research question. A dynamic model was used for the HPV vaccination research question, which captures the ecological externality that vaccination provides by reducing transmission to non-vaccinees. A dynamic model was used because this programme would likely reach a high proportion of MSM who attend GUM clinics in England, and therefore the subsequent knock-on impact of disease transmission in the population was likely to be substantial.

The policy research question was different for PrEP and a static model was more suitable since the objective was to advise NHS England on whether and how such a programme, with relatively small numbers of patients over an initial time-limited period, may represent value for money in England. We first considered a public health control programme, with promising new efficacy data from the 500-person PrEP pilot study (the UK-based PROUD trial) and additional information from per protocol participants in the earlier iPrEx study. The initial consideration was to maintain the preventative effect of a drug that needs to be taken on a daily basis (compared with near one-off HPV vaccination – three doses in total delivered within a year’s time). Regular monitoring of STI and patient’s renal function meant there were clinical service capacity issue to consider, which was likely to limit access initially. Thus, a static model that did not take into account transmission was used.

However, dynamic modelling would be useful to inform policy decisions as PrEP usage expands. Firstly, because it would then be important to capture the indirect effect on infection transmission. Secondly, because when the force of infection begins to fall as incidence declines, dynamic modelling will inform future delivery of a programme that maintains its value. These represent important areas for future research.

Finally, the model designed for the research question on HIV screening was quite straightforward as its aim is primarily to advise local commissioners on financial implications of offering routine screening in their local area, which is dependent on local clinical resources and local disease prevalence.

Did you draw any important conclusions from your literature reviews?

Two literature reviews were conducted: 1) a review on economic parameters i.e. cost and utility estimates for HPV-related outcomes, and 2) a review on published MSM HPV vaccination economic evaluations.

In relation to the first review, most economic models of HPV-related interventions selected economic parameters in a pretty ad hoc way, without reviewing the entirety of the literature. We found substantial variations in cost and utility estimates for all diseases considered in our systematic review, wherever there were more than one publication. These variations in value estimates could result from the differences in cancer site, disease stages, study population, treatment pathway/settings, treatment country and utility elicitation methods used. It would be important for future models to be transparent about parameter sources and assumptions, and to recognise that as patient disease management changes over time, there will be corresponding effects on both cost and utility, necessitating future updates to the estimates. These must be considered when applied to future economic evaluations, to ensure that assumptions are up-to-date and closely reflect the case mix of patients being evaluated.

In relation to the second review, despite limited models, different modelling approaches and assumptions, a general theme from these studies reveal modelling outcomes to be most sensitive to assumptions around vaccine efficacy and price. Future studies could consider synchronising parameter assumptions to test outputs generated by different models.

What can your research tell us about the ‘cost-effective but unaffordable’ paradox?

A key finding and concluding remark of this thesis was that “findings around cost-effectiveness should not be considered independently of budget impact and affordability considerations, as the two are interlinked”. Ultimately, cost-effectiveness is linked to the budget and, in an ideal world, a cost-effectiveness threshold should correspond to the opportunity cost of replacing least cost-effective care at the margin of the whole healthcare budget spend. This willingness to pay threshold should be linked to the amount of budgetary resources an intervention displaces. After all, the concept of opportunity cost in a fixed budget setting means that decisions to invest in something translates to funding being displaced elsewhere.

Since most health economies do not have unlimited resources, even if investment in a new intervention gives high returns and therefore is worthwhile from a value for money perspective, without the necessary resources it cannot always be afforded despite its high return on investment. Having a limited budget means that funding an expensive new intervention may mean moving funding away from existing services, which may be more cost-effective than the new intervention. Hence, the services from which funds are moved from will lose out, and this may leave society worse-off.

A simple analogy may be that buying a property that guarantees return over a defined period is worthwhile, but if I cannot afford it in the first place, is this still an option?

This was clearly demonstrated in the PrEP example, where despite potential to be cost-effective, the high cost of the intervention at list price carried with it a very high budget impact. The size of the population needed to be given PrEP to achieve substantial public health benefits is large, which meant that a public health programme could pose an affordability challenge to the national health care system.

Based on your findings, how might HIV and HPV prevention strategies be made more cost-effective?

Two strategies could influence cost effectiveness: optimizing the population covered and using an appropriate comparator price.

The most obvious way to improve cost-effectiveness is to optimise the population covered. For example, we know that HIV risk, as measured by HIV incidence, is higher among GUM-attending MSM. Therefore, delivering a PrEP programme to this population (at least in the initial phase until the intervention becomes more affordable) will likely result in a higher number of new HIV infections prevented. Similarly, HIV screening offered to areas with high local prevalence would likely give a higher number of new diagnoses.

The other important factor to consider around cost-effectiveness is the comparator price on which the technology appraisal is based. In the chapter on estimating HIV care cost in England, we demonstrated that with imminent availability of generic antiretrovirals, the lifetime care cost for a person living with HIV will reduce substantially. This reduced cost, representing cost of care with existing intervention, should be used as comparator for newer HIV interventions, as they would represent what society will be paying in the absence of the new interventions, allowing corresponding reduced price expectations for new interventions to ensure cost-effectiveness is maintained.

How did you find the experience of completing your thesis by publication?

It was brilliant! I must acknowledge all the contributions from my supervisors and co-authors in making this possible and for the very positive experience of this process. A major advantage of doing a PhD by publication is that the work conducted was regularly peer-reviewed, hence providing an extra check of the robustness of the analyses. And also the fact that these works are out for public consumption almost immediately, making the science available for other researchers to consider and to move the science to the next stage.

Thesis Thursday: Caroline Chuard

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Caroline Chuard who has a PhD from the University of Zurich. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Three essays on the health effects of family policies
Supervisors
Hannes Schwandt, Josef Zweimüller
Repository link
https://www.zora.uzh.ch/id/eprint/172853/

Is there a strong health economics evidence base on family policies?

The literature on parental leave and family health is relatively young. This literature emphasises that the returns depend on several key features. First, the timing of measurement matters. Therefore, the effects differ according to whether they are measured in the short- versus long-run. Second, the initial level of parental leave and the extent to which parental leave is increased are both key influencing factors. As such, an introduction is more beneficial than an increase at an already generous level of parental leave. Third, the results depend on the targeted group.

But keep in mind that the effects of family policies on health outcomes are just one part of a large literature that studies the effect on other outcomes such as maternal labour market outcomes, fertility, and child cognitive and non-cognitive development (e.g. Ruhm (2000), Lalive and Zweimüller (2009), Baker and Milligan (2008), Dustmann and Schönberg (2012), Lalive et al. (2014), Carneiro et al. (2015), Dahl et al. (2016), Danzer and Lavy (2018), Butikofer et al. (2018) and many more which have recently been reviewed by Olivetti and Petrongolo (2017) and Rossin-Slater (2018)).

What policy changes were you able to evaluate in your research?

I exploit two types of family policy changes in two countries. On the one hand, I use three changes in parental leave duration in Austria and, on the other hand, I use cantonal variation in family allowances across Switzerland.

More specifically, Austria increased parental leave by 1 year to 2 years in July 1990. This was partially reversed again in July 1996, by exclusively reserving 6 months to fathers so that maternal leave was essentially reduced to 1.5 years. Finally, in July 2000, there was another large extension in paid parental leave by 1 year to 2.5 years. Enforcement of all these changes was very strict, changing from one day to another depending on giving birth in June or July. This sharp discontinuity allows me to employ a regression discontinuity design.

In the case of Switzerland, I analyse the impact of birth allowances (so-called baby bonuses) on fertility, newborn health and birth scheduling. I exploit a unique quasi-experimental setting of Switzerland’s family allowances system. In this system, cantons are free to choose whether they want to implement birth allowances and how much they want to pay. During the last 50 years, 11 cantons have introduced a baby bonus, all increase the amount paid thereafter, and two cantons even abolished the baby bonus after all. This gives rise to a lot of cantonal variation. Thus, I use a difference-in-differences setting where I can analyse both the introduction and the intensity of the treatment.

What were the key strengths of the data sets that you used?

For all my studies I rely on administrative data. Thus, I can use the universe of observations delivered with high quality, as both Austria and Switzerland have very reliable administrative data.

In the Austrian case, I can even combine several different data sets. Namely, I use the Austrian Social Security Database (ASSD), which covers the complete working history of every worker in Austria. The ASSD covers every birth of employed mothers and their actual duration of parental leave. I can link the ASSD to the Austrian Birth Register (ABR) recording newborn health outcomes and additional individual-level characteristics of the mother. Finally, for a part of Austria, I additionally merge the data to health outcomes recorded in the health insurance data. This data set records every outpatient doctor visit, prescribed medication, and hospital stays including diagnosis code.

All of this, together, gives a huge variety of different variables on an individual basis allowing me to study a broad set of outcomes (such as health outcomes next to the directly targeted labour market outcomes). Furthermore, the detailed level of information allows me to study the impact of labour market behaviour on two margins—the extensive margin of mothers who choose to work or not and the intensive margin of how much mothers choose to work. The richness of the data also makes it possible to analyse heterogeneous effects across mothers and by work environment.

Did the policies achieve what they were designed to achieve?

This is a little hard to tell from looking at my results only. For example, in Austria the initial increase of parental leave duration by 1 year was introduced so that fathers could take up to 6 months of the full duration. This policy reform was a result of parliamentary procedural requests which wanted to introduce paternal leave. Due to the flat benefit structure almost no fathers were taking up parental leave, which essentially resulted in an increase of maternal leave from 1 to 2 years and, ultimately, led to the second policy change by exclusively reserving 6 months out of the total 2 years for fathers.

However, what I want to mention here, note that I explicitly evaluated side effects. All three chapters of my dissertation highlight the importance of studying alternative and indirect outcome measures in addition to the direct measures targeted by policymakers.

For example, in the Swiss study, we only find little fertility effects, the directly targeted outcome measure of birth allowances, but a sizable and significant reduction in the stillbirth rate as well as a positive impact on birth weight. A policymaker, who would now only study fertility, would argue that birth allowances were expensive to implement with little to no result, which, however, does not capture the full story.

Is there heterogeneity in how family policy reforms affect families?

The answer depends on the person affected and the studied outcome. For example, the Austrian parental leave duration reform affects maternal work behaviour during pregnancy regardless of the mother’s socioeconomic background and the industry. This change in prenatal maternal work status doesn’t affect newborn health at all.

However, when I study the same reforms with respect to maternal health, there is substantial heterogeneity. The initial increase in leave length is especially good for low-wage and unmarried mothers. Reducing leave duration harms mothers with unhealthy babies, proxied by a preterm birth or low birth weight baby. Substantially increasing leave duration is, though, especially bad for maternal health of those mothers who already suffered from mental diseases pre-birth. Also, for the paper on the Swiss baby bonus, we find a more beneficial impact in the decline of stillbirths for low socioeconomic status mothers.

Based on your research, how would you design parental leave policies?

With my research, I tried to give a more complete picture on the impact of family policies by taking into account health outcomes which have vastly been neglected so far. Nevertheless, for a policy recommendation it is crucial to take the findings from the previous literature into account.

Firstly, introducing parental leave has generally been shown to be very beneficial for the cognitive development of children (Carneiro et al., 2015). Secondly, these returns are, however, quickly declining (Butikofer et al., 2018). In combination with my findings of no impact of working during pregnancy on child health and a negative impact of too long parental leave policies for maternal health (Chuard, 2018), I would clearly put the focus on mandatory leave in the first months of a newborn’s life. While this might seem obvious for many European countries, this is still not the case in the US. And even Europe might face the risk on the other end of the parental leave duration scale. Many European countries tend to expand leave rather generously both pre- and post-natal, which seems from my research not necessary (always keep in mind, these policies are extremely expensive) and could potentially even be harmful in the long-run.

Thesis Thursday: Andrea Gabrio

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Andrea Gabrio who has a PhD from University College London. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Full Bayesian methods to handle missing data in health economic evaluation
Supervisors
Gianluca Baio, Alexina Mason, Rachael Hunter
Repository link
http://discovery.ucl.ac.uk/10072087

What kind of assumptions about missing data are made in trial-based economic evaluations?

In any analysis, assumptions about the missing values are always made, about those values which are not observed. Since the final results may depend on these assumptions, it is important that they are as plausible as possible within the context considered. For example, in trial-based economic evaluations, missing values often occur when data are collected through self-reported patient questionnaires and in many cases it is plausible that patients with unobserved responses are different from the others (e.g. have worse health states). In general, it is very important that a range of plausible scenarios (defined according to the available information) are considered, and that the robustness of our conclusions across them is assessed in sensitivity analysis. Often, however, analysts prefer to ignore this uncertainty and rely on ‘default’ approaches (e.g. remove the missing data from the analysis) which implicitly make unrealistic assumptions and possibly lead to biased results. For a more in-depth overview of current practice, I refer to my published review.

Given that any assumption about the missing values cannot be checked from the data at hand, an ideal approach to handle missing data should combine a well-defined model for the observed data and explicit assumptions about missingness.

What do you mean by ‘full Bayesian’?

The term ‘full Bayesian’ is a technicality and typically indicates that, in the Bayesian analysis, the prior distributions are freely specified by the analyst, rather than being based on the data (e.g. ’empirical Bayesian’). Being ‘fully’ Bayesian has some key advantages for handling missingness compared to other approaches, especially in small samples. First, a flexible choice of the priors may help to stabilise inference and avoid giving too much weight to implausible parameter values. Second, external information about missingness (e.g. expert opinion) can be easily incorporated into the model through the priors. This is essential when performing sensitivity analysis to missingness, as it allows assessment of the robustness of the results to a range of assumptions, with the uncertainty of any unobserved quantity (parameters or missing data) being fully propagated and quantified in the posterior distribution.

How did you use case studies to support the development of your methods?

In my PhD I had access to economic data from two small trials, which were characterised by considerable amounts of missing outcome values and which I used as motivating examples to implement my methods. In particular, individual-level economic data are characterised by a series of complexities that make it difficult to justify the use of more ‘standardised’ methods and which, if not taken into account, may lead to biased results.

Examples of these include the correlation between effectiveness and costs, the skewness in the empirical distributions of both outcomes, the presence of identical values for many individuals (e.g. excess zeros or ones), and, on top of that, missingness. In many cases, the implementation of methods to handle these issues is not straightforward, especially when multiple types of complexities affect the data.

The flexibility of the Bayesian framework allows the specification of a model whose level of complexity can be increased in a relatively easy way to handle all these problems simultaneously, while also providing a natural way to perform probabilistic sensitivity analysis. I refer to my published work to see an example of how Bayesian models can be implemented to handle trial-based economic data.

How does your framework account for longitudinal data?

Since the data collected within a trial have a longitudinal nature (i.e. collected at different times), it is important that any missingness methods for trial-based economic evaluations take into account this feature. I therefore developed a Bayesian parametric model for a bivariate health economic longitudinal response which, together with accounting for the typical complexities of the data (e.g. skewness), can be fitted to all the effectiveness and cost variables in a trial.

Time dependence between the responses is formally taken into account by means of a series of regressions, where each variable can be modelled conditionally on other variables collected at the same or at previous time points. This also offers an efficient way to handle missingness, as the available evidence at each time is included in the model, which may provide valuable information for imputing the missing data and therefore improve the confidence in the final results. In addition, sensitivity analysis to a range of missingness assumptions can be performed using a ‘pattern mixture’ approach. This allows the identification of certain parameters, known as sensitivity parameters, on which priors can be specified to incorporate external information and quantify its impact on the conclusions. A detailed description of the longitudinal model and the missing data analyses explored is also available online.

Are your proposed methods easy to implement?

Most of the methods that I developed in my project were implemented in JAGS, a software specifically designed for the analysis of Bayesian models using Markov Chain Monte Carlo simulation. Like other Bayesian software (e.g. OpenBUGS and STAN), JAGS is freely available and can be interfaced with different statistical programs, such as R, SAS, Stata, etc. Therefore, I believe that, once people are willing to overcome the initial barrier of getting familiar with a new software language, these programs provide extremely powerful tools to implement Bayesian methods. Although in economic evaluations analysts are typically more familiar with frequentist methods (e.g. multiple imputations), it is clear that as the complexity of the analysis increases, the implementation of these methods would require tailor-made routines for the optimisation of non-standard likelihood functions, and a full Bayesian approach is likely to be a preferable option as it naturally allows the propagation of uncertainty to the wider economic model and to perform sensitivity analysis.