Thesis Thursday: David Mott

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr David Mott who has a PhD from Newcastle University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

How do preferences for public health interventions differ? A case study using a weight loss maintenance intervention
Luke Vale, Laura Ternent
Repository link

Why is it important to understand variation in people’s preferences?

It’s not all that surprising that people’s preferences for health care interventions vary, but we don’t have a great understanding of what might drive these differences. Increasingly, preference information is being used to support regulatory decisions and, to a lesser but increasing extent, health technology assessments. It could be the case that certain subgroups of individuals would not accept the risks associated with a particular health care intervention, whereas others would. Therefore, identifying differences in preferences is important. However, it’s also useful to try to understand why this heterogeneity might occur in the first place.

The debate on whose preferences to elicit for health state valuation has traditionally focused on those with experience (e.g. patients) and those without (e.g. the general population). Though this dichotomy is problematic; it has been shown that health state utilities systematically differ between these two groups, presumably due to the difference in relative experience. My project aimed to explore whether experience also affects people’s preferences for health care interventions.

How did you identify different groups of people, whose preferences might differ?

The initial plan for the project was to elicit preferences for a health care intervention from general population and patient samples. However, after reviewing the literature, it seemed highly unlikely that anyone would advocate for preferences for treatments to be elicited from general population samples. It has long been suggested that discrete choice experiments (DCEs) could be used to incorporate patient preferences into decision-making, and it turned out that patients were the focus of the majority of the DCE studies that I reviewed. Given this, I took a more granular approach in my empirical work.

We recruited a very experienced group of ‘service users’ from a randomised controlled trial (RCT). In this case, it was a novel weight loss maintenance intervention aimed at helping obese adults that had lost at least 5% of their overall weight to maintain their weight loss. We also recruited an additional three groups from an online panel. The first group were ‘potential service users’ – those that met the trial criteria but could not have experienced the intervention. The second group were ‘potential beneficiaries’ – those that were obese or overweight and did not meet the trial criteria. The final group were ‘non-users’ – those with a normal BMI.

What can your study tell us about preferences in the context of a weight loss maintenance intervention?

The empirical part of my study involved a DCE and an open-ended contingent valuation (CV) task. The DCE was focused on the delivery of the trial intervention, which was a technology-assisted behavioural intervention. It had a number of different components but, briefly, it involved participants weighing themselves regularly on a set of ‘smart scales’, which enabled the trial team to access and monitor the data. Participants received text messages from the trial team with feedback, reminders to weigh themselves (if necessary), and links to online tools and content to support the maintenance of their weight loss.

The DCE results suggested that preferences for the various components of the intervention varied significantly between individuals and between the different groups – and not all were important. In contrast, the efficacy and cost attributes were important across the board. The CV results suggested that a very significant proportion of individuals would be willing to pay for an effective intervention (i.e. that avoided weight regain), with very few respondents expressing a willingness to pay for an intervention that led to more than 10-20% weight regain.

Do alternative methods for preference elicitation provide a consistent picture of variation in preferences?

Existing evidence suggests that willingness to pay (WTP) estimates from CV tasks might differ from those derived from DCE data, but there aren’t a lot of empirical studies on this in health. Comparisons were planned in my study, but the approach taken in the end was suboptimal and ultimately inconclusive. The original plan was to obtain WTP estimates for an entire WLM intervention using the DCE and to compare this with the estimates from the CV task. Due to data limitations, it wasn’t possible to make this comparison. However, the CV task was a bit unusual because we asked for respondents’ WTP at various different efficacy levels. So instead the comparison made was between average WTP values for a percentage point of weight re-gain. The differences were statistically insignificant.

Are some people’s preferences ‘better defined’ than others’?

We hypothesised that those with experience of the trial intervention would have ‘better defined’ preferences. To explore this, we compared the data quality across the different user groups. From a quick glance at the DCE results, it is pretty clear that the data were much better for the most experienced group; the coefficients were larger, and a much higher proportion was statistically significant. However, more interestingly, we found that the most experienced group were 23% more likely to have passed all of the rationality tests that were embedded in the DCE. Therefore, if you accept that better quality data is an indicator of ‘better defined’ preferences, then the data do seem reasonably supportive of the hypothesis. That being said, there were no significant differences between the other three groups, begging the question: was it the difference in experience, or some other difference between RCT participants and online panel respondents?

What does your research imply for the use of preferences in resource allocation decisions?

While there are still many unanswered questions, and there is always a need for further research, the results from my PhD project suggest that preferences for health care interventions can differ significantly between respondents with differing levels of experience. Had my project been applied to a more clinical intervention that is harder for an average person to imagine experiencing, I would expect the differences to have been much larger. I’d love to see more research in this area in future, especially in the context of benefit-risk trade-offs.

The key message is that the level of experience of the participants matters. It is quite reasonable to believe that a preference study focusing on a particular subgroup of patients will not be generalisable to the broader patient population. As preference data, typically elicited from patients, is increasingly being used in decision-making – which is great – it is becoming increasingly important for researchers to make sure that their respondent samples are appropriate to support the decisions that are being made.

Thesis Thursday: Logan Trenaman

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Logan Trenaman who has a PhD from the University of British Columbia. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Economic evaluation of interventions to support shared decision-making: an extension of the valuation framework
Nick Bansback, Stirling Bryan
Repository link

What is shared decision-making?

Shared decision-making is a process whereby patients and health care providers work together to make decisions. For most health care decisions, where there is no ‘best’ option, the most appropriate course of action depends on the clinical evidence and the patient’s informed preferences. In effect, shared decision-making is about reducing information asymmetry, by allowing providers to inform patients about the potential benefits and harms of alternative tests or treatments, and patients to express their preferences to their provider. The goal is to reach agreement on the most appropriate decision for that patient.

My thesis focused on individuals with advanced osteoarthritis who were considering whether to undergo total hip or knee replacement, or use non-surgical treatments such as pain medication, exercise, or mobility aids. Joint replacement alleviates pain and improves mobility for most patients, however, as many as 20-30% of recipients have reported insignificant improvement in symptoms and/or dissatisfaction with results. Shared decision-making can help ensure that those considering joint replacement are aware of alternative treatments and have realistic expectations about the potential benefits and harms of each option.

There are different types of interventions available to help support shared decision-making, some of which target the patient (e.g. patient decision aids) and some of which target providers (e.g. skills training). My thesis focused on a randomized controlled trial that evaluated a pre-consultation patient decision aid, which generated a summary report for the surgeon that outlined the patient’s knowledge, values, and preferences.

How can the use of decision aids influence health care costs?

The use of patient decision aids can impact health care costs in several ways. Some patient decision aids, such as those evaluated in my thesis, are designed for use by patients in preparation for a consultation where a treatment decision is made. Others are designed to be used during the consultation with the provider. There is some evidence that decision aids may increase up-front costs, by increasing the length of consultations, requiring investments to integrate decision aids into routine care, or train clinicians. These interventions may impact downstream costs by influencing treatment decision-making. For example, the Cochrane review of patient decision aids found that, across 18 studies in major elective surgery, those exposed to decision aids were less likely to choose surgery compared to those in usual care (RR: 0.86, 95% CI: 0.75 to 1.00).

This was observed in the trial-based economic evaluation which constituted the first chapter of my thesis. This analysis found that decision aids were highly cost-effective, largely due to a smaller proportion of patients undergoing joint replacement. Of course, this conclusion could change over time. One of the challenges of previous cost-effectiveness analysis (CEA) of patient decision aids has been a lack of long-term follow-up. Patients who choose not to have surgery over the short-term may go on to have surgery later. To look at the longer-term impact of decision aids, the third chapter of my thesis linked trial participants to administrative data with an average of 7-years follow-up. I found that, from a resource use perspective, the conclusion was the same as observed during the trial: fewer patients exposed to decision aids had undergone surgery, resulting in lower costs.

What is it about shared decision-making that patients value?

On the whole, the evidence suggests that patients value being informed, listened to, and offered the opportunity to participate in decision-making (should they wish!). To better understand how much shared decision-making is valued, I performed a systematic review of discrete choice experiments (DCEs) that had valued elements of shared decision-making. This review found that survey respondents (primarily patients) were willing to wait longer, pay, and in some cases willing to accept poorer health outcomes for greater shared decision-making.

It is important to consider preference heterogeneity in this context. The last chapter of my PhD performed a DCE to value shared decision-making in the context of advanced knee osteoarthritis. The DCE included three attributes: waiting time, health outcomes, and shared decision-making. The latent class analysis found four distinct subgroups of patients. Two groups were balanced, and traded between all attributes, while one group had a strong preference for shared decision-making, and another had a strong preference for better health outcomes. One important finding from this analysis was that having a strong preference for shared decision-making was not associated with demographic or clinical characteristics. This highlights the importance of each clinical encounter in determining the appropriate level of shared decision-making for each patient.

Is it meaningful to estimate the cost-per-QALY of shared decision-making interventions?

One of the challenges of my thesis was grappling with the potential conflict between the objectives of CEA using QALYs (maximizing health) and shared decision-making interventions (improved decision-making). Importantly, encouraging shared decision-making may result in patients choosing alternatives that do not maximize QALYs. For example, informed patients may choose to delay or forego elective surgery due to potential risks, despite it providing more QALYs (on average).

In cases where a CEA finds that shared decision-making interventions result in poorer health outcomes at lower cost, I think this is perfectly acceptable (provided patients are making informed choices). However, it becomes more complicated when shared decision-making interventions increase costs, result in poorer health outcomes, but provide other, non-health benefits such as informing patients or involving them in treatment decisions. In such cases, decision-makers need to consider whether it is justified to allocate scarce health care resources to encourage shared decision-making when it requires sacrificing health outcomes elsewhere. The latter part of my thesis tried to inform this trade-off, by valuing the non-health benefits of shared decision-making which would not otherwise be captured in a CEA that uses QALYs.

How should the valuation framework be extended, and is this likely to indicate different decisions?

I extended the valuation framework by attempting to value non-health benefits of shared decision-making. I followed guidelines from the Canadian Agency for Drugs and Technologies in Health, which state that “the value of non-health effects should be based on being traded off against health” and that societal preferences be used for this valuation. Requiring non-health benefits to be valued relative to health reflects the opportunity cost of allocating resources toward these outcomes. While these guidelines do not specifically state how to do so, I chose to value shared decision-making relative to life-years using a chained (or two-stage) valuation approach so that they could be incorporated within the QALY.

Ultimately, I found that the value of the process of shared decision-making was small, however, this may have an impact on cost-effectiveness. The reasons for this are twofold. First, there are few cases where shared decision-making interventions improve health outcomes. A 2018 sub-analysis of the Cochrane review of patient decision aids found little evidence that they impact health-related quality of life. Secondly, the up-front cost of implementing shared decision-making interventions may be small. Thus, in cases where shared decision-making interventions require a small investment but provide no health benefit, the non-health value of shared decision-making may impact cost-effectiveness. One recent example from Dr Victoria Brennan found that incorporating process utility associated with improved consultation quality, resulting from a new online assessment tool, increased the probability that the intervention was cost-effective from 35% to 60%.

Thesis Thursday: Firdaus Hafidz

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Firdaus Hafidz who has a PhD from the University of Leeds. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Assessing the efficiency of health facilities in Indonesia
Tim Ensor, Sand Tubeuf
Repository link

What are some of the key features of health and health care in Indonesia?

Indonesia is a diverse country, with more than 17 thousand islands and 500 districts. Thus, there is a wide discrepancy of health outcomes across Indonesia, which also reflects the country’s double burden of both communicable and emerging non-communicable diseases. Communicable diseases such as tuberculosis, diarrhoea and lower respiratory tract infections remain as significant issues in Indonesia, especially in remote areas. At the same time, non-communicable diseases are becoming a major public health problem, especially in urban areas.

Total healthcare expenditure per capita grew rapidly, but in certain outcomes, such as maternal mortality rate, Indonesia performs less well than other low- and middle-income countries. Health facilities represent the largest share of healthcare expenditures, but utilisation is still considered low in both hospitals and primary healthcare facilities. Given the scarcity of public healthcare resources, out-of-pocket expenditure remains considerably higher than the global average.

To reduce financial barriers, the Government of Indonesia introduced health insurance in 1968. Between 2011 and 2014, there were three major insurance schemes: 1) Jamkesmas – poor scheme; 2) Jamsostek – formal sector workers scheme; and 3) Askes – civil servant scheme. In 2014, the three schemes were combined into a single-entity National Health Insurance scheme.

What methods can be used to measure the efficiency of health care in low and middle-income countries?

We reviewed measurements of efficiency in empirical analyses conducted in low- and middle-income countries. Methods, including techniques, variables, and efficiency indicators were summarised. There was no consensus on the most appropriate technique to measure efficiency, though most existing studies have relied on ratio analysis and data envelopment analysis because it is simple, easy to compute, low-cost and can be performed on small samples. The physical inputs included the type of capital (e.g. the number of beds and size of health facilities) and the type of labour (e.g. the number of medical and non-medical staff). Most of the published literature used health services as outputs (e.g. the number of outpatient visits, admission and inpatient days). However, because of poor data availability, fewer studies used case-mix and quality indicators to adjust outputs. So most of the studies in the literature review assumed that there was no difference in the severity and effectiveness of healthcare services. Despite the complexity of the techniques, researchers are responsible to provide interpretable results to the policymakers to guide their decisions for a better health policy on efficiency. Adopting appropriate methods that have been used globally would be beneficial to benchmark empirical studies.

Were you able to identify important sources of inefficiency in Indonesia?

We used several measurement techniques including frontier analysis and ratio analysis. We explored contextual variables to assess factors determining efficiency. The range of potential models produced help policymakers in the decision-making process according to their priority and allow some control over the contextual variables. The results revealed that the efficiency of primary care facilities can be explained by population health insurance coverage, especially through the insurance scheme for the poor. Geographical factors, such as the main islands (Java or Bali), better access to health facility, and location in an urban area also have a strong impact on efficiency. At the hospitals, the results highlighted higher efficiency levels in larger hospitals; they were more likely to present in deprived areas with low levels of education; and they were located on Java or Bali. Greater health insurance coverage also had a positive and significant influence on efficiency.

How could policymakers improve the efficiency of health care in Indonesia or other similar settings?

I think there are several ideas. First, we need to have a careful tariff adjustment as we found an association between low unit costs and high efficiency scores. Case base group tariffs need to account for efficiency scores to prevent unnecessary incentives for the providers, exacerbating inefficiency in the health system.

Secondly, we need flexibility in employment contracts, particularly for the less productive civil servant worker so the less productive worker could be reallocated. We also need a better remuneration policy to attract skilled labour and improve health facilities efficiency.

From the demand side, reducing physical barriers by improving infrastructure could increase efficiency in the rural health care facilities through higher utilisation of care. Facilities with very low utilisation rates still incur a fixed cost and thus create inefficiency. Through the same argument we also need to reduce financial barriers using incentives programmes and health insurance, thus patients who are economically disadvantaged can access healthcare services.

How would you like to see other researchers build on your work?

Data quality is crucial in secondary data analysis research, and it was quite a challenge in an Indonesian setting. Meticulous data management is needed to mitigate data errors such as inconsistency, outliers and missing values.

As this study used a 2011 cross-sectional dataset, replicating this study using a more recent and even longitudinal data would highlight changes in efficiency due to policy changes or interventions. Particularly interesting is the effect of the 2014 implementation of Indonesian national health insurance.

My study has some limitations and thus warrants further investigation. The stochastic frontier analysis failed to identify any inefficiency at hospitals when outpatient visits were included. The statistical errors of the frontier function cannot be distinguished from the inefficiency effect of the model. It might be related to the volume and heterogeneity of outpatient services which swamps the total volume of services and masks any inefficiency.