Thesis Thursday: Lidia Engel

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Lidia Engel who graduated with a PhD from Simon Fraser University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Going beyond health-related quality of life for outcome measurement in economic evaluation
Supervisors
David Whitehurst, Scott Lear, Stirling Bryan
Repository link
https://theses.lib.sfu.ca/thesis/etd10264

Your thesis explores the potential for expanding the ‘evaluative space’ in economic evaluation. Why is this important?

I think there are two answers to this question. Firstly, methods for economic evaluation of health care interventions have existed for a number of years but these evaluations have mainly been applied to more narrowly defined ‘clinical’ interventions, such as drugs. Interventions nowadays are more complex, where benefits cannot be simply measured in terms of health. You can think of areas such as public health, mental health, social care, and end-of-life care, where interventions may result in broader benefits, such as increased control over daily life, independence, or aspects related to the process of health care delivery. Therefore, I believe there is a need to re-think the way we measure and value outcomes when we conduct an economic evaluation. Secondly, ignoring broader outcomes of health care interventions that go beyond the narrow focus of health-related quality of life can potentially lead to misallocation of scarce health care resources. Evidence has shown that the choice of outcome measure (such as a health outcome or a broader measure of wellbeing) can have a significant influence on the conclusions drawn from an economic evaluation.

You use both qualitative and quantitative approaches. Was this key to answering your research questions?

I mainly applied quantitative methods in my thesis research. However, Chapter 3 draws upon some qualitative methodology. To gain a better understanding of ‘benefits beyond health’, I came across a novel approach, called Critical Interpretive Synthesis. It is similar to meta-ethnography (i.e. a synthesis of qualitative research), with the difference that the synthesis is not of qualitative literature but of methodologically diverse literature. It involves an iterative approach, where searching, sampling, and synthesis go hand in hand. It doesn’t only produce a summary of existing literature but enables the development of new interpretations that go beyond those originally offered in the literature. I really liked this approach because it enabled me to synthesise the evidence in a more effective way compared with a conventional systematic review. Defining and applying codes and themes, as it is traditionally done in qualitative research, allowed me to organize the general idea of non-health benefits into a coherent thematic framework, which in the end provided me with a better understanding of the topic overall.

What data did you analyse and what quantitative methods did you use?

I conducted three empirical analyses in my thesis research, which all made use of data from the ICECAP measures (ICECAP-O and ICECAP-A). In my first paper, I used data from the ‘Walk the Talk (WTT)‘ project to investigate the complementarity of the ICECAP-O and the EQ-5D-5L in a public health context using regression analyses. My second paper used exploratory factor analysis to investigate the extent of overlap between the ICECAP-A and five preference-based health-related quality of life measures, using data from the Multi Instrument Comparison (MIC) project. I am currently finalizing submission of my third empirical analysis, which reports findings from a path analysis using cross-sectional data from a web-based survey. The path analysis explores three outcome measurement approaches (health-related quality of life, subjective wellbeing, and capability wellbeing) through direct and mediated pathways in individuals living with spinal cord injury. Each of the three studies addressed different components of the overall research question, which, collectively, demonstrated the added value of broader outcome measures in economic evaluation when compared with existing preference-based health-related quality of life measures.

Thinking about the different measures that you considered in your analyses, were any of your findings surprising or unexpected?

In my first paper, I found that the ICECAP-O is more sensitive to environmental features (i.e. social cohesion and street connectivity) when compared with the EQ-5D-5L. As my second paper has shown, this was not surprising, as the ICECAP-A (a measure for adults rather than older adults) and the EQ-5D-5L measure different constructs and had only limited overlap in their descriptive classification systems. While a similar observation was made when comparing the ICECAP-A with three other preference-based health-related quality of life measures (15D, HUI-3, and SF-6D), a substantial overlap was observed between the ICECAP-A and the AQoL-8D, which suggests that it is possible for broader benefits to be captured by preference-based health-related measures (although some may not consider the AQoL-8D to be exclusively ‘health-related’, despite the label). The findings from the path analysis confirmed the similarities between the ICECAP-A and the AQoL-8D. However, the findings do not imply that the AQoL-8D and ICECAP-A are interchangeable instruments, as a mediation effect was found that requires further research.

How would you like to see your research inform current practice in economic evaluation? Is the QALY still in good health?

I am aware of the limitations of the QALY and although there are increasing concerns that the QALY framework does not capture all benefits of health care interventions, it is important to understand that the evaluative space of the QALY is determined by the dimensions included in preference-based measures. From a theoretical point of view, the QALY can embrace any characteristics that are important for the allocation of health care resources. However, in practice, it seems that QALYs are currently defined by what is measured (e.g. the dimensions and response options of EQ-5D instruments) rather than the conceptual origin. Therefore, although non-health benefits have been largely ignored when estimating QALYs, one should not dismiss the QALY framework but rather develop appropriate instruments that capture such broader benefits. I believe the findings of my thesis have particular relevance for national HTA bodies that set guidelines for the conduct of economic evaluation. While the need to maintain methodological consistency is important, the assessment of the real benefits of some health care interventions would be more accurate if we were less prescriptive in terms of which outcome measure to use when conducting an economic evaluation. As my thesis has shown, some preference-based measures already adopt a broad evaluative space but are less frequently used.

Chris Sampson’s journal round-up for 5th September 2016

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The effect of complementary private health insurance on the use of health care services. International Journal of Health Economics and Management Published 31st August 2016

Moral hazard is one of the key ideas taught to fledgling health economists, but having taken flight you don’t hear all that much about it. That’s because most of us live in Europe, enjoying our universal publicly funded health care systems. But I quite like papers that remind me that moral hazard is still a going concern and that my MSc was relevant. This paper looks at the impact of complementary private health insurance – that is, alongside a national health service. There aren’t so many studies of moral hazard in this setting. Private health insurance (let’s call it PHI) might decrease use of public health care (let’s call it NHS), but it might also increase pressure on the NHS by creating additional demand. For example, people might need a referral from an NHS GP in order to qualify for PHI coverage. This study uses cross-sectional questionnaire data from Denmark, collected from 5447 individuals for the purpose of this study. The questionnaire collected all sorts of data relating to health care use and sociodemographics. People who gave ‘Don’t know’ or ‘Other’ responses were dropped, meaning that only 4362 were analysed. 49% of the sample had PHI – the ‘treatment’ of interest. The authors use a bivariate probit model with propensity score matching to predict health care use. Furthermore, an instrumental variable is used to improve identification. Having PHI seems to increase use of services, with strong effects for prescription medicine, dentist visits and chiropractors. This suggests that PHI coverage may contribute to increasing national health care costs. There are some major limitations to this study, which the authors acknowledge. The response rate was 41%, and the sample wasn’t particularly representative. The one thing I can’t get my head around is the authors’ identification strategy. The instrumental variable chosen was whether or not an individual wears glasses, as in this case PHI is particularly favourable. Even controlling for the covariates used in this analysis, I cannot see (no pun intended) how this could be unrelated to health care use.

The value of disease prevention vs treatment. Journal of Health Economics Published 29th August 2016

The public’s view of pharma just keeps getting worse“, apparently. One probably-entirely-made-up-but-sort-of-reasonable-sounding thing I’ve heard Joe Public say in the past is that Pharma would like us all to remain sickly cash cows. New treatments = milk. Prevention is just… soya. That analogy made no sense, but there are also more reasoned arguments that we spend too much on treatment and too little on prevention. There are also numerous studies characterising people’s preferences regarding prevention and treatment under different conditions. This study builds on this background by developing a utility model of disease valuation in order to derive willingness-to-pay values for reductions in incidence (prevention), mortality (treatment) or deterioration in quality of life (palliative care). The basis for the model is 3 possible states – healthy, ill and dead – through which people can progress in only one direction (i.e. there is no cure). The ‘ill’ state relates to a specific disease and has a value somewhere between 1 (healthy) and 0 (dead). The authors use the model to determine – for example – how willingness to pay for improvement in the ‘ill’ state might be affected by the mortality rate. Two key implications of the model are that i) when the risk of dying from a disease is greater than the incidence rate, prevention is more valuable than treatment and ii) when the incidence rate is greater than the decline in quality of life, prevention is more valuable than palliative care. The model is also used to incorporate probability weighting to give a more realistic characterisation of people’s risk preferences. In most cases, the two previous findings will hold. An interesting finding of this part of the analysis is that it seems to partly explain people’s disproportionately strong preferences for treating more severe diseases. The model suggests that prevention is more valuable than treatment for most real-world situations, and so we’ve probably got the balance all wrong.

Does one size fit all? Assessing the preferences of older and younger people for attributes of quality of life. Quality of Life Research [PubMed] Published 23rd August 2016

There’s plenty of talk nowadays about the idea that QALYs don’t reflect the most important objects of value for particular groups of people, especially older people. Non-health improvements in quality of life might be more important. Whether we’re using EQ-5D, SF-6D, HUI3 or your personally preferred multi-attribute utility measure, the idea is that they’re measuring the same thing. But they’re not. They consistently give systematically different results. This study sought to find out if older people value quality of life attributes used in these measures differently to younger people. The authors elicit preferences for different domains using a web-based survey of two groups of 500 people: over 65s and 18-64 year olds. Individuals were presented with 12 descriptors from the EQ-5D, AQoL and ASCOT and asked to complete both a ranking and a best worst exercise. Socioeconomic data were also collected. The two cohorts ranked the domains differently, but perhaps not as differently as we might expect. ‘Independence’ was important to both groups, with 36% of over 65s and 20% of 18-64 year olds ranking it first. Physical mobility, mental health and pain also ranked highly for both groups. Older people ranked control, self-care and vision more highly than younger people, who in turn ranked safety, social relationships, dignity, sleep and hearing more highly. The results from the ranking exercise and the best worst exercise were similar. So, non-health attributes matter to everyone and older people’s preferences differ to younger people’s. But so what? We could probably find differences between a sample of men and a sample of women, or between an urban and a rural population. The question is: which differences matter? Studies like this are useful, but they can’t tell us how we ought to handle heterogeneous preferences.

From representing views to representativeness of views: Illustrating a new (Q2S) approach in the context of health care priority setting in nine European countries. Social Science & Medicine [PubMedPublished 22nd August 2016

Asking the public what they think; it’s a dangerous game (nb Brexit, Boaty McBoatface, Mrs Brown’s Boys). But there are good grounds for doing so when it comes to health care resource allocation. This paper comes from an ongoing research project that I’ve written about on a couple of occasions. A previous paper used Q methodology and identified 5 viewpoints regarding the fundamental basis for the allocation of resources in health care, titled: 1) ‘egalitarianism, entitlement and equality of access’, 2) ‘severity and the magnitude of health gains’, 3) ‘fair innings, young people and maximising health benefits’, 4) ‘the intrinsic value of life and healthy living’ and 5) ‘quality life is more important than simply staying alive’. This study developed a new methodology called Q2S, designed to extract features from the viewpoints elicited through the original Q study and create a survey to find out how these different viewpoints are represented in society. Data were collected from 39,560 respondents from 9 European countries. Participants were presented with a series of descriptions with which to identify agreement on a 7-point Likert scale from “very unlike my point of view” to “very much like my point of view”. 41% of respondents gave their highest score to a single viewpoint, while the rest tied across two or more viewpoints and were subsequently asked to identify which one would best reflect their view. 43% of respondents were allocated to Viewpoint 1. This viewpoint asserts that health care is a basic right, that treatment effectiveness is essentially irrelevant because all life has the same value, and that scarcity is not a concern. It was predominant in all 9 countries. Gulp! Next up with 17% was Viewpoint 2, which is a bit closer to health maximisation but with a preference for allocation to life-saving treatment and more severe health states. Viewpoint 3 was not popular, with only 4% of people identifying it as most like their point of view. The authors identify various associations between sociodemographic variables and likelihood of particular viewpoints. There’s a lot of food for thought in this paper. Where do you sit? My position changes depending on how revolutionary I’m feeling.

Photo credit: Antony Theobald (CC BY-NC-ND 2.0)

The ‘Q’ in the QALY: are we fudging it?

Some recent research from the Centre for Health Economics at Monash University has quantified something that we are all aware of: fudging in the measurement of health-related quality of life. They have found that, on average, randomly changing from one health-related quality of life measure to another changes cost-effectiveness results by 41%. This is clearly huge.

Health-related quality of life?

I am of the opinion that health-related quality of life is not something that, at least in any objective way, actually exists. The extent to which health-related aspects of life affect overall quality of life differs across people, places and time. Discrepancies can become apparent on two levels:

  1. What we perceive as dimensions of health may or may not affect an individual’s subjective level of overall health, and
  2. The relative importance of health in defining overall quality of life, compared with other aspects of life, can vary. This issue has been addressed in relation to adaptation.

These discrepancies translate into an inconsistency in the valuation processes we currently use. The people from whom values are being elicited are seeking to maximise utility (at least, this is what we assume), while the researcher’s chosen maximand is health-related quality of life. This means that any dimension of the chosen measure that can affect non-health-related quality of life will skew the results. As such we end up with a fudge that combines (objective) health characteristics and (subjective) preferences. I believe that, eventually, we will have to settle on a stricter definition of the ‘Q’ in the QALY, and that this will have to be based entirely in either objective heath or (subjective) utility.

Health

An approach to measuring ‘health’ would not be entirely dissimilar to our current approach, but an ‘objective’ health measure would have to be more comprehensive than the EQ-5D, SF-6D, AQoL and other similar measures. Of existing measures, the 15D comes closest. It could include items such as mobility, sensory function, pain, sexual function, fatigue, anxiety, depression and cognition, which the individual may or may not consider dimensions of their health, but which could define health objectively. These would involve a level of subjectivity in that they are being assessed by the individual, but they are less contextual; items such as self-care, emotion, usual activities and relationships, from current measures, are heavily influenced by the context in which they are being completed. The instrument could then be valued using ranking exercises to establish a complete set of health states, ranked from best health to worst health. Dead can be equated to the worst possible health state, as all other outcomes are, in terms of health, an improvement upon death. If all valuations are completed relative to other health states, rather than to ‘death’, much of the distortion of non-health-related considerations will be removed.

I see no reason why the process should involve the elicitation of preferences. A health service does not seek to maximise utility. Evidence-based medicine does not aim to make people happier. Health care – particularly that which is publicly funded – should seek to maximise health, not health-related quality of life. If a person does not wish to improve their own health in a given way, they can choose not to consume that health care (so long as this is not detrimental to the health of others). For example, an individual may choose not to have a cochlear implant if their social network consists largely of deaf people [thanks, Scrubs]. Surely this should be the role of preferences in the allocation of health care.

Quality of life

At the other end of the scale we have a measure of general well-being. In some respects this is the easier approach, though there remains unanswered questions; for example, do we wish to measure present well-being or satisfaction with life? These approaches are simpler insofar as they require only one question such as, ‘how happy are you right now?’ or ‘how satisfied are you with your life overall?’. These questions should be posed to patients. Again, I do not see any benefit of using preferences or capturing decision utility in this case; experienced utility gives a better indication of the impact of health states upon quality of life. This approach could provide us with a measure of utility, so we could implement a cost-utility analysis (which is by no means what we do currently).

The two approaches described here could be used in conjunction. They would provide very different results, as the early findings from Monash demonstrate. A public health service should maintain health as its maximand, but other government departments or private individuals could provide funding for interventions that also benefit people in ways other than their health, or improve the rate at which individuals can derive utility from their health (e.g. education, social housing, social care).

I have little doubt that our current fudging approach – of maximising mainly-but-not-completely-health-related quality of life – is the best thing to do in the meantime, but I suspect it isn’t a long-term solution.

DOI: 10.6084/m9.figshare.1186883