Thesis Thursday: Lidia Engel

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Lidia Engel who graduated with a PhD from Simon Fraser University. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Going beyond health-related quality of life for outcome measurement in economic evaluation
David Whitehurst, Scott Lear, Stirling Bryan
Repository link

Your thesis explores the potential for expanding the ‘evaluative space’ in economic evaluation. Why is this important?

I think there are two answers to this question. Firstly, methods for economic evaluation of health care interventions have existed for a number of years but these evaluations have mainly been applied to more narrowly defined ‘clinical’ interventions, such as drugs. Interventions nowadays are more complex, where benefits cannot be simply measured in terms of health. You can think of areas such as public health, mental health, social care, and end-of-life care, where interventions may result in broader benefits, such as increased control over daily life, independence, or aspects related to the process of health care delivery. Therefore, I believe there is a need to re-think the way we measure and value outcomes when we conduct an economic evaluation. Secondly, ignoring broader outcomes of health care interventions that go beyond the narrow focus of health-related quality of life can potentially lead to misallocation of scarce health care resources. Evidence has shown that the choice of outcome measure (such as a health outcome or a broader measure of wellbeing) can have a significant influence on the conclusions drawn from an economic evaluation.

You use both qualitative and quantitative approaches. Was this key to answering your research questions?

I mainly applied quantitative methods in my thesis research. However, Chapter 3 draws upon some qualitative methodology. To gain a better understanding of ‘benefits beyond health’, I came across a novel approach, called Critical Interpretive Synthesis. It is similar to meta-ethnography (i.e. a synthesis of qualitative research), with the difference that the synthesis is not of qualitative literature but of methodologically diverse literature. It involves an iterative approach, where searching, sampling, and synthesis go hand in hand. It doesn’t only produce a summary of existing literature but enables the development of new interpretations that go beyond those originally offered in the literature. I really liked this approach because it enabled me to synthesise the evidence in a more effective way compared with a conventional systematic review. Defining and applying codes and themes, as it is traditionally done in qualitative research, allowed me to organize the general idea of non-health benefits into a coherent thematic framework, which in the end provided me with a better understanding of the topic overall.

What data did you analyse and what quantitative methods did you use?

I conducted three empirical analyses in my thesis research, which all made use of data from the ICECAP measures (ICECAP-O and ICECAP-A). In my first paper, I used data from the ‘Walk the Talk (WTT)‘ project to investigate the complementarity of the ICECAP-O and the EQ-5D-5L in a public health context using regression analyses. My second paper used exploratory factor analysis to investigate the extent of overlap between the ICECAP-A and five preference-based health-related quality of life measures, using data from the Multi Instrument Comparison (MIC) project. I am currently finalizing submission of my third empirical analysis, which reports findings from a path analysis using cross-sectional data from a web-based survey. The path analysis explores three outcome measurement approaches (health-related quality of life, subjective wellbeing, and capability wellbeing) through direct and mediated pathways in individuals living with spinal cord injury. Each of the three studies addressed different components of the overall research question, which, collectively, demonstrated the added value of broader outcome measures in economic evaluation when compared with existing preference-based health-related quality of life measures.

Thinking about the different measures that you considered in your analyses, were any of your findings surprising or unexpected?

In my first paper, I found that the ICECAP-O is more sensitive to environmental features (i.e. social cohesion and street connectivity) when compared with the EQ-5D-5L. As my second paper has shown, this was not surprising, as the ICECAP-A (a measure for adults rather than older adults) and the EQ-5D-5L measure different constructs and had only limited overlap in their descriptive classification systems. While a similar observation was made when comparing the ICECAP-A with three other preference-based health-related quality of life measures (15D, HUI-3, and SF-6D), a substantial overlap was observed between the ICECAP-A and the AQoL-8D, which suggests that it is possible for broader benefits to be captured by preference-based health-related measures (although some may not consider the AQoL-8D to be exclusively ‘health-related’, despite the label). The findings from the path analysis confirmed the similarities between the ICECAP-A and the AQoL-8D. However, the findings do not imply that the AQoL-8D and ICECAP-A are interchangeable instruments, as a mediation effect was found that requires further research.

How would you like to see your research inform current practice in economic evaluation? Is the QALY still in good health?

I am aware of the limitations of the QALY and although there are increasing concerns that the QALY framework does not capture all benefits of health care interventions, it is important to understand that the evaluative space of the QALY is determined by the dimensions included in preference-based measures. From a theoretical point of view, the QALY can embrace any characteristics that are important for the allocation of health care resources. However, in practice, it seems that QALYs are currently defined by what is measured (e.g. the dimensions and response options of EQ-5D instruments) rather than the conceptual origin. Therefore, although non-health benefits have been largely ignored when estimating QALYs, one should not dismiss the QALY framework but rather develop appropriate instruments that capture such broader benefits. I believe the findings of my thesis have particular relevance for national HTA bodies that set guidelines for the conduct of economic evaluation. While the need to maintain methodological consistency is important, the assessment of the real benefits of some health care interventions would be more accurate if we were less prescriptive in terms of which outcome measure to use when conducting an economic evaluation. As my thesis has shown, some preference-based measures already adopt a broad evaluative space but are less frequently used.

The potential of the super QALY to reconcile the key contentions in health economics

Economics is largely about trade-offs and compromise. Academics study the former but don’t often engage in the latter. In health economics, as in other fields, a key trade-off is between equity and efficiency. We’ve been studying this for a.very.long.time. Despite this, as Culyer has identified, equity is hardly considered in current health technology assessments. We all agree it should be, but just can’t seem to figure it out. Indeed, ihas been argued that incorporating equity concerns into cost-effectiveness analyses could still be a long time coming.

But let’s be a bit more positive. The elusive `Super QALY’, as it has been described, should come eventually. And when it does, it’ll be great! One of the reasons, I propose here, is that it has the power to reconcile many of the disagreements that currently fuel (hamper?) debate in our field. Hence, the super QALY might just allow us to get on with fussing over minutia issues of economic evaluation.


There are necessary trade-offs in decisions of resource allocation. These might be described as the ‘positive’ tensions economists deal with; they relate to decisions that must be made, regardless of our values. The equity–efficiency trade-off is the main one here. But there are others. For example, health care interventions have the dual aim of increasing both the quantity and quality of an individual’s life. The QALY attempts to address this. However, the way we value quality of life also incorporates considerations of length of life in so much as ‘death’ is used in the valuation of health states. This is problematic, as has been discussed. Economists haven’t really gotten round to disagreeing about this yet, but there’s plenty else on which we disagree.


These might be described as ‘normative’ tensions. They concern what different economists think should and should not be done; mainly relating to the process of valuing health states. There are welfarists and non-welfarists. There are those who support societal preferences, and those who support capturing patient experience. It should be clear to most that neither side in these debates is wrong. Most health economists acknowledge the value of capturing utility as well as the importance of capabilities. Most will attach some value to society’s preferences and some to those of the individual.

A super-QALY solution

It’s never been completely clear what the ‘extra’ in extra-welfarism (as currently practiced) actually consists. The super QALY will surely formalise this; it could involve some completely non-welfarist notions. The most common idea of the super QALY is one where the current health-related QALY is weighted based on some equity considerations. So, if this is where economic evaluation is heading, we’re likely to end up with an extra step of estimating the equity impact of an intervention. But, while most studies seem to suggest that this might just be an add-on process, I think it would require a realignment of the methods we already use.

Equity analysis

There’s no need for me to reiterate the importance of equity considerations. Plainly we (economists, the public) care about needs, capabilities, opportunities and equality. How we define the equity analysis is incidental. More important is that we get on with doing it and just see what happens. There are lots of measures we could use and different approaches we could take. For arguments sake (and because I quite like it), let’s say the equity analysis is characterised by a ‘minimum capabilities‘ approach. Something similar to Daniels’s normal opportunity range. People could have the normal opportunity range, have fewer opportunities or have more opportunities. We can argue later about where the threshold lies. People below the threshold could be said to be in ‘need’. Again, argue about this later. States could be defined using a capabilities measure; let’s just say the ICECAP-A for now (though I don’t much like it). Here in the world of health economics we like 0-1 scales, so the ICECAP-A could be valued based on these anchors. So, let’s say 1 is the minimum capabilities or normal opportunity range threshold. Zero equates to being dead. Values can drop below zero where opportunity sets represent a state worse that non-existence. For the equity analysis we are not interested in utility or satisfaction, so the valuation would not be by the individual. Values could be elicited from society, possibly. The valuation technique could be a person trade-off, maybe. Or we could let ethicists come up with weightings. This framework, surely, would satisfy the non-welfarists.

Health utility analysis

I see no reason why the estimation of health benefits cannot be utility-based. Utilitarian satisfaction is sufficient if non-welfarist concerns are incorporated in an equity analysis. Personally I believe that whether this is based on experiences or preferences is largely inconsequential and that, in terms of health, most of the differences demonstrated between the 2 are a function of the elicitation methods. Therefore, utility analysis would remain largely unchanged. However, the value of 0 would change. Zero currently represents either being dead or in a health state equivalent to being dead, despite these two things not being of equivalent value to a person. Under the new framework there is no need to incorporate death into the health utility analysis, as it is accounted for in the equity analysis. 0 should represent the worst health state imaginable. There would be no negative values.

Cost-effectiveness analysis

These 2 analyses would then be combined to form a relatively routine cost-effectiveness analysis to address the efficiency of the intervention. The QALY would be calculated in the usual way, but the ‘Q’ would become ‘super’ by being a function of the 2 different outcomes. Tentatively this could be done by multiplying the two values (alternative formulations could be defined by societal values or by ethicists, depending on your wont). Costings would be carried out in the usual manner and a super ICER could be calculated. Furthermore, the net benefit approach could be implemented in the usual way; possibly with separate willingness-to-pay values for each input to the super QALY (indeed, they may be willingness to pay values from different agents). The table below summarises how the approach might accommodate the various tensions in health economics.

Equity analysis Health utility analysis
Equity Effectiveness
Life Morbidity
Non-welfarism Welfarism
Flourishing Satisfaction
Society The individual

All public policies could be subject to an equity analysis in the way set out above. It is in no way health-specific. Each policy field could then us this to weight their usual outcomes measures – preferably utility-based – to estimate the cost-effectiveness of their intervention. At this point the super QALY makes it onto daytime TV and health economists form a new unelected chamber at the Palace of Westminster.

No doubt this explicitly extra-welfarist approach to the super QALY raises more questions than it is currently able to answer, but we need to get on with trying stuff like this. The super QALY has proven elusive to date but, if we do make it, it may solve a lot of our problems. We may find ourselves having to invent new things to argue about.