From time to time, a little puff of steam emerges from my Twitter feed as Paul Kind (one of the founding members of the EuroQol Group and a key figure in the development of the EQ-5D) objects to yet another paper describing the EQ-5D as a ‘preference-based measure’ (PBM). I find myself agreeing with Paul. Of course, this may simply mean I have become old and cranky. While nomenclature isn’t a very exciting topic, I thought it might be useful to set out the reasons why I think the term ‘preference-based measure’ (PBM) is unhelpful and potentially misleading.
What do health economists mean when we use the term PBM?
The Brazier et al (2017) book describes the estimation of the quality of life bit of QALYs as a two-step procedure: describing health states and valuing those descriptions. They go on to say:
“An important class of measures is the generic preference-based measure of health (eg. EQ-5D, HUI3, or SF-6D). These measures have health state descriptive systems that are accompanied by a set of health state utility values.” (p.10)
This is the sense in which ‘PBM’ is used in the literature. However, the word ‘measure’ in ‘PBM‘ is a little vague, because it could mean either (i) a type of questionnaire which is used for patients and populations to self-report their health1, which is distinct from other types of these questionnaires because they are accompanied by utilities to facilitate the estimation of QALYs, or (ii) a measure that arises when the description and valuation elements are combined into a single number. I think that when people write ‘EQ-5D is a PBM’, they mostly mean the first of those two things.
Saying “EQ-5D-3L is a PBM” is basically a convenient way of saying “EQ-5D-3L is a questionnaire, based on an underlying health state classification system comprising 5 dimensions with 3 levels in each, and all 243 states described by it can be summarised by a number based on stated preferences, anchored at 0 and 1, to create a measure of health related quality of life, as required for estimating QALYs”. (And no wonder we need a short cut for saying that!)
But saying “EQ-5D-3L is a PBM” goes further than that; it implies the EQ-5D-3L questionnaire itself is ‘based on preferences’.
Were ‘PBMs’ developed with reference to (‘based on’) people’s preferences?
Well, maybe in the very limited sense that there is an ordinal structure imposed whereby (for example) ‘being able to walk’ is preferred to ‘not being able to walk’– but these ordinal structures apply to all such questionnaires, not just the ones being referred to as PBMs.
What about the choice of dimensions/domains/items in PBMs – are they based on or informed by stated preference methods? Not usually, and not primarily. They do try to capture the aspects of health broadly considered to be most important, and which are valid, reliable, and responsive to change. The process of selecting items and refining these questionnaires generally involves a mix of qualitative research and quantitative (psychometric) testing. But this aim of choosing the most ‘important’ items/domains is common to all questionnaires seeking to measure self-reported health status – both generic and condition-specific – and not just the ones that authors are referring to when they use the term PBM.
Having said that, evidence on stated preferences probably did shape the current form of the EQ-5D: for example, Brooks (1996) reports that there was consideration of an additional ‘energy/tiredness’ dimension, but that this was dropped when it was found to have ‘no effects’ on either self-reported health state (as measured by self-reported VAS) or valuations. But the point is, it is possible to design a concise, generic questionnaire without any reference to stated preferences, and to then subsequently use stated preferences to produce utilities to accompany it. So a questionnaire having been developed using information on stated preferences is neither a necessary nor a sufficient condition for it to be called a PBM.
Most PBMs have in common that they are concise measures; in part because the task of valuing health states using state preference methods requires the descriptions of each state to be comprehensible in order properly to be ‘imagined’. But some PBMs are not especially concise (e.g. PROMIS-29, which is accompanied by utilities in the form of PROPr). And while many PBMs are generic, not all are – there are increasing numbers of examples of condition-specific instruments accompanied by utilities. Hence the earlier quote noting generic PBMs. So being concise and generic is not a strict requirement for being a PBM.
It is also important to note that there are good reasons for wanting a concise, generic measure other than estimating QALYs. A concise, generic descriptive system provides a means of comparing health problems across different disease areas, patient groups, regions or sub-groups of the population. And, importantly, there are lots of ways of analysing the descriptive data generated by questionnaires like the EQ-5D that do not require the use of utilities (as we show here). And there are uses of the EQ-5D (e.g. in clinical contexts) where the use of utilities (which are usually obtained from the general public) might actually be difficult to defend. Describing these instruments as PBMs deflects attention from their usefulness in other contexts where QALYs are not the focus.
PAMs
Still, it’s handy having a term to save words in journal articles and, returning to the earlier quote, to distinguish between the set of questionnaires/’measures’ that “have health state descriptive systems that are accompanied by a set of health state utility values” from those that are not.
My conclusion is that we should replace the term Preference-Based Measure (PBM) with something like Preference-Accompanied Measure (PAM)2. And we should continue to take every possible opportunity to remind users of PAMs that just because they are accompanied by utilities/value sets doesn’t mean that should be the focus of data analysis.
1 While we’re here…
There is also a definitional issue with the widely used term Patient Reported Outcome measure or PRO, which is why I avoided using it above. An ‘outcome’ is, ‘the way a thing turns out; a consequence’. It implies something happened, and there was a consequence – in our case, for someone’s self-reported health. But when we use these questionnaires, they measure self-reported health at a single point in time – it is a ‘snapshot’. It is only their use in repeated measurement, and controlling for other relevant factors, that allows the attribution of change to an intervention, and the identification of an outcome.
2 There’s an in-joke here for my fellow kiwis – PAM’s is a widely-used generic range of groceries in New Zealand.
PAM has made it into a journal! PAM is real! PAM is now! https://link.springer.com/article/10.1007/s41669-021-00270-x
I just searched Google Scholar for “preference accompanied measure” and found 11 papers – all published since this blog post!
[…] study reports on a review to identify which multi-attribute utility instruments (‘preference accompanied measures‘?) are recommended by HTA agencies. The study was funded by the EuroQol group and you won’t be […]
You are totally right Nancy to make sure we question the words we use, and I think you make a good case for a change. I think you hit on a bigger issue here than just definitions though. Chris already hinted at this.
I think the preference-based measure wording started with the SF-6D, and, you could argue, the SF-6D was a preference-based measure because that was its ratio essendi (although interestingly one of John’s earlier paper uses the phrase “developing a preference-based single index”).
With a narrow HTA+QALY focus in mind, this “preference-based measure” wording was probably taken over to the EQ-5D. But as you point out, most papers that describe the EQ-5D should not have this very narrow focus because the EQ-5D should not have this narrow focus (many say)…
Yet, a definition should specify the essential feature of a concept that distinguishes it from other concepts. So should there be a word to distinguish measures that are entirely or mostly designed with the aim of health state valuation in mind? (Are there enough of those measures to justify the new word? Should those measures continue to be developed? Do they need to? Would you make different choices knowing you had to ‘accompany your measure with preferences’ at the end of the development than if not?)
Old and cranky?! Never! How about ‘Partition Based Measure’ https://en.m.wikipedia.org/wiki/Disjoint_sets
Or MECE (mutually exclusive and collectively exhaustive) measure https://en.m.wikipedia.org/wiki/MECE_principle
An interesting case may be the SDQ measure of childhood social problems – not MECE by design but has various domains which I guess could be converted into a MECE classification system if one wanted to.
That’s the problem with MCDA as usually practised by the way – ie mashing up diverse items without worrying about the MECE principle.
Richard C
Great stuff! Totally agree that PAM makes a lot more sense than PBM. And “accompanied” at least has some precedent from the Brazier book. I’d probably tweak it further, as I think “accompanied” still implies some sort of parallel development of questionnaire and value set, which is usually not the case. “Preference augmentable”? Perhaps we should run a survey.