OHE Lunchtime Seminar: An Overview of the US National Institutes of Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS®) and the PROMIS-Preference (PROPr) Scoring System

The EQ-5D, Health Utilities Index, SF-6D, and the Quality of Well-being Index are among the most widely used generic preference-based health-related quality of life measures. However, they each have some of the following limitations: (1) large proportions of the respondents scoring at the very top or very bottom of the scale in some populations of interest (i.e., ceiling effects in the very healthy or floor effects in the very ill), (2) imprecise measurement for individuals, (3) poorly-worded questions such as those that combine two concepts (double-barreled questions), and (4) differences in range of domains covered. While modification of a particular instrument may overcome some of these problems, modification also results in concerns about comparability of results obtained with different versions of the same instrument.
The US NIH Patient Reported Outcomes Measurement Information System (PROMIS®) provides an opportunity to address several limitations of the existing generic preference measures including: (1) fully capturing the entire range of a construct, (2) measuring an individual’s health status with greater precision, and (3) creating a standardised valuation methodology for future studies. The PROMIS® measures stand to be highly applicable across clinical, research, and population studies. Thus, creating a preference-based scoring system for PROMIS would allow efficient use of study resources to collect both health profile and health utility scores.
Combining both psychometric theory (item-response theory; IRT) and econometric theory (multi-attribute utility theory; MAUT), Janel Hanmer will discuss the creation of the PROMIS-Preference (PROPr) scoring system.  She will discuss how IRT is used to create measures of health domains, the linking of IRT calibrated questions to MAUT scoring, and the resulting scoring system.
Janel Hanmer is an Assistant Professor in the Department of Medicine at the University of Pittsburgh.  She is also the Medical Director for Patient Reported Outcomes at the University of Pittsburgh Medical Center. She leads the effort to develop the PROPr scoring system.
If you would like to attend this seminar, please reply to ohegeneral@ohe.org.
Click here to see the full invite.

Webinar facilities will also be available for this lunchtime seminar, however registration is needed. Please send an email to ohegeneral@ohe.org if you wish to join. Details of the webinar will be sent out closer to the event date.

Chris Sampson’s journal round-up for 4th June 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

A qualitative investigation of the health economic impacts of bariatric surgery for obesity and implications for improved practice in health economics. Health Economics [PubMed] Published 1st June 2018

Few would question the ‘economic’ nature of the challenge of obesity. Bariatric surgery is widely recommended for severe cases but, in many countries, the supply is not sufficient to satisfy the demand. In this context, this study explores the value of qualitative research in informing economic evaluation. The authors assert that previous economic evaluations have adopted a relatively narrow focus and thus might underestimate the expected value of bariatric surgery. But rather than going and finding data on what they think might be additional dimensions of value, the authors ask patients. Emotional capital, ‘societal’ (i.e. non-health) impacts, and externalities are identified as theories for the types of value that might be derived from bariatric surgery. These theories were used to guide the development of questions and prompts that were used in a series of 10 semi-structured focus groups. Thematic analysis identified the importance of emotional costs and benefits as part of the ‘socioemotional personal journey’ associated with bariatric surgery. Out-of-pocket costs were also identified as being important, with self-funding being a challenge for some respondents. The information seems useful in a variety of ways. It helps us understand the value of bariatric surgery and how individuals make decisions in this context. This information could be used to determine the structure of economic evaluations or the data that are collected and used. The authors suggest that an EQ-5D bolt-on should be developed for ’emotional capital’ but, given that this ‘theory’ was predefined by the authors and does not arise from the qualitative research as being an important dimension of value alongside the existing EQ-5D dimensions, that’s a stretch.

Developing accessible, pictorial versions of health-related quality-of-life instruments suitable for economic evaluation: a report of preliminary studies conducted in Canada and the United Kingdom. PharmacoEconomics – Open [PubMed] Published 25th May 2018

I’ve been telling people about this study for ages (apologies, authors, if that isn’t something you wanted to read!). In my experience, the need for more (cognitively / communicatively) accessible outcome measures is widely recognised by health researchers working in contexts where this is relevant, such as stroke. If people can’t read or understand the text-based descriptors that make up (for example) the EQ-5D, then we need some alternative format. You could develop an entirely new measure. Or, as the work described in this paper set out to do, you could modify existing measures. There are three descriptive systems described in this study: i) a pictorial EQ-5D-3L by the Canadian team, ii) a pictorial EQ-5D-3L by the UK team, and iii) a pictorial EQ-5D-5L by the UK team. Each uses images to represent the different levels of the different dimensions. For example, the mobility dimension might show somebody walking around unaided, walking with aids, or in bed. I’m not going to try and describe what they all look like, so I’ll just encourage you to take a look at the Supplementary Material (click here to download it). All are described as ‘pilot’ instruments and shouldn’t be picked up and used at this stage. Different approaches were used in the development of the measures, and there are differences between the measures in terms of the images selected and the ways in which they’re presented. But each process referred to conventions in aphasia research, used input from clinicians, and consulted people with aphasia and/or their carers. The authors set out several remaining questions and avenues for future research. The most interesting possibility to most readers will be the notion that we could have a ‘generic’ pictorial format for the EQ-5D, which isn’t aphasia-specific. This will require continued development of the pictorial descriptive systems, and ultimately their validation.

QALYs in 2018—advantages and concerns. JAMA [PubMed] Published 24th May 2018

It’s difficult not to feel sorry for the authors of this article – and indeed all US-based purveyors of economic evaluation in health care. With respect to social judgments about the value of health technologies, the US’s proverbial head remains well and truly buried in the sand. This article serves as a primer and an enticement for the use of QALYs. The ‘concerns’ cited relate almost exclusively to decision rules applied to QALYs, rather than the underlying principles of QALYs, presumably because the authors didn’t feel they could ignore the points made by QALY opponents (even if those arguments are vacuous). What it boils down to is this: trade-offs are necessary, and QALYs can be used to promote value in those trade-offs, so unless you offer some meaningful alternative then QALYs are here to stay. Thankfully, the Institute for Clinical and Economic Review (ICER) has recently added some clout to the undeniable good sense of QALYs, so the future is looking a little brighter. Suck it up, America!

The impact of hospital costing methods on cost-effectiveness analysis: a case study. PharmacoEconomics [PubMed] Published 22nd May 2018

Plugging different cost estimates into your cost-effectiveness model could alter the headline results of your evaluation. That might seems obvious, but there are a variety of ways in which the selection of unit costs might be somewhat arbitrary or taken for granted. This study considers three alternative sources of information for hospital-based unit costs for hip fractures in England: (a) spell-level tariffs, (b) finished consultant episode (FCE) reference costs, and (c) spell-level reference costs. Source (b) provides, in theory, a more granular version of (a), describing individual episodes within a person’s hospital stay. Reference costs are estimated on the basis of hospital activity, while tariffs are prices estimated on the basis of historic reference costs. The authors use a previously reported cohort state transition model to evaluate different models of care for hip fracture and explore how the use of the different cost figures affects their results. FCE-level reference costs produced the highest total first-year hospital care costs (£14,440), and spell-level tariffs the lowest (£10,749). The more FCEs within a spell, the greater the discrepancy. This difference in costs affected ICERs, such that the net-benefit-optimising decision would change. The study makes an important point – that selection of unit costs matters. But it isn’t clear why the difference exists. It could just be due to a lack of precision in reference costs in this context (rather than a lack of accuracy, per se), or it could be that reference costs misestimate the true cost of care across the board. Without clear guidance on how to select the most appropriate source of unit costs, these different costing methodologies represent another source of uncertainty in modelling, which analysts should consider and explore.

Credits

Chris Sampson’s journal round-up for 5th March 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Healthy working days: the (positive) effect of work effort on occupational health from a human capital approach. Social Science & Medicine Published 28th February 2018

If you look at the literature on the determinants of subjective well-being (or happiness), you’ll see that unemployment is often cited as having a big negative impact. The same sometimes applies for its impact on health, but here – of course – the causality is difficult to tease apart. Then, in research that digs deeper, looking at hours worked and different types of jobs, we see less conclusive results. In this paper, the authors start by asserting that the standard approach in labour economics (on which I’m not qualified to comment) is to assume that there is a negative association between work effort and health. This study extends the framework by allowing for positive effects of work that are related to individuals’ characteristics and working conditions, and where health is determined in a Grossman-style model of health capital that accounts for work effort in the rate of health depreciation. This model is used to examine health as a function of work effort (as indicated by hours worked) in a single wave of the European Working Conditions Survey (EWCS) from 2010 for 15 EU member states. Key items from the EWCS included in this study are questions such as “does your work affect your health or not?”, “how is your health in general?”, and “how many hours do you usually work per week?”. Working conditions are taken into account by looking at data on shift working and the need to wear protective equipment. One of the main findings of the study is that – with good working conditions – greater work effort can improve health. The Marxist in me is not very satisfied with this. We need to ask the question, compared to what? Working fewer hours? For most people, that simply isn’t an option. Aren’t the people who work fewer hours the people who can afford to work fewer hours? No attention is given to the sociological aspects of employment, which are clearly important. The study also shows that overworking or having poorer working conditions reduces health. We also see that, for many groups, longer hours do not negatively impact on health until we reach around 120 hours a week. This fails a good sense check. Who are these people?! I’d be very interested to see if these findings hold for academics. That the key variables are self-reported undermines the conclusions somewhat, as we can expect people to adjust their expectations about work effort and health in accordance with their colleagues. It would be very difficult to avoid a type 2 error (with respect to the negative impact of effort on health) using these variables to represent health and the role of work effort.

Agreement between retrospectively and contemporaneously collected patient-reported outcome measures (PROMs) in hip and knee replacement patients. Quality of Life Research [PubMed] Published 26th February 2018

The use of patient-reported outcomes (PROMs) in elective care in the NHS has been a boon for researchers in our field, providing before-and-after measurement of health-related quality of life so that we can look at the impact of these interventions. But we can’t do this in emergency care because the ‘before’ is never observed – people only show up when they’re in the middle of the emergency. But what if people could accurately recall their pre-emergency health state? There’s some evidence to suggest that people can, so long as the recall period is short. This study looks at NHS PROMs data (n=443), with generic and condition-specific outcomes collected from patients having hip or knee replacements. Patients included in the study were additionally asked to recall their health state 4 weeks prior to surgery. The authors assess the extent to which the contemporary PROM measurements agree with the retrospective measurements, and the extent to which any disagreement relates to age, socioeconomic status, or the length of time to recall. There wasn’t much difference between contemporary and retrospective measurements, though patients reported slightly lower health on the retrospective questionnaires. And there weren’t any compelling differences associated with age or socioeconomic status or the length of recall. These findings are promising, suggesting that we might be able to rely on retrospective PROMs. But the elective surgery context is very different to the emergency context, and I don’t think we can expect the two types of health care to impact recollection in the same way. In this study, responses may also have been influenced by participants’ memories of completing the contemporary questionnaire, and the recall period was very short. But the only way to find out more about the validity of retrospective PROM collection is to do more of it, so hopefully we’ll see more studies asking this question.

Adaptation or recovery after health shocks? Evidence using subjective and objective health measures. Health Economics [PubMed] Published 26th February 2018

People’s expectations about their health can influence their behaviour and determine their future health, so it’s important that we understand people’s expectations and any ways in which they diverge from reality. This paper considers the effect of a health shock on people’s expectations about how long they will live. The authors focus on survival probability, measured objectively (i.e. what actually happens to these patients) and subjectively (i.e. what the patients expect), and the extent to which the latter corresponds to the former. The arguments presented are couched within the concept of hedonic adaptation. So the question is – if post-shock expectations return to pre-shock expectations after a period of time – whether this is because people are recovering from the disease or because they are moving their reference point. Data are drawn from the Health and Retirement Study. Subjective survival probability is scaled to whether individuals expect to survive for 2 years. Cancer, stroke, and myocardial infarction are the health shocks used. The analysis uses some lagged regression models, separate for each of the three diagnoses, with objective and subjective survival probability as the dependent variable. There’s a bit of a jumble of things going on in this paper, with discussions of adaptation, survival, self-assessed health, optimism, and health behaviours. So it’s a bit difficult to see the wood for the trees. But the authors find the effect they’re looking for. Objective survival probability is negatively affected by a health shock, as is subjective survival probability. But then subjective survival starts to return to pre-shock trends whereas objective survival does not. The authors use this finding to suggest that there is adaptation. I’m not sure about this interpretation. To me it seems as if subjective life expectancy is only weakly responsive to changes in objective life expectancy. The findings seem to have more to do with how people process information about their probability of survival than with how they adapt to a situation. So while this is an interesting study about how people process changes in survival probability, I’m not sure what it has to do with adaptation.

3L, 5L, what the L? A NICE conundrum. PharmacoEconomics [PubMed] Published 26th February 2018

In my last round-up, I said I was going to write a follow-up blog post to an editorial on the EQ-5D-5L. I didn’t get round to it, but that’s probably best as there has since been a flurry of other editorials and commentaries on the subject. Here’s one of them. This commentary considers the perspective of NICE in deciding whether to support the use of the EQ-5D-5L and its English value set. The authors point out the differences between the 3L and 5L, namely the descriptive systems and the value sets. Examples of the 5L descriptive system’s advantages are provided: a reduced ceiling effect, reduced clustering, better discriminative ability, and the benefits of doing away with the ‘confined to bed’ level of the mobility domain. Great! On to the value set. There are lots of differences here, with 3 main causes: the data, the preference elicitation methods, and the modelling methods. We can’t immediately determine whether these differences are improvements or not. The authors stress the point that any differences observed will be in large part due to quirks in the original 3L value set rather than in the 5L value set. Nevertheless, the commentary is broadly supportive of a cautionary approach to 5L adoption. I’m not. Time for that follow-up blog post.

Credits