Don Husereau’s journal round-up for 25th November 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Development and validation of the TRansparent Uncertainty ASsessmenT (TRUST) tool for assessing uncertainties in health economic decision models. PharmacoEconomics [PubMed] Published 11th November 2019

You’re going to quickly see that all three papers in today’s round-up align with some strong personal pet peeves that I harbour toward the nebulous world of market access and health technology assessment – most prominent is how loose we seem to be with language and form without overarching standards. This may be of no surprise to some when discussing a field which lacks a standard definition and for which many international standards of what constitutes good practice have never been defined.

This first paper deals with both issues and provides a useful tool for characterizing uncertainty. The authors state the purpose of the tool is “for systematically identifying, assessing, and reporting uncertainty in health economic models.” They suggest, to the best of their knowledge, no such tool exists. They also support the need for the tool by asserting that uncertainty in health economic modelling is often not fully characterized. The reasons, they suggest, are twofold: (1) there has been too much emphasis on imprecision; and (2) it is difficult to express all uncertainty.

I couldn’t agree more. What I sometimes deeply believe about those planning and conducting economic evaluation is that they obsess too often about uncertainty that is is less relevant (but more amenable to statistical adjustment) and don’t address uncertainty that payers actually care about. To wit, while it may be important to explore and adopt methods that deal with imprecision (dealing with known unknowns), such as improving utility variance estimates (from an SE of 0.003 to 0.011, yes sorry Kelvin and Feng for the callout), not getting this right is unlikely to lead to truly bad decisions. (Kelvin and Feng both know this.)

What is much more important for decision makers is uncertainty that stems from a lack of knowledge. These are unknown unknowns. In my experience this typically has to do with generalizability (how well will it work in different patients or against a different comparator?) and durability (how do I translate 16 weeks of data into a lifetime?); not things resolved by better variance estimates and probabilistic analysis. In Canada, our HTA body has even gone so far as to respond to the egregious act of not providing different parametric forms for extrapolation with the equally egregious act of using unrealistic time horizon adjustments to deal with this. Two wrongs don’t make a right.

To develop the tool, the authors first conducted a (presumably narrative) review of uncertainty frameworks and then ran identified concepts across a bunch of HTA expert committee types. They also used a previously developed framework as a basis for identifying all the places where uncertainty in HTA could occur. Using the concepts and the HTA areas they developed a tool which was presented a few times, and then validated through semi-structured interviews with different international stakeholders (N = 11), as well as insights into barriers to its use, user-friendliness, and feasibility.

Once the tool was developed, six case studies were worked up with an illustration of one of them (pembrolizumab for Hodgkin’s lymphoma) in the manuscript. While the tool does not provide a score or coefficient to adjust estimates or deal with uncertainty, it is not supposed to. What it is trying to do is make sure you are aware of them all so that you can make some determination as to whether the uncertainties are dealt with. One of the challenges of developing the tool is the lack of standardized terminology regarding uncertainty itself. While a short primer exists in the manuscript, for those who have looked into it, uncertainty terminology is far more uncertain than even the authors let on.

While I appreciate the tool and the attempt to standardize things, I do suspect the approach could have been strengthened (a systematic review and possibly a nominal group technique as is done for reporting guidelines). However, I’m not sure this would have gotten us much closer to the truth. Uncertainty needs to be sorted first and I am happy at their attempt. I hope it raises some awareness of how we can’t simply say we are “uncertain” as if that means something.

Unmet medical need: an introduction to definitions and stakeholder perceptions. Value in Health [PubMed] Published November 2019

The second, and also often-abused, term without an obvious definition is unmet medical need (UMN). My theory is that some confusion has arisen due to a confluence of marketing and clinical development teams and regulators. UMN has come to mean patients with rare diseases, drugs with ‘novel’ mechanisms of action, patients with highly prevalent disease, drugs with a more convenient formulation, or drugs with fewer side effects. And yet payers (in my experience) usually recognize none of these. Payers tend to characterize UMN in different ways: no drugs available to treat the condition, available drugs do not provide consistent or durable responses, and there have been no new medical developments in the area for > 10 years.

The purpose of this research then was to unpack the term UMN further. The authors conducted a comprehensive (gray) literature review to identify definitions of UMN in use by different stakeholders and then unpacked their meaning through definitions consultations with multi-European stakeholder discussions, trying to focus on the key elements of unmet medical need with a regulatory and reimbursement lens. This consisted of six one-hour teleconference calls and two workshops held in 2018. One open workshop involved 69 people from regulatory agencies, industry, payers, HTA bodies, patient organizations, healthcare, and academia.

A key finding of this work was that, yes indeed, UMN means different things to different people. A key dimension is whether unmet need is being defined in terms of individuals or populations. Population size (whether prevalent or rare) was not felt to be an element of the definition while there was general consensus that disease severity was. This means UMN should really only consider the UMNs of individual patients, not whether very few or very many patients are at need. It also means we see people who have higher rates of premature mortality and severe morbidity as having more of an unmet need, regardless of how many people are affected by the condition.

And last but not least was the final dimension of how many treatments are actually available. This, the authors point out, is the current legal definition in Europe (as laid down in Article 4, paragraph 2 of Commission Regulation [EC] No. 507/2006). And while this seems the most obvious definition of ‘need’ (we usually need things that are lacking) there was some acknowledgement by stakeholders that simply counting existing therapies is not adequate. There was also acknowledgement that there may be existing therapies available and still an UMN. Certainly this reflects my experience on the pan-Canadian Oncology Drug Review expert review committee, where unmet medical need was an explicit subdomain in their value framework, and where on more than one occasion it was felt, to my surprise, there was an unmet need despite the availability of two or more treatments.

Like the previous paper, the authors did not conduct a systematic review and could have consulted more broadly (no clinician stakeholders were consulted) or used more objective methods, a limitation they acknowledge but also unlikely to get them much further ahead in understanding. So what to do with this information? Well, the authors do propose an HTA approach that would triage reimbursement decision based on UMN. However, stakeholders commented that the method you use really depends on the HTA context. As such, the authors conclude that “the application of the definition within a broader framework depends on the scope of the stakeholder.” In other words, HTA must be fit for purpose (something we knew already). However, like uncertainty, I’m happy someone is actually trying to create reasonable coherent definitions of such an important concept.

On value frameworks and opportunity costs in health technology assessment. International Journal of Technology Assessment in Health Care [PubMed] Published 18th September 2019

The final, and most-abused term is that of ‘value’. While value seems an obvious prerequisite to those making investments in healthcare, and that we (some of us) are willing to acknowledge that value is what we are willing to give up to get something, what is less clear is what we want to get and what we want to give up.

The author of this paper, then, hopes to remind us of the various schools of thought on defining value in health that speak to these trade-offs. The first is broadly consistent with the welfarist school of economics and proposes that the value of health care used by decision makers should reflect individuals’ willingness to pay for it. An alternative approach – sometimes referred to as the extra-welfarist framework, argues that the value of a health technology should be consistent with the policy objectives of the health care system, typically health (the author states it is ‘health’ but I’m not sure it has to be). The final school of thought (which I was not familiar with and neither might you be which is the point of the paper) is what he terms ‘classical’, where the point is not to maximize a maximand or be held up to notions of efficiency but rather to discuss how consumers will be affected. The reference cited to support this framework is this interesting piece although I couldn’t find any allusion to the framework within.

What follows is a relatively fair treatment of extra-welfarist and welfarist applications to decision-making with a larger critical swipe at the former (using legitimate arguments that have been previously published – yes, extra-welfarists assume resources are divisible and, yes, extra-welfarists don’t identify the health-producing resources that will actually be displaced and, yes, using thresholds doesn’t always maximize health) and much downplay of the latter (how we might measure trade-offs reliably under a welfarist framework appears to be a mere detail until this concession is finally mentioned: “On account of the measurement issues surrounding [willingness to pay], there may be many situations in which no valid and reliable methods of operationalizing [welfarist economic value frameworks] exist.”) Given the premise of this commentary is that a recent commentary by Culyer seemed to overlook concepts of value beyond extra-welfarist ones, the swipe at extra-welfarist views is understandable. Hence, this paper can be seen as a kind of rebuttal and reminder that other views should not be ignored.

I like the central premise of the paper as summarized here:

“Although the concise term “value for money” may be much easier to sell to HTA decision makers than, for example, “estimated mean valuation of estimated change in mean health status divided by the estimated change in mean health-care costs,” the former loses too much in precision; it seems much less honest. Because loose language could result in dire consequences of economic evaluation being oversold to the HTA community, it should be avoided at all costs”

However, while I am really sympathetic to warning against conceptual shortcuts and loose language, I wonder if this paper misses the bigger point. Firstly, I’m not convinced we are making such bad decisions as those who wish the lambda to be silenced tend to want us to believe. But more importantly, while it is easy to be critical about economics applied loosely or misapplied, this paper (like others) offers no real practical solutions other than the need to acknowledge other frameworks. It is silent on the real reason extra-welfarist approaches and thresholds seem to have stuck around, namely, they have provided a practical and meaningful way forward for difficult decision-making and the HTA processes that support them. They make sense to decision-makers who are willing to overlook some of the conceptual wrinkles. And I’m a firm believer that conceptual models are a starting point for pragmatism. We shouldn’t be slaves to them.

Credits

Chris Sampson’s journal round-up for 18th November 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

A conceptual map of health-related quality of life dimensions: key lessons for a new instrument. Quality of Life Research [PubMed] Published 1st November 2019

EQ-5D, SF-6D, HUI3, AQoL, 15D; they’re all used to describe health states for the purpose of estimating health state utility values, to get the ‘Q’ in the QALY. But it’s widely recognised (and evidenced) that they measure different things. This study sought to better understand the challenge by doing two things: i) ‘mapping’ the domains of the different instruments and ii) advising on the domains to be included in a new measure.

The conceptual model described in this paper builds on two standard models of health – the ICF (International Classification of Functioning, Disability, and Health), which is endorsed by the WHO, and the Wilson and Cleary model. The new model is built around four distinctions, which can be used to define the dimensions included in health state utility instruments: cause vs effect, specific vs broad, physical vs psychological, and subjective vs objective. The idea is that each possible dimension of health can relate, with varying levels of precision, to one or the other of these alternatives.

The authors argue that, conveniently, cause/effect and specific/broad map to one another, as do physical/psychological and objective/subjective. The framework is presented visually, which makes it easy to interpret – I recommend you take a look. Each of the five instruments previously mentioned is mapped to the framework, with the HUI and 15D coming out as ‘symptom’ oriented, EQ-5D and SF-6D as ‘functioning’ oriented, and the AQoL as a hybrid of a health and well-being instrument. Based (it seems) on the Personal Wellbeing Index, the authors also include two social dimensions in the framework, which interact with the health domains. Based on the frequency with which dimensions are included in existing instruments, the authors recommend that a new measure should include three physical dimensions (mobility, self-care, pain), three mental health dimensions (depression, vitality, sleep), and two social domains (personal relationships, social isolation).

This framework makes no sense to me. The main problem is that none of the four distinctions hold water, let alone stand up to being mapped linearly to one another. Take pain as an example. It could be measured subjectively or objectively. It’s usually considered a physical matter, but psychological pain is no less meaningful. It may be a ‘causal’ symptom, but there is little doubt that it matters in and of itself as an ‘effect’. The authors themselves even offer up a series of examples of where the distinctions fall down.

It would be nice if this stuff could be drawn-up on a two-dimensional plane, but it isn’t that simple. In addition to oversimplifying complex ideas, I don’t think the authors have fully recognised the level of complexity. For instance, the work seems to be inspired – at least in part – by a desire to describe health state utility instruments in relation to subjective well-being (SWB). But the distinction between health state utility instruments and SWB isn’t simply a matter of scope. Health state utility instruments (as we use them) are about valuing states in relation to preferences, whereas SWB is about experienced utility. That’s a far more important and meaningful distinction than the distinction between symptoms and functioning.

Careless costs related to inefficient technology used within NHS England. Clinical Medicine Journal [PubMed] Published 8th November 2019

This little paper – barely even a single page – was doing the rounds on Twitter. The author was inspired by some frustration in his day job, waiting for the IT to work. We can all relate to that. This brief analysis sums the potential costs of what the author calls ‘careless costs’, which is vaguely defined as time spent by an NHS employee on activity that does not relate to patient care. Supposing that all doctors in the English NHS wasted an average of 10 minutes per day on such activities, it would cost over £143 million (per year, I assume) based on current salaries. The implication is that a little bit of investment could result in massive savings.

This really bugs me, for at least two reasons. First, it is normal for anybody in any profession to have a bit of downtime. Nobody operates at maximum productivity for every minute of every day. If the doctor didn’t have their downtime waiting for a PC to boot, it would be spent queuing in Costa, or having a nice relaxed wee. Probably both. Those 10 minutes that are displaced cannot be considered equivalent in value to 10 minutes of patient contact time. The second reason is that there is no intervention that can fix this problem at little or no cost. Investments cost money. And if perfect IT systems existed, we wouldn’t all find these ‘careless costs’ so familiar. No doubt, the NHS lags behind, but the potential savings of improvement may very well be closer to zero than to the estimates in this paper.

When it comes to clinical impacts, people insist on being able to identify causal improvements from clearly defined interventions or changes. But when it comes to costs, too many people are confident in throwing around huge numbers of speculative origin.

Socioeconomic disparities in unmet need for student mental health services in higher education. Applied Health Economics and Health Policy [PubMed] Published 5th November 2019

In many countries, the size of the student population is growing, and this population seems to have a high level of need for mental health services. There are a variety of challenges in this context that make it an interesting subject for health economists to study (which is why I do), including the fact that universities are often the main providers of services. If universities are going to provide the right services and reach the right people, a better understanding of who needs what is required. This study contributes to this challenge.

The study is set in the context of higher education in Ireland. If you have no idea how higher education is organised in Ireland, and have an interest in mental health, then the Institutional Context section of this paper is worth reading in its own right. The study reports on findings from a national survey of students. This analysis is a secondary analysis of data collected for the primary purpose of eliciting students’ preferences for counselling services, which has been described elsewhere. In this paper, the authors report on supplementary questions, including measures of psychological distress and use of mental health services. Responses from 5,031 individuals, broadly representative of the population, were analysed.

Around 23% of respondents were classified as having unmet need for mental health services based on them reporting both a) severe distress and b) not using services. Arguably, it’s a sketchy definition of unmet need, but it seems reasonable for the purpose of this analysis. The authors regress this binary indicator of unmet need on a selection of sociodemographic and individual characteristics. The model is also run for the binary indicator of need only (rather than unmet need).

The main finding is that people from lower social classes are more likely to have unmet need, but that this is only because these people have a higher level of need. That is, people from less well-off backgrounds are more likely to have mental health problems but are no less likely to have their need met. So this is partly good news and partly bad news. It seems that there are no additional barriers to services in Ireland for students from a lower social class. But unmet need is still high and – with more inclusive university admissions – likely to grow. Based on the analyses, the authors recommend that universities could reach out to male students, who have greater unmet need.

Credits

Thesis Thursday: Elizabeth Lemmon

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Elizabeth Lemmon who has a PhD from the University of Stirling. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
Essays on the provision of long term care to older adults in Scotland
Supervisors
David Bell, Alasdair Rutherford
Repository link
http://hdl.handle.net/1893/29369

What long term care provision is available to older people in Scotland?

Long term care (LTC) in Scotland comprises both formal care and unpaid care. Formal care encompasses care provided by professionals in a person’s own home as well as care in a residential care setting. Unpaid care is care that is provided by family members, friends, or neighbours. Long term care is provided to older people who need help because they are ill, frail or have a disability. It might mean help with more administrative tasks such as filling in forms, paying bills, shopping, and housework, but can also mean help with things of a more personal nature such as washing and dressing. Since 2002, individuals living in Scotland aged 65 or over are entitled to free personal care (FPC) at home, subject to a needs assessment. This makes Scotland quite different to England, where personal care costs are borne by the service user and their families, and provides a unique opportunity to conduct research.

What were the pros and cons of your chosen data sources?

I used three data sources in my PhD. Those included the Family Resources Survey (FRS), the Scottish Government’s administrative Social Care Survey (SCS) and publically available data zone level data. The key benefit of using survey data like the FRS was that they captured information about care recipients and their caregivers. I used these data in my third paper to look at unpaid carers’ Standard of Living (SoL). The down side of the FRS is that it only captures a subset of the population, which might be systematically different from the population at large. At the same time, although there is information on carers and the person they are caring for, this information is very limited for those who are not living with the care recipient. On the other hand, the benefit of using the SCS, which I used in my first two papers, is that it captures population level information about the provision of LTC services. However, unlike the FRS, the SCS was designed for administrative purposes meaning that it lacks the richness of information on client circumstances and characteristics. One solution to this is to use data zone level information as a proxy for those characteristics, but often this is not enough. Overall, the PhD highlighted both the strengths and weaknesses of working with these different data sources, pointing to the potential of using linked administrative and survey data in future research.

How did you identify sources of inequity in the provision of long term care?

Inequity in the provision of LTC exists if there are differences in LTC provision after accounting for differences in need. I investigated this issue of inequity in my first paper. In particular, we observed from the raw data that there are big differences in FPC provision between the 32 Scottish local authorities. As I mentioned, FPC is available to anyone in Scotland aged 65+ who needs it. Perhaps those differences are due to differences in need. But I didn’t find that this was the case. It seemed that, even after accounting for the need of local authority populations, via the proportion of disability benefit claims, there were still large differences in provision of FPC. I modelled this inequity using a simple regression framework. The results from the regressions found that there is evidence of geographic inequity in the provision of FPC in Scotland. In particular, the analysis suggests that the differences between the FPC rate and the rate of disability are not consistent across local authorities, suggesting that a needy individual might be more or less likely to receive care depending on where they live. One explanation for this is that local authorities differ in terms of their practices for managing the demand for FPC. However, this is an area that would require more detailed investigation with individual local authorities to understand their practices.

What is the role of unpaid care, and how did you model that?

Unpaid care is defined as care that is provided by family members, partners, or friends to those who require help because they are ill, frail, or have a disability. The care that they provide is unpaid and often considered as having a zero cost in economic evaluations. This might lead to inefficient resource allocation and poor policy decisions. In my second paper, I tried to model the effect that unpaid carers have on the FPC use of the cared for. This was difficult due to the potential reverse causality that occurs between the two. I compared different models to try to estimate this effect. Overall, my findings suggest that unpaid carers likely complement FPC services in Scotland. This relationship might be due to unpaid carers advocating on behalf of the cared for, and demanding services from the local authority for them. They might do this because they require more support to enable them to remain in the labour force. It could also be that the type of care unpaid carers provide is different to that provided by formal carers.

Why did you use a ‘standard of living’ approach and what can it tell us about the cost of unpaid care?

The motivation for using the SoL approach, as implemented by Morciano et al (2015), was really that we felt it might capture more of the trade-offs that are involved in providing care. Specifically, it is expected that unpaid carers have to divert resources in order to pay for goods and services for the person they are caring for. Compared to the conventional costing methods which have focused on attaching a monetary value to the time a carer gives up in order to provide care, we argue that the SoL approach may capture a wider array of the trade-offs that are involved in providing unpaid care. For example, are unpaid carers less able to afford to go on holiday or to take part in a regular leisure activity? If it is the case that unpaid carers have to invest resources into providing care then they might have fewer resources to devote to their own needs and wants, resulting in unpaid carers having a lower SoL compared to non-carers. The results suggest that unpaid carers who are living with the person they are caring for would require compensation of £229 per week in order for them to reach the same SoL as a non-carer.

What are the key steps necessary to identify and address unmet need in this context?

My research highlighted that there is possibly unmet need for FPC in Scotland and that this could potentially be more likely for older people who don’t have an unpaid carer helping them to access FPC services. Understanding this unmet need is a key area which requires further research. Unfortunately, it is difficult to measure and we only ever observe the met need for care, i.e. those who end up receiving formal care services. Thus, prior to addressing unmet need, it is important that we can measure it. One step necessary in doing so would be to carry out detailed investigations with individual local authorities. This would help us understand more about the needs of those individuals who apply for FPC but who are turned down. But this is only part of the picture. Understanding where individuals need FPC but don’t apply, either due to transaction costs, a lack of information on how to access those services, or other reasons, is far more difficult. One approach to gaining insight on these individuals could be to conduct qualitative interviews with them and their families.