Some reflections on the new NICE Principles

koonalkiritshah Leave a comment

A few weeks ago, NICE published its ‘updated principles‘ – a guide to what the Institute takes into account when developing its guidance, and to the “morals, ethics and values” that underpin its recommendations. This follows a public consultation in which comments were sought on a draft version of the guide in early 2019. The principles are intended to replace NICE’s guide to its social value judgments (available to download here), which was first published in 2005 and updated in 2008. This reflects the fact that NICE’s remit has widened and its guidance now draws on a wider range of evidence than before.

The contents seem sensible. The principles set out NICE’s careful approach to considering evidence, producing guidance, and making, publishing, and updating recommendations. But, as is often the case, what is interesting are the things that are missing.

Principle 4 describes how NICE takes into account the advice and experience of service users, carers, health and social care professionals, commissioners, providers, and the general public. These groups may be invited to submit evidence and to comment on draft recommendations. This seems reasonably clear for input into specific guidance (e.g. the appraisal of a specific technology) but it is less clear whether and how these groups will have the opportunity to provide input into more general guidance and value judgments adopted by NICE and its advisory committees.

The superseded social value judgments guide states that: “The NHS is funded from general taxation, and it is right that UK citizens have the opportunity to be involved in the decisions about how the NHS’s limited resources should be allocated” (p.10). The new principles guide lacks such a statement. A link to the reports of the Citizens Council (a lay panel that provides NICE with a public perspective on issues relating to social values) appears at the end of the guide, but the current status of the Council is unclear – its members last met in 2015. Researchers interested in the application of social value judgments in health care decision making will likely be curious to know whether NICE plans to revive these kinds of exercises, particularly given the increasing interest in deliberative democracy in other fields.

Related to this, it would be helpful to see a commitment from NICE to consider research that generates and analyses evidence on the views of different groups of society on priority setting issues. The evidence base on societal preferences and the social value of a QALY continues to grow – see here for a just-published study conducted in Australia – but HTA agencies’ appetite for this kind of work remains unclear.

Principle 7 – which commits NICE to basing its recommendations “on an assessment of population benefits and value for money” – is particularly relevant for health economists and readers of this blog. This principle notes that when certain criteria are satisfied, treatments for very rare conditions (‘highly specialised treatments’) may be recommended with cost-effectiveness estimates exceeding the normally acceptable threshold. Curiously, there is no mention of life-extending end of life treatments (for which a higher threshold also applies), though the technology appraisal method guide does describe this along with other factors influencing judgments about the acceptability of new treatments. It is also mentioned explicitly in NICE’s charter.

This section also recognises “that decisions about a person’s care are often sensitive to their preferences”. This is a nod to the Institute’s increasing interest in patient preference research. A recent article authored by Jacoline Bouvy and colleagues neatly describes how NICE sees a role for evidence on patient preferences to be submitted alongside other types of evidence, but not to be directly incorporated into economic models.

Principle 9 states that NICE’s guidance, where possible, “aims to reduce and not increase identified health inequalities” and “should support strategies that improve population health as a whole, while offering particular benefit to the most disadvantaged”. As pondered by Simon McNamara, could this mean a move towards economic evaluations that measure and value inequality? If the distribution of health is to be taken into account, will this be done algorithmically – for example, via QALY weights or distributional cost-effectiveness analysis – or more implicitly via appraisal committee deliberations? The Citizens Council discussed inequalities in health back in 2006 (PDF). But there have since been several methodological and empirical developments, so now might be a good time to once again seek the views of the panel on how NICE should achieve this aim.

NICE’s approach to explicitly identifying and publicly discussing social value judgements has been pioneering. It will be fascinating to see how its principles develop over time, and in particular how they will inform the review and update of its technology evaluation methods.

Image credit

Chris Sampson’s journal round-up for 24th February 2020

Chris Sampson Leave a comment

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Should health economic evaluations undertaken from a societal perspective include net government spending multiplier effects? Applied Health Economics and Health Policy [PubMed] Published 4th February 2020

Any mention of “macroeconomics” usually causes my eyes to glaze over and my mind to return to 2007 and to ECN202. With some luck, I might scrape through the conversation as I did my second year of undergrad. Perhaps other health economists share my affliction, and that’s why fiscal multipliers haven’t been paid much attention. This paper seeks to redress the balance by considering the possible importance of accounting for the general effects of health expenditure on the economy.

The title of the paper raises the question of ‘should‘, but it’s really more about the ‘could‘. The authors start out by reminding us what a fiscal multiplier is and how it is commonly used and estimated. In short, it’s the amount of expansion we might expect to see in an economy relative to government expenditure. So, with a fiscal multiplier of 1.5, the government spending £1 would expand the economy by £1.50. This study is conducted in the Australian context, so the authors do a bit of groundwork to identify a multiplier for Australia of 1.1. Then, the authors proceed to use this multiplier for domestic expenditure in health care, to demonstrate the potential importance of its use in estimating the societal benefits of health care expenditure.

Two previously conducted economic evaluations are used as test cases. One case study is for a pharmaceutical intervention and the other is for physiotherapy. The key difference between the two – as far as this analysis is concerned – is that the pharmaceutical is produced outside of Australia, with 77% of the expenditure falling outside of the country. Physiotherapy is largely based in expenditure on domestic labour, with only 3% of expenditure falling outside of Australia. What this means is that the effective cost-effectiveness of the pharmaceutical is reduced, while the cost-effectiveness of the physiotherapy is increased. The difference in the case of the pharmaceutical is quite large, with the incremental cost-effectiveness ratio shifting from $31,244 to $47,311 per QALY. Clearly, this could affect decision-making, with implications for cost-effectiveness thresholds.

Personally, I’m a societal perspective sceptic. So, from my position, the main value of considering fiscal multipliers is not from using them to estimate the cost-effectiveness of individual interventions, but rather to help to determine industrial policy. As the authors explain, this approach can be used to identify the value of having health care industries base their operations within a country. Perhaps we could get to the point where a pharmaceutical manufacturer enjoys a higher threshold in a given country if they also pay more taxes. From a (national) societal perspective, there’s a satisfying logic in that.

Whom should we ask? A systematic literature review of the arguments regarding the most accurate source of information for valuation of health states. Quality of Life Research [PubMed] Published 3rd February 2020

For many years, there has been debate about ‘patient’ vs ‘public’ values when it comes to preferences for health states. Gradually, researchers are realising that this is not a useful distinction, as all patients are members of the public and the public are all past and future patients. Nevertheless, there is a useful distinction between valuing a health state based on one’s own current experience and valuing a hypothetical health state. Which should be used to generate quality-adjusted life years (QALYs) to inform resource allocation decisions? In this paper, the authors review the arguments that have been made in the literature.

A literature search was conducted and studies were included in the review if they contained arguments about the source of health state valuations, with 82 articles included. The authors conducted a descriptive qualitative content analysis, grouping arguments into themes at three levels of granularity. The arguments favouring ‘patient’ values related to issues such as the superiority of patient knowledge, adaptation, and patient interests. Arguments in favour of ‘public’ values related to issues such as valuation difficulties for patients, socially directed values, and practical advantages.

This study provides a valuable review of a complex literature, but I believe it is flawed in its conception. There is no such thing as “the most accurate source of information” in this context. For this to be the case, there would need to be some true value that we were seeking to identify. But the value that is most close to the truth depends on our conception of value. Therefore, the answer to the question “which is most accurate?” and “whom should we ask?” are one in the same. This is easily demonstrated by the unanswerable question “how do we define accuracy?” The authors’ attempt to separate the two inevitably fails. Arguments that they frame as ‘irrelevant’ are only irrelevant if we accept the premise that value and decision-making imperatives are separable. The authors’ concluding argument in favour of patient values is both pre-determined by their conception of ‘accuracy’ and groundless.

Nevertheless, I like the paper a lot, as it digests a large amount of information and provides a taxonomy of quotations that will be extremely valuable to researchers developing ideas in this context. My only complaint is that they did not include blog posts in their review!

Health-related quality of life in neonates and infants: a conceptual framework. Quality of Life Research [PubMed] Published 29th January 2020

I’m working on an evaluative study in the context of newborns, and it’s the first time I’ve worked on an intervention where consideration of health-related quality of life (in the short term) – let alone QALYs – is barely even on the table. Yet many interventions and support services for newborns might reasonably be expected to have huge impacts on their quality of life. This study attempts to bring us closer to a world where we can identify QALYs for newborns and infants.

A qualitative study was conducted in a Toronto hospital with two focus groups and five interviews with caregivers of children with intestinal failure and with 14 health care professionals. The focus groups and interviews used open-ended questioning, allowing participants to state what they saw as the most important factors. Fourteen different factors arose, including things like sleep, feeding needs, safety, hygiene, and physical abilities. The authors arranged these into four levels, relating to i) basic needs, ii) non-basic needs, iii) caregivers and family, and iv) society and community.

Much the same dimensions were identified for neonates (0-28 days) and infants (up to one year), but the weighting attributed to the dimensions differed. The importance of non-basic needs increased with age. One implication of this is that time in hospital, where basic needs can be met but other aspects of HRQoL may be limited, may be very good for a neonate but not so good for an older infant. As a result of this, the authors suggest that a weighting algorithm according to age or developmental status may be appropriate. An interesting characterisation of HRQoL that comes out of the study is as a surrogate for the effort required by caregivers to obtain some degree of normalcy in the child’s life. The HRQoL of the individual and of the caregiver(s) is clearly inseparable. But the authors aren’t able to offer much guidance on how this affects measurement.

The authors set out from the position that measuring health-related quality of life in this cohort is important. I don’t know the ethics literature well enough to disagree, so I have to take their word for it. But it does seem that it might be OK for clinical decision-making in this context to be guided by a very different set of criteria to those used in older populations. Whatever the right solution, it will be guided by this important study.

Credits

Ian Cromwell’s journal round-up for 17th February 2020

Ian Cromwell Leave a comment

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Does the use of health technology assessment have an impact on the utilisation of health care resources? Evidence from two European countries. European Journal of Health Economics [PubMed] Published 5th February 2020

The ostensible purpose of health technology assessment (HTA) is to provide health care decision-makers with the information they need when considering whether to change existing policies. One of the questions I’ve heard muttered sotto voce (and that I will admit to having asked myself in more cynical moments) is whether or not HTAs actually make a difference. We are generating lots of evidence, but does it have any real impact on decision making? Do the complex analyses health economists undertake make any impact on policy?

This paper used data from Catalonia and England to estimate the impact of a positive HTA recommendation from the regulatory bodies – the National Institute for Health and Care Excellence (NICE) in England and a collection of regional approval bodies in Catalonia and Spain – to assess trends in medical usage prior to and following the publication of HTA-guided recommendations for new cancer drugs between 2011 and the end of 2016. Utilization (volume of drugs dispensed) and expenditure were extracted from retrospective records. The authors built a Poisson regression model that allowed them to observe temporal effects of usage before and following a positive recommendation.

The authors noted that a lack of pre-recommendation utilization data made it difficult to compute a model of negative recommendations (which is the more cynical version of the question!), so it is important to recognize that as a limitation of the approach. They also note, however, that it is typically the case in the UK and Catalonia that approvals for new drugs are conditional on a positive recommendation. Spain has a different system in which medicines may still be available even if they are not recommended.

The results of the model are a bit more complex than is easy to fit into a blog post, but the bottom line is that a positive recommendation does produce an increase in utilization. What stuck out to me about the descriptive findings was the consistent presence of a trend toward increased usage happening before the recommendation was published. But the Poisson model found a significant effect of the recommendation even controlling for that temporal trend. The authors helpfully noted that the criteria going into a recommendation are different between England and Spain (cost per QALY in England, clinical effectiveness alone sometimes in Spain), which makes inter-country comparisons challenging.

Health‐related quality of life in oncology drug reimbursement submissions in Canada: a review of submissions to the pan‐Canadian Oncology Drug Review. Cancer [PubMed] Published 1st January 2020

In Canada, newly-developed cancer drugs undergo HTA through the pan-Canadian Oncology Drug Review (pCODR), a program run under the auspices of the Canadian Agency for Drugs and Technologies in Health (CADTH). Unlike NICE in the UK, the results of CADTH’s pCODR recommendations are not binding; they are intended instead to provide provincial decision-makers with expert evidence they can use when deciding whether or not to add drugs to their formulary.

This paper, written by researchers at the Canadian Centre for Applied Research in Cancer Control (ARCC), reviewed the publicly-available reports governing 43 pCODR recommendations between 2015 and 2018. The paper summarizes the findings of the cost-effectiveness analyses generated in each report, including incremental costs and incremental QALYs (incremental cost per QALY being the reference case used by CADTH). The authors also appraised the methods chosen within each submission, both in terms of decision model structure and data inputs.

Interestingly, and perhaps disconcertingly, the paper reports a notable discrepancy between the ICERs reported by the submitting manufacturer and those calculated by CADTH’s Economics Guidance Panel. This appeared to be largely driven by the kind of health-related quality of life (HRQoL) data used to generate the QALYs in each submission. The authors note that the majority (56%) of the submissions provided to pCODR didn’t collect HRQoL data alongside clinical trials, preferring instead to use values published in the literature. In the face of high levels of uncertainty and relatively small incremental benefits (the median change in QALYs was 0.86), it seems crucial to have reliable information about HRQoL for making these kinds of decisions.

Regulatory and advisory agencies like CADTH have a rather weighty responsibility, not only to help decision makers identify which new drugs and technologies the health care system should adopt, but also which ones they should reject. When manufacturers’ submissions rely on inappropriate data with high levels of uncertainty, this task becomes much more difficult. The authors suggest that manufacturers should be collecting their own HRQoL data in clinical trials they fund. After all, if we want HTAs to have an effect on policy-making, we should also make sure they’re having a positive effect.

The cost-effectiveness of limiting federal housing vouchers to use in low-poverty neighborhoods in the United States. Public Health [PubMed] Published January 2020

My undergraduate education was heavily steeped in discussions of the social determinants of health. Another cynical opinion I’ve heard (again sometimes from myself) is that health economics is disproportionately concerned with the adoption of new drugs that have a marginal effect on health, often at the expense of investment in the other non-health-care determinants. This is a particularly persuasive bit of cynicism when you consider cancer drugs like in our previous two examples, where the incremental benefits are typically modest and the costs typically high. That’s why I was especially excited to see this paper published by my friend Dr. Zafar Zafari, applying health economic analysis frameworks to something atypical: housing policy.

The authors evaluated a trial running alongside a program providing housing vouchers to 4600 low-income households. The experimental condition in this case was that the vouchers could only be used in well-off neighbourhoods (i.e., those with a low level of poverty). The authors considered the evidence showing a link between neighbourhood wealth and lowering rates of obesity-related health conditions like diabetes, and used that evidence to construct a Markov decision model to measure incremental cost per QALY over the length of the study (10-15 years). Cohort characteristics, relative clinical effectiveness, and costs of the voucher program were estimated from trial results, with other costs and probabilities derived from the literature.

Compared to the control group (public housing), use of the housing vouchers provided an additional 0.23 QALYs per person, at a lower cost (about $750 less per person). Importantly, these findings were highly robust to parameter uncertainty, with 99% of ICERs falling below a willingness-to-pay threshold of $20,000/QALY (>90% below a WTP threshold of $0/QALY). The model was highly sensitive to the discount rate, which makes sense considering that we would expect, for a chronic condition like diabetes and a distal relationship like housing, that all the incremental health gains would be occurring years after the initial intervention.

There are a lot of things to like about this paper, but the one that stands out to me is the way they’ve framed the question:

We seek to inform the policy debate over the wisdom of spending health dollars on non-health sectors of the economy by defining the trade-off, or ‘opportunity cost’ of such a decision.

The idea that “health funds” should be focussed on “health care” robs us of the opportunity to consider the health impact of interventions in other policy areas. By bringing something like housing explicitly into the realm of cost-per-QALY analysis, the authors invite us all to consider the kinds of trade-offs we make when we relegate our consideration of health only to the kinds of things that happen inside hospitals.

A multidimensional array representation of state-transition model dynamics. Medical Decision Making [PubMed] Published 28th January 2020

I’ve been building models in R for a few years now, and developed a method of my own more or less out of necessity. So I’ve always been impressed with and drawn to the work of the group Decision Analysis in R for Technologies in Health (the amazingly-named DARTH). I’ve had the opportunity to meet a couple of their scientists and have followed their work for a while, and so I was really pleased to see the publication of this paper, hot on the heels of another paper discussing a formalized approach to model construction in R, and timed to coincide with the publication of a step-by-step guidebook on how to build models according to the DARTH recipe.

The DARTH approach (and, as a happy coincidence, mine too) involves tapping into R’s powerful ability to organize data into multidimensional arrays. The paper talks in depth about how R arrays can be used to represent health states, and how to set up and program models of essentially any level of complexity using a set of basic R commands. As a bonus they include publicly-accessible sample code that you can follow along as you read (which is the best way to learn something like this).

The authors argue that the method they propose is ideal for capturing and reflecting “transition rewards” – that is, effects on the cohort that occur during transitions between health states – in addition to “state rewards” (effects that happen as a consequence of being within a state). The key to this Dynamics Array approach is the use of a three-dimensional array to store the transitions, with the third array representing the passage of time. After walking the reader through the theory, the authors present a sample three-state model and show that the new method is fast, efficient, and accurate.

I hope that I have been sufficiently clear that I am a big fan of DARTH and admire their work a great deal. Because there is one big criticism I have to level at them, which is that this paper (and the others I have cited) is not terribly easy to follow. It sort of presumes that you already understand a lot of the topics that are discussed, which I personally do not. And if I, someone who has built many array-based models in R, am having a tough time understanding the explanation of their approach then woe betide anyone else who is reading this paper without a firm grasp of R, decision modelling theory, matrix algebra, and a handful of the other topics required to benefit from this (truly excellent) work.

DARTH is laying down a well-thought-out path to revolutionizing the standard approach to model building, but they can only do that if people start adopting their approach. If I were a grad student hoping to build my first model, this paper would likely intimidate me enough to maybe go back to the default of building it in Excel. As a postdoc with my own way of doing things there is a big opportunity cost of switching, and part of that cost is feeling too dumb to follow the instructions. I know that DARTH has tutorials and courses and workshops to help people get up to speed, but I hope that they also have a plan to translate some of this knowledge into a form that is more accessible for casual coders, non-economists, and other people who need this info but who (like me) might find this format opaque.

Credits