Chris Sampson’s journal round-up for 27th January 2020

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

A general framework for classifying costing methods for economic evaluation of health care. The European Journal of Health Economics [PubMed] Published 20th January 2020

When it comes to health state valuation and quality of life, I’m always very concerned about the use of precise terminology, and it bugs me when people get things wrong. But when it comes to costing methods, I’m pretty shoddy. So I’m pleased to see this very useful paper, which should help us all to gain some clarity in our reporting of costing studies.

The authors start out by clearly distinguishing between micro-costing and gross-costing in the identification of costs and between top-down and bottom-up valuation of these costs. I’m ashamed to say that I had never properly grasped the four distinct approaches that can be adopted based on these classifications, but the authors make it quite clear. Micro-costing means detailed identification of cost components, while gross-costing considers resource use in aggregate. Top-down methods use expenditure data collected at the organisational level, while bottom-up approaches use patient-level data.

A key problem is that our language – as health economists – is in several respects contradictory to the language used by management accountants. It’s the accountants who are usually preparing the cost information that we might use in analyses, and these data are not normally prepared for the types of analysis that we wish to conduct, so there is a lot that can go awry. Perhaps most important is that financial accounting is not concerned with opportunity costs. The authors provide a kind of glossary of terms that can support translation between the two contexts, as well as a set of examples of the ways in which the two contexts differ. They also point out the importance of different accounting practices in different countries and the ways in which these might necessitate adjustment in costing methods for economic evaluation.

The study includes a narrative review of costing studies in order to demonstrate the sorts of errors in terminology that can arise and the lack of clarity that results. The studies included in the review provide examples of the different approaches to costing, though no study is identified as ‘bottom-up gross-costing’. One of the most useful contributions of the paper is to provide two methodological checklists, one for top-down and one for bottom-up costing studies. If you’re performing, reviewing, or in any way making use of costing studies, this will be a handy reference.

Health state values of deaf British Sign Language (BSL) users in the UK: an application of the BSL version of the EQ-5D-5L. Applied Health Economics and Health Policy [PubMed] Published 16th January 2020

The BSL translation of the EQ-5D is like no other. It is to be used – almost exclusively – by people who have a specific functional health impairment. For me, this raises questions about whether or not we can actually consider it simply a translation of the EQ-5D and compare values with other translations in the way we would any other language. This study uses data collected during the initial development and validation of the EQ-5D-5L BSL translation. The authors compared health state utility values from Deaf people (BSL users) with a general population sample from the Health Survey for England.

As we might expect, the Deaf sample reported a lower mean utility score (0.78) than the general population (0.84). Several other health measures were used in the BSL study. A staggering 43% of the Deaf participants had depression and a lot of the analysis in the paper is directed towards comparing the groups with and without psychological distress. The authors conduct some simple regression analyses to explore what might be the determinants of health state utility values in the Deaf population, with long-standing physical illness having the biggest impact.

I had hoped that the study might be able to tell us a bit more about the usefulness of the BSL version of the EQ-5D-5L, because the EQ-5D has previously been shown to be insensitive to hearing problems. The small sample (<100) can’t tell us a great deal on its own, so it’s a shame that there isn’t some attempt at matching with individuals from the Health Survey for England for the sake of comparison. Using the crosswalk from the EQ-5D-3L to obtain 5L values is also a problem, as it limits the responsiveness of index values. Nevertheless, it’s good to see data relating to this under-represented population.

A welfare-theoretic model consistent with the practice of cost-effectiveness analysis and its implications. Journal of Health Economics [PubMed] Published 11th January 2020

There are plenty of good reasons to deviate from a traditional welfarist approach to cost-benefit analysis in the context of health care, as health economists have debated for decades. But it is nevertheless important to understand the ways in which cost-effectiveness analysis, as we conduct it, deviates from welfarism, and to aim for some kind of consistency in our handling of different issues. This paper attempts to draw together disparate subjects of discussion on the theoretical basis for aspects of cost-effectiveness analysis. The author focuses on issues relating to the inclusion of future (unrelated) costs, to discounting, and to consistency with welfarism, in the conduct of cost-per-QALY analyses. The implications are given consideration with respect to adopting a societal perspective, recognising multiple budget holders, and accounting for distributional impacts.

All of this is based on the description of an intertemporal utility model and a model of medical care investment. The model hinges especially on how we understand consumption to be affected by our ambition to maximise QALYs. For instance, the author argues that, once we consider time preferences in an overall utility function, we don’t need to worry about differential discounting in health and consumption. The various implications of the model are compared to the recommendations of the Second Panel on Cost-Effectiveness in Health and Medicine. In general, the model supports the recommendations of the Panel, where others have been critical. As such, it sets out some of the theoretical basis for those recommendations. It also implies other recommendations, not considered by the Panel. For example, the optimal cost-effectiveness threshold is likely to be higher than GDP per capita.

It’s difficult to judge the validity of the framework from a first read. The paper is dense with theoretical exposition. My first instinct is ‘so what’. One of the great things about the practice of cost-effectiveness analysis in health care is that it isn’t constrained by restrictive theoretical frameworks, and so the very idea of a kind of unified theoretical framework is a bit worrying to me. But my second thought is that this is a valuable paper, as it attempts to gather up several loose threads. Whether or not these can be gathered up within a welfarist framework is debatable, but the exercise is revealing. I suspect this paper will help to trigger further inquiry, which can only be a good thing.

Registered reports: time to radically rethink peer review in health economics. PharmacoEconomics – Open [PubMed] Published 23rd January 2020

As a discipline, health economics isn’t great when it comes to publication practices. We excel in neither the open access culture of medical sciences nor the discussion paper culture of economics proper. In this article, the authors express concern about publication bias, and the fact that health economics journals – and health economists in general – aren’t doing much to combat it. In fairness to the discipline, there isn’t really any evidence that publication bias abounds. But that isn’t really the point. We should be able to prove and ensure that it doesn’t if we want our research to been seen as credible.

One (partial) solution to publication bias is the adoption – by journals – of registered reports. Under such a system, researchers would submit study protocols to journals for peer review. If the journal were satisfied with the methods then they could guarantee to publish the study once the results are in, regardless of how sexy the results may or may not be. The authors of this paper identify the prevalence of studies in major health economics journals that could benefit from registered reports. These would be prospectively designed experimental or quasi-experimental studies. It seems that there are plenty.

I’ve used this blog in the past to propose more transparent research practices and to complain about publication practices in health economics generally, while others have complained about the use of p-values in our discipline. The adoption of registered reports is one tactic that could bring improvements and I hope it will be given proper consideration by those in a position to enact change.

Credits

Chris Sampson’s journal round-up for 5th August 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The barriers and facilitators to model replication within health economics. Value in Health Published 16th July 2019

Replication is a valuable part of the scientific process, especially if there are uncertainties about the validity of research methods. When it comes to cost-effectiveness modelling, there are endless opportunities for researchers to do things badly, even with the best intentions. Attempting to replicate modelling studies can therefore support health care decision-making. But replication studies are rarely conducted, or, at least, rarely reported. The authors of this study sought to understand the factors that can make replication easy or difficult, with a view to informing reporting standards.

The authors attempted to replicate five published cost-effectiveness modelling studies, with the aim of recreating the key results. Each replication attempt was conducted by a different author and we’re even given a rating of the replicator’s experience level. The characteristics of the models were recorded and each replicator detailed – anecdotally – the things that helped or hindered their attempt. Some replications were a resounding failure. In one case, the replicated cost per patient was more than double the original, at more than £1,000 wide of the mark. Replicators reported that having a clear diagram of the model structure was a big help, as was the provision of example calculations and explicit listing of the key assumptions. Various shortcomings made replication difficult, all relating to a lack of clarity or completeness in reporting. The impact of this on the validation attempt was exacerbated if the model either involved lots of scenarios that weren’t clearly described or if the model had a long time horizon.

The quality of each study was assessed using the Philips checklist, and all did pretty well, suggesting that the checklist is not sufficient for ensuring replicability. If you develop and report cost-effectiveness models, this paper could help you better understand how end-users will interpret your reporting and make your work more replicable. This study focusses on Markov models. They’re definitely the most common approach, so perhaps that’s OK. It might be useful to produce prescriptive guidance specific to Markov models, informed by the findings of this study.

US integrated delivery networks perspective on economic burden of patients with treatment-resistant depression: a retrospective matched-cohort study. PharmacoEconomics – Open [PubMed] Published 28th June 2019

Treatment-resistant depression can be associated high health care costs, as multiple lines of treatment are tried, with patients experiencing little or no benefit. New treatments and models of care can go some way to addressing these challenges. In the US, there’s some reason to believe that integrated delivery networks (IDNs) could be associated with lower care costs, because IDNs are based on collaborative care models and constitute a single point of accountability for patient costs. They might be particularly useful in the case of treatment-resistant depression, but evidence is lacking. The authors of this study investigated the difference in health care resource use and costs for patients with and without treatment-resistant depression, in the context of IDNs.

The researchers conducted a retrospective cohort study using claims data for people receiving care from IDNs, with up to two years follow-up from first antidepressant use. 1,582 people with treatment-resistant depression were propensity score matched to two other groups – patients without depression and patients with depression that was not classified as treatment-resistant. Various regression models were used to compare the key outcomes of all-cause and specific categories of resource use and costs. Unfortunately, there is no assessment of whether the selected models are actually any good at estimating differences in costs.

The average costs and resource use levels in the three groups ranked as you would expect: $25,807 per person per year for the treatment-resistant group versus $13,701 in the non-resistant group and $8,500 in the non-depression group. People with treatment-resistant depression used a wider range of antidepressants and for a longer duration. They also had twice as many inpatient visits as people with depression that wasn’t treatment-resistant, which seems to have been the main driver of the adjusted differences in costs.

We don’t know (from this study) whether or not IDNs provide a higher quality of care. And the study isn’t able to compare IDN and non-IDN models of care. But it does show that IDNs probably aren’t a full solution to the high costs of treatment-resistant depression.

Rabin’s paradox for health outcomes. Health Economics [PubMed] [RePEc] Published 19th June 2019

Rabin’s paradox arises from the theoretical demonstration that a risk-averse individual who turns down a 50:50 gamble of gaining £110 or losing £100 would, if expected utility theory is correct, turn down a 50:50 gamble of losing £1,000 or gaining millions. This is because of the assumed concave utility function over wealth that is used to model risk aversion and it is probably not realistic. But we don’t know about the relevance of this paradox in the health domain… until now.

A key contribution of this paper is that it considers both decision-making about one’s own health and decision-making from a societal perspective. Three different scenarios are set-up in each case, relating to gains and losses in life expectancy with different levels of health functioning. 201 students were recruited as part of a larger study on preferences and each completed all six gamble-pairs (three individual, three societal). To test for Rabin’s paradox, the participants were asked whether they would accept each gamble involving a moderate stake and a large stake.

In short, the authors observe Rabin’s proposed failure of expected utility theory. Many participants rejected small gambles but did not reject the larger gambles. The effect was more pronounced for societal preferences. Though there was a large minority for whom expected utility theory was not violated. The upshot of all this is that our models of health preferences that are based on expected utility may be flawed where uncertain outcomes are involved – as they often are in health. This study adds to a growing body of literature supporting the relevance of alternative utility theories, such as prospect theory, to health and health care.

My only problem here is that life expectancy is not health. Life expectancy is everything. It incorporates the monetary domain, which this study did not want to consider, as well as every other domain of life. When you die, your stock of cash is as useful to you as your stock of health. I think it would have been more useful if the study focussed only on health status and outcomes and excluded all considerations of death.

Credits

Chris Sampson’s journal round-up for 23rd July 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Quantifying life: understanding the history of quality-adjusted life-years (QALYs). Social Science & Medicine [PubMed] Published 3rd July 2018

We’ve had some fun talking about the history of the QALY here on this blog. The story of how the QALY came to be important in health policy has been obscured. This paper seeks to address that. The research adopts a method called ‘multiple streams analysis’ (MSA) in order to explain how QALYs caught on. The MSA framework identifies three streams – policy, politics, and problems – and considers the ‘policy entrepreneurs’ involved. For this study, archival material was collected from the National Archives, Department of Health files, and the University of York. The researchers also conducted 44 semi-structured interviews with academics and civil servants.

The problem stream highlights shocks to the UK economy in the late 1960s, coupled with growth in health care costs due to innovations and changing expectations. Cost-effectiveness began to be studied and, increasingly, policymaking was meant to be research-based and accountable. By the 80s, the likes of Williams and Maynard were drawing attention to apparent inequities and inefficiencies in the health service. The policy stream gets going in the 40s and 50s when health researchers started measuring quality of life. By the early 60s, the idea of standardising these measures to try and rank health states was on the table. Through the late 60s and early 70s, government economists proliferated and proved themselves useful in health policy. The meeting of Rachel Rosser and Alan Williams in the mid-70s led to the creation of QALYs as we know them, combining quantity and quality of life on a 0-1 scale. Having acknowledged inefficiencies and inequities in the health service, UK politicians and medics were open to new ideas, but remained unconvinced by the QALY. Yet it was a willingness to consider the need for rationing that put the wheels in motion for NICE, and the politics stream – like the problem and policy stream – characterises favourable conditions for the use of the QALY.

The MSA framework also considers ‘policy entrepreneurs’ who broker the transition from idea to implementation. The authors focus on the role of Alan Williams and of the Economic Advisers’ Office. Williams was key in translating economic ideas into forms that policymakers could understand. Meanwhile, the Economic Advisers’ Office encouraged government economists to engage with academics at HESG and later the QoL Measurement Group (which led to the creation of EuroQol).

The main takeaway from the paper is that good ideas only prevail in the right conditions and with the right people. It’s important to maintain multi-disciplinary and multi-stakeholder networks. In the case of the QALY, the two-way movement of economists between government and academia was crucial.

I don’t completely understand or appreciate the MSA framework, but this paper is an enjoyable read. My only reservation is with the way the authors describe the QALY as being a dominant aspect of health policy in the UK. I don’t think that’s right. It’s dominant within a niche of a niche of a niche – that is, health technology assessment for new pharmaceuticals. An alternative view is that the QALY has in fact languished in a quiet corner of British policymaking, and been completely excluded in some other countries.

Accuracy of patient recall for self‐reported doctor visits: is shorter recall better? Health Economics [PubMed] Published 2nd July 2018

In designing observational studies, such as clinical trials, I have always recommended that self-reported resource use be collected no less frequently than every 3 months. This is partly based on something I once read somewhere that I can’t remember, but partly also on some logic that the accuracy of people’s recall decays over time. This paper has come to tell me how wrong I’ve been.

The authors start by highlighting that recall can be subject to omission, whereby respondents forget relevant information, or commission, whereby respondents include events that did not occur. A key manifestation of the latter is ‘telescoping’, whereby events are included from outside the recall period. We might expect commission to be more likely in short recalls and omission to be more common for long recalls. But there’s very little research on this regarding health service use.

This study uses data from a large trial in diabetes care in Australia, in which 5,305 participants were randomised to receive either 2-week, 3-month, or 12-month recall for how many times they had seen a doctor. Then, the trial data were matched with Medicare data to identify the true levels of resource use.

Over 92% of 12-month recall participants made an error, 76% of the 3-month recall, and 46% of the 2-week recall. The patterns of errors were different. There was very little under-reporting in the 2-week recall sample, with 3-month giving the most over-reporting and 12-month giving the most under-reporting. 12-month recall was associated with the largest number of days reported in error. However, when the authors account for the longer period being considered, and estimate a relative error, the impact of misreporting is smallest for the 12-month recall and greatest for the 2-week recall. This translates into a smaller overall bias for the longest recall period. The authors also find that older, less educated, unemployed, and low‐income patients exhibit higher measurement errors.

Health surveys and comparative studies that estimate resource use over a long period of time should use 12-month recall unless they can find a reason to do otherwise. The authors provide some examples from economic evaluations to demonstrate how selecting shorter recall periods could result in recommending the wrong decisions. It’s worth trying to understand the reasons why people can more accurately recall service use over 12 months. That way, data collection methods could be designed to optimise recall accuracy.

Who should receive treatment? An empirical enquiry into the relationship between societal views and preferences concerning healthcare priority setting. PLoS One [PubMed] Published 27th June 2018

Part of the reason the QALY faces opposition is that it has been used in a way that might not reflect societal preferences for resource allocation. In particular, the idea that ‘a QALY is a QALY is a QALY’ may conflict with notions of desert, severity, or process. We’re starting to see more evidence for groups of people holding different views, which makes it difficult to come up with decision rules to maximise welfare. This study considers some of the perspectives that people adopt, which have been identified in previous research – ‘equal right to healthcare’, ‘limits to healthcare’, and ‘effective and efficient healthcare’ – and looks at how they are distributed in the Netherlands. Using four willingness to trade-off (WTT) exercises, the authors explore the relationship between these views and people’s preferences about resource allocation. Trade-offs are between quality vs quantity of life, health maximisation vs equality, children vs the elderly, and lifestyle-related risk vs adversity. The authors sought to test several hypotheses: i) that ‘equal right’ respondents have a lower WTT; ii) ‘limits to healthcare’ people express a preference for health gains, health maximisation, and treating people with adversity; and iii) ‘effective and efficient’ people support health maximisation, treating children, and treating people with adversity.

A representative online sample of adults in the Netherlands (n=261) was recruited. The first part of the questionnaire collected socio-demographic information. The second part asked questions necessary to allocate people to one of the three perspectives using Likert scales based on a previous study. The third part of the questionnaire consisted of the four reimbursement scenarios. Participants were asked to identify the point (in terms of the relevant quantities) at which they would be indifferent between two options.

The distribution of the viewpoints was 65% ‘equal right’, 23% ‘limits to healthcare’, and 7% ‘effective and efficient’. 6% couldn’t be matched to one of the three viewpoints. In each scenario, people had the option to opt out of trading. 24% of respondents were non-traders for all scenarios and, of these, 78% were of the ‘equal right’ viewpoint. Unfortunately, a lot of people opted out of at least one of the trades, and for a wide variety of reasons. Decisionmakers can’t opt out, so I’m not sure how useful this is.

The authors describe many associations between individual characteristics, viewpoints, and WTT results. But the tested hypotheses were broadly supported. While the findings showed that different groups were more or less willing to trade, the points of indifference for traders within the groups did not vary. So while you can’t please everyone in health care priority setting, this study shows how policies might be designed to satisfy the preferences of people with different perspectives.

Credits