Chris Sampson’s journal round-up for 2nd December 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

The treatment decision under uncertainty: the effects of health, wealth and the probability of death. Journal of Health Economics Published 16th November 2019

It’s important to understand how people make decisions about treatment. At the end of life, the question can become a matter of whether to have treatment or to let things take their course such that you end up dead. In order to consider this scenario, the author of this paper introduces the probability of death to some existing theoretical models of decision-making under uncertainty.

The diagnostic risk model and the therapeutic risk model can be used to identify risk thresholds that determine decisions about treatment. The diagnostic model relates to the probability that disease is present and the therapeutic model relates to the probability that treatment is successful. The new model described in this paper builds on these models to consider the impact on the decision thresholds of i) initial health state, ii) probability of death, and iii) wealth. The model includes wealth after death, in the form of a bequest. Limited versions of the model are also considered, excluding the bequest and excluding wealth (described as a ‘QALY model’). Both an individual perspective and an aggregate perspective are considered by excluding and including the monetary cost of diagnosis and treatment, to allow for a social insurance type setting.

The comparative statics show a lot of ambiguity, but there are a few things that the model can tell us. The author identifies treatment as having an ‘insurance effect’, by reducing diagnostic risk, a ‘protective effect’, by lowering the probability of death, and a risk-increasing effect associated with therapeutic risk. A higher probability of death increases the propensity for treatment in both the no-bequest model and the QALY model, because of the protective effect of treatment. In the bequest model, the impact is ambiguous, because treatment costs reduce the bequest. In the full model, wealthier individuals will choose to undergo treatment at a lower probability of success because of a higher marginal utility for survival, but the effect becomes ambiguous if the marginal utility of wealth depends on health (which it obviously does).

I am no theoretician, so it can take me a long time to figure these things out in my head. For now, I’m not convinced that it is meaningful to consider death in this way using a one-period life model. In my view, the very definition of death is a loss of time, which plays little or no part in this model. But I think my main bugbear is the idea that anybody’s decision about life saving treatment is partly determined by the amount of money they will leave behind. I find this hard to believe. The author links the finding that a higher probability of death increases treatment propensity to NICE’s end of life premium. Though I’m not convinced that the model has anything to do with NICE’s reasoning on this matter.

Moving toward evidence-based policy: the value of randomization for program and policy implementation. JAMA [PubMed] Published 15th November 2019

Evidence-based policy is a nice idea. We should figure out whether something works before rolling it out. But decision-makers (especially politicians) tend not to think in this way, because doing something is usually seen to be better than doing nothing. The authors of this paper argue that randomisation is the key to understanding whether a particular policy creates value.

Without evidence based on random allocation, it’s difficult to know whether a policy works. This, the authors argue, can undermine the success of effective interventions and allow harmful policies to persist. A variety of positive examples are provided from US healthcare, including trials of Medicare bundled payments. Apparently, such trials increased confidence in the programmes’ effects in a way that post hoc evaluations cannot, though no evidence of this increased confidence is actually provided. Policy evaluation is not always easy, so the authors describe four preconditions for the success of such studies: i) early engagement with policymakers, ii) willingness from policy leaders to support randomisation, iii) timing the evaluation in line with policymakers’ objectives, and iv) designing the evaluation in line with the realities of policy implementation.

These are sensible suggestions, but it is not clear why the authors focus on randomisation. The paper doesn’t do what it says on the tin, i.e. describe the value of randomisation. Rather, it explains the value of pre-specified policy evaluations. Randomisation may or may not deserve special treatment compared with other analytical tools, but this paper provides no explanation for why it should. The authors also suggest that people are becoming more comfortable with randomisation, as large companies employ experimental methods, particularly on the Internet with A/B testing. I think this perception is way off and that most people feel creeped out knowing that the likes of Facebook are experimenting on them without any informed consent. In the authors’ view, it being possible to randomise is a sufficient basis on which to randomise. But, considering the ethics, as well as possible methodological contraindications, it isn’t clear that randomisation should become the default.

A new tool for creating personal and social EQ-5D-5L value sets, including valuing ‘dead’. Social Science & Medicine Published 30th November 2019

Nobody can agree on the best methods for health state valuation. Or, at least, some people have disagreed loud enough to make it seem that way. Novel approaches to health state valuation are therefore welcome. Even more welcome is the development and testing of methods that you can try at home.

This paper describes the PAPRIKA method (Potentially All Pairwise RanKings of all possible Alternatives) of discrete choice experiment, implemented using 1000Minds software. Participants are presented with two health states that are defined in terms of just two dimensions, each lasting for 10 years, and asked to choose between them. Using the magical power of computers, an adaptive process identifies further choices, automatically ranking states using transitivity so that people don’t need to complete unnecessary tasks. In order to identify where ‘dead’ sits on the scale, a binary search procedure asks participants to compare EQ-5D states with being dead. What’s especially cool about this process is that everybody who completes it is able to view their own personal value set. These personal value sets can then be averaged to identify a social value set.

The authors used their tool to develop an EQ-5D-5L value set for New Zealand (which is where the researchers are based). They recruited 5,112 people in an online panel, such that the sample was representative of the general public. Participants answered 20 DCE questions each, on average, and almost half of them said that they found the questions difficult to answer. The NZ value set showed that anxiety/depression was associated with the greatest disutility, though each dimension has a notably similar level of impact at each level. The value set correlates well with numerous existing value sets.

The main limitation of this research seems to be that only levels 1, 3, and 5 of each EQ-5D-5L domain were included. Including levels 2 and 4 would more than double the number of questions that would need to be answered. It is also concerning that more than half of the sample was excluded due to low data quality. But the authors do a pretty good job of convincing us that this is for the best. Adaptive designs of this kind could be the future of health state valuation, especially if they can be implemented online, at low cost. I expect we’ll be seeing plenty more from PAPRIKA.

Credits

Hawking is right, Jeremy Hunt does egregiously cherry pick the evidence

I’m beginning to think Jeremy Hunt doesn’t actually care what the evidence says on the weekend effect. Last week, renowned physicist Stephen Hawking criticized Hunt for ‘cherry picking’ evidence with regard to the ‘weekend effect’: that patients admitted at the weekend are observed to be more likely than their counterparts admitted on a weekday to die. Hunt responded by doubling down on his claims:

Some people have questioned Hawking’s credentials to speak on the topic beyond being a user of the NHS. But it has taken a respected public figure to speak out to elicit a response from the Secretary of State for Health, and that should be welcomed. It remains the case though that a multitude of experts do continue to be ignored. Even the oft-quoted Freemantle paper is partially ignored where it notes of the ‘excess’ weekend deaths, “to assume that [these deaths] are avoidable would be rash and misleading.”

We produced a simple tool to demonstrate how weekend effect studies might estimate an increased risk of mortality associated with weekend admissions even in the case of no difference in care quality. However, the causal model underlying these arguments is not always obvious. So here it is:

weekend

A simple model of the effect of the weekend on patient health outcomes. The dashed line represents unobserved effects

 

So what do we know about the weekend effect?

  1. The weekend effect exists. A multitude of studies have observed that patients admitted at the weekend are more likely to die than those admitted on a weekday. This amounts to having shown that E(Y|W,S) \neq E(Y|W',S). As our causal model demonstrates, being admitted is correlated with health and, importantly, the day of the week. So, this is not the same as saying that risk of adverse clinical outcomes differs by day of the week if you take into account propensity for admission, we can’t say E(Y|W) \neq E(Y|W'). Nor does this evidence imply care quality differs at the weekend, E(Q|W) \neq E(Q|W'). In fact, the evidence only implies differences in care quality if the propensity to be admitted is independent of (unobserved) health status, i.e. Pr(S|U,X) = Pr(S|X) (or if health outcomes are uncorrelated with health status, which is definitely not the case!).
  2. Admissions are different at the weekend. Fewer patients are admitted at the weekend and those that are admitted are on average more severely unwell. Evidence suggests that the better patient severity is controlled for, the smaller the estimated weekend effect. Weekend effect estimates also diminish in models that account for the selection mechanism.
  3. There is some evidence that care quality may be worse at the weekend (at least in the United States). So E(Q|W) \neq E(Q|W'). Although this has not been established in the UK (we’re currently investigating it!)
  4. Staffing levels, particularly specialist to patient ratios, are different at the weekend, E(X|W) \neq E(X|W').
  5. There is little evidence to suggest how staffing levels and care quality are related. While the relationship seems evident prima facie, its extent is not well understood, for example, we might expect a diminishing return to increased staffing levels.
  6. There is a reasonable amount of evidence on the impact of care quality (preventable errors and adverse events) on patient health outcomes.

But what are we actually interested in from a policy perspective? Do we actually care that it is the weekend per se? I would say no, we care that there is potentially a lapse in care quality. So, it’s a two part question: (i) how does care quality (and hence avoidable patient harm) differ at the weekend E(Q|W) - E(Q|W') = ?; and (ii) what effect does this have on patient outcomes E(Y|Q)=?. The first question answers to what extent policy may affect change and the second gives us a way of valuing that change and yet the vast majority of studies in the area address neither. Despite there being a number of publicly funded research projects looking at these questions right now, it’s the studies that are not useful for policy that keep being quoted by those with the power to make change.

Hawking is right, Jeremy Hunt has egregiously cherry picked and misrepresented the evidence, as has been pointed out again and again and again and again and … One begins to wonder if there isn’t some motive other than ensuring long run efficiency and equity in the health service.

Credits

Conference round-up: Medicine, Markets and Morals Network Meeting Three

Last October saw the first meeting of the Medicine, Markets and Morals Network. Today and yesterday I attended their third and final (for now) meeting in London. The network brings together researchers from across the social sciences and humanities to discuss issues relating to resource allocation in health care. A book is planned and the website will be maintained, so hopefully the Network has a future and will produce something more permanent. For now, here are some of the overarching themes that I saw being discussed at the conference.

To deliberate or not to deliberate?

I don’t know what it is about philosophers that enables them to talk so well without slides. The conference started with a prime example in the shape of Jonathan Wolff discussing alternative approaches to valuing health care. Should we consider preferences, experienced utility or capabilities? And whose? Jo suggested that there is no single best approach and argued in favour of a discursive, deliberative approach.

This became a recurring subject for the event, to which we returned on the final day with Ruben Andreas Sakowsky arguing that deliberative evaluation is a better way of eliciting individual preferences. To this end, he suggested that we ‘upgrade’ the way we do discrete choice experiments. Leah Rand argued that we need to upgrade the notion of ‘accountability for reasonableness’, which is at the heart of decision-making processes in health care. Leah argued that there is a need to ensure legitimacy in the process of decision-making and a requirement for fair consideration of reasons.

But deliberative processes present challenges. Do people know enough to make informed decisions about the allocation of health care resources? If they don’t then there is still an argument to be made that in a democracy it is the views of these people that should be acted upon. But can deliberation still be meaningful if the people involved have no understanding of the context or implications of their decisions?

Information – particularly who has it and which bits of it matter – was the major theme of the conference for me. I think a lot of what we discussed related to the incompatibility of evidence-based policy and democracy. I’ll come back to that later, once I’ve discussed some of the other speakers’ talks.

Systems, structures and marketisation

Another major subject was the Health and Social Care Act 2012 (HSCA). Richard Taunt discussed the HSCA, noting the tendency for people to view the apparent fiasco as undemocratic. Richard argued that this is not the case, but rather issues like this have “no salience in electoral politics”. Even if people say that NHS policy will determine which way they vote, in reality votes don’t correspond to what people say is important. One option is to build sustainable development into democracy, such as has been tried with the Well-being of Future Generations (Wales) Act or the Finnish ‘Committee for the Future’. But, Richard argued, we already have the Health Select Committee and plenty of think tanks; ‘upgrading democracy’ should not be about structures. Rather, we should focus on encouraging the right kinds of behaviours, such as humility and continual learning.

Later in the day, Allyson Pollock presented a quite different story about the Health and Social Care Act; as providing the mechanisms for the deconstruction of the NHS. Allyson explained that the Act does away with the duty to provide universal health care according to need, and that the closure of hospitals is the evidence of this possibility being realised. The risk presented was that we might end up with a US-style system of health care funding and provision.

Therese Feiler presented a theological basis for the use of economics-type thinking in health care, which took us all out of our comfort zone; a great thing! Therese suggested that the use of diagnosis-related groups (DRGs) is a part of the process of the commodification of health care.

Mary Guy discussed the legal situation in the UK and The Netherlands regarding the application of competition law to health care, which is key to the potential for marketisation of the NHS. The take home message is that whether or not the NHS could be subject to competition law (in its current organisation) is still to be decided in the courts.

Rudolf Klein gave a notably lo-fi talk on the need to ration expectations rather than treatments or procedures. He argued that the NHS agenda is determined by the state of the economy. While the NHS Mandate (which we were all encouraged to read) lists many goals, the top item on the list of NHS ‘must-do’s relates to financial stability. The question is, what are we willing to sacrifice from the long list of NHS goals? (There was support in the room for allowing longer A&E waiting times.) And what about the cost of transparency and statistics in order to maintain ‘anticipative accountability’, which is held to be very important? Can we sacrifice some of these costly processes which in turn help create more goals for the NHS mandate?

Evidence, narratives and democracy

Chris Newdick argued that bioethics hasn’t done us much good, in that the focus on autonomy has bolstered neoliberalism. He discussed the notion of public health ethics and asked why community is not the starting point for the ethical debate rather than the individual. In this sense, has the neoliberal agenda been more successful in the debate about health care, as well as more broadly?

I think this brings us back to the issue of information. The idea that autonomy and/or the free market is paramount is an a priori notion. The invisible hand is a nice story, and people are compelled by it. Evidence, on the other hand (the visible hand), is not compelling. One can always find evidence to support a claim and as such the non-expert may be inclined to distrust all evidence. A compelling story is more difficult to disregard.

In his closing remarks, Cam Donaldson mentioned Robert Evans’s notion of zombie policies. These are not usually evidence-based zombies but about stories. Cam presented Alan Williams’s old argument about the distinctiveness of health care as akin to a duck-billed platypus. This provides a nice analogy, but it still depends on evidence. This is why we see the ‘health care is just another good’ argument coming back time and time again; recently in the broccoli debate in the US.

The problem is that the basis for publicly provided health care requires a lot of thought and a lot of foundational understanding of how things work in a particular economic, political, legal, historical and ethical context. For example, to know that health care is not broccoli you need to understand moral hazard (etc), and that this has been observed. There is evidence. However, in order to support neoliberalism and a free market in health care you don’t need any of that. All you need is basic intuition and a compelling heuristic: leave people, institutions and corporations to do as they please and the invisible hand will sort it all out.

This at least partly explains why academics tend to think differently about resource allocation in health care. Rachel Baker and Helen Mason presented their work on eliciting public views about the allocation of resources at the end of life. Using Q methodology (for which we were given a brief tutorial) they have identified three different viewpoints that people tend to adopt. The first group support the notion of value for money, with no special cases; QALY-maximisers would fit into this group. The second believe that life is precious, and they are not likely to accept any restriction of health care due to costs. The third is a more nuanced group that value wider benefits while also acknowledging the importance of opportunity cost.

On the first day we attendees were presented with 18 statements, with 7-point likert scales to elicit our agreement. The same questionnaire was used in a large online survey of a representative sample of the UK. This survey is designed to elicit the prevalence of the different viewpoints. Viewpoint 2 was by far the most common in the general population. Meanwhile, in the room, viewpoint 3 was the leader. As for me, the stats showed that I was the strongest supporter of viewpoint 3. This discrepancy between the public and a group of academics may not come as a surprise, but it is noteworthy. Viewpoint 3 is the most nuanced position. It acknowledges the messyness of health care resource allocation decisions. The findings of the survey suggest to me that the public do not recognise this messyness, or at least employ a simpler decision process that ignores some of the messyness. They may be right to do so, but I suspect not. As academics we know more about what we do not know; we have more known unknowns. The general public on the other hand don’t – and can’t be expected to – understand all of the challenges of resource allocation in health.

One of the very last points to be raised in the concluding discussion was whether homeopathy should be funded on the NHS. Most of us agreed it shouldn’t, but Ruben bravely stuck his neck out and suggested that if people want it then who are we to deny it? I think there’s an analogy to be drawn here between homeopathy and free market health care. We know homeopathy isn’t good for people. We know market forces in health care (broadly speaking) aren’t good for people. But both of these assertions are empirical; dependent on understanding a long history of evidence and fundamental notions of how we determine what is good for people. The public don’t have this understanding. The ‘invisible hand’ and ‘like cures like’ make for far more compelling and easily attainable interpretations of reality. So people will vote for them. But that is not a good thing.