Chris Sampson’s journal round-up for 17th September 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

Does competition from private surgical centres improve public hospitals’ performance? Evidence from the English National Health Service. Journal of Public Economics Published 11th September 2018

This study looks at proper (supply-side) privatisation in the NHS. The subject is the government-backed introduction of Independent Sector Treatment Centres (ISTCs), which, in the name of profit, provide routine elective surgical procedures to NHS patients. ISTCs were directed to areas with high waiting times and began rolling out from 2003.

The authors take pre-surgery length of stay as a proxy for efficiency and hypothesise that the entry of ISTCs would improve efficiency in nearby NHS hospitals. They also hypothesise that the ISTCs would cream-skim healthier patients, leaving NHS hospitals to foot the bill for a more challenging casemix. Difference-in-difference regressions are used to test these hypotheses, the treatment group being those NHS hospitals close to ISTCs and the control being those not likely to be affected. The authors use patient-level Hospital Episode Statistics from 2002-2008 for elective hip and knee replacements.

The key difficulty here is that the trend in length of stay changed dramatically at the time ISTCs began to be introduced, regardless of whether a hospital was affected by their introduction. This is because there was a whole suite of policy and structural changes being implemented around this period, many targeting hospital efficiency. So we’re looking at comparing new trends, not comparing changes in existing levels or trends.

The authors’ hypotheses prove right. Pre-surgery length of stay fell in exposed hospitals by around 16%. The ISTCs engaged in risk selection, meaning that NHS hospitals were left with sicker patients. What’s more, the savings for NHS hospitals (from shorter pre-surgery length of stay) were more than undermined by an increase in post-surgery length of stay, which may have been due to the change in casemix.

I’m not sure how useful difference-in-difference is in this case. We don’t know what the trend would have been without the intervention because the pre-intervention trend provides no clues about it and, while the outcome is shown to be unrelated to selection into the intervention, we don’t know whether selection into the ISTC intervention was correlated with exposure to other policy changes. The authors do their best to quell these concerns about parallel trends and correlated policy shocks, and the results appear robust.

Broadly speaking, the study satisfies my prior view of for-profit providers as leeches on the NHS. Still, I’m left a bit unsure of the findings. The problem is, I don’t see the causal mechanism. Hospitals had the financial incentive to be efficient and achieve a budget surplus without competition from ISTCs. It’s hard (for me, at least) to see how reduced length of stay has anything to do with competition unless hospitals used it as a basis for getting more patients through the door, which, given that ISTCs were introduced in areas with high waiting times, the hospitals could have done anyway.

While the paper describes a smart and thorough analysis, the findings don’t tell us whether ISTCs are good or bad. Both the length of stay effect and the casemix effect are ambiguous with respect to patient outcomes. If only we had some PROMs to work with…

One method, many methodological choices: a structured review of discrete-choice experiments for health state valuation. PharmacoEconomics [PubMed] Published 8th September 2018

Discrete choice experiments (DCEs) are in vogue when it comes to health state valuation. But there is disagreement about how they should be conducted. Studies can differ in terms of the design of the choice task, the design of the experiment, and the analysis methods. The purpose of this study is to review what has been going on; how have studies differed and what could that mean for our use of the value sets that are estimated?

A search of PubMed for valuation studies using DCEs – including generic and condition-specific measures – turned up 1132 citations, of which 63 were ultimately included in the review. Data were extracted and quality assessed.

The ways in which the studies differed, and the ways in which they were similar, hint at what’s needed from future research. The majority of recent studies were conducted online. This could be problematic if we think self-selecting online panels aren’t representative. Most studies used five or six attributes to describe options and many included duration as an attribute. The methodological tweaks necessary to anchor at 0=dead were a key source of variation. Those using duration varied in terms of the number of levels presented and the range of duration (from 2 months to 50 years). Other studies adopted alternative strategies. In DCE design, there is a necessary trade-off between statistical efficiency and the difficulty of the task for respondents. A variety of methods have been employed to try and ease this difficulty, but there remains a lack of consensus on the best approach. An agreed criterion for this trade-off could facilitate consistency. Some of the consistency that does appear in the literature is due to conformity with EuroQol’s EQ-VT protocol.

Unfortunately, for casual users of DCE valuations, all of this means that we can’t just assume that a DCE is a DCE is a DCE. Understanding the methodological choices involved is important in the application of resultant value sets.

Trusting the results of model-based economic analyses: is there a pragmatic validation solution? PharmacoEconomics [PubMed] Published 6th September 2018

Decision models are almost never validated. This means that – save for a superficial assessment of their outputs – they are taken at good faith. That should be a worry. This article builds on the experience of the authors to outline why validation doesn’t take place and to try to identify solutions. This experience includes a pilot study in France, NICE Evidence Review Groups, and the perspective of a consulting company modeller.

There are a variety of reasons why validation is not conducted, but resource constraints are a big part of it. Neither HTA agencies, nor modellers themselves, have the time to conduct validation and verification exercises. The core of the authors’ proposed solution is to end the routine development of bespoke models. Models – or, at least, parts of models – need to be taken off the shelf. Thus, open source or otherwise transparent modelling standards are a prerequisite for this. The key idea is to create ‘standard’ or ‘reference’ models, which can be extensively validated and tweaked. The most radical aspect of this proposal is that they should be ‘freely available’.

But rather than offering a path to open source modelling, the authors offer recommendations for how we should conduct ourselves until open source modelling is realised. These include the adoption of a modular and incremental approach to modelling, combined with more transparent reporting. I agree; we need a shift in mindset. Yet, the barriers to open source models are – I believe – the same barriers that would prevent these recommendations from being realised. Modellers don’t have the time or the inclination to provide full and transparent reporting. There is no incentive for modellers to do so. The intellectual property value of models means that public release of incremental developments is not seen as a sensible thing to do. Thus, the authors’ recommendations appear to me to be dependent on open source modelling, rather than an interim solution while we wait for it. Nevertheless, this is the kind of innovative thinking that we need.

Credits

Chris Sampson’s journal round-up for 11th June 2018

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

End-of-life healthcare expenditure: testing economic explanations using a discrete choice experiment. Journal of Health Economics Published 7th June 2018

People incur a lot of health care costs at the end of life, despite the fact that – by definition – they aren’t going to get much value from it (so long as we’re using QALYs, anyway). In a 2007 paper, Gary Becker and colleagues put forward a theory for the high value of life and high expenditure on health care at the end of life. This article sets out to test a set of hypotheses derived from this theory, namely: i) higher willingness-to-pay (WTP) for health care with proximity to death, ii) higher WTP with greater chance of survival, iii) societal WTP exceeds individual WTP due to altruism, and iv) societal WTP may exceed individual WTP due to an aversion to restricting access to new end-of-life care. A further set of hypotheses relating to the ‘pain of risk-bearing’ is also tested. The authors conducted an online discrete choice experiment (DCE) with 1,529 Swiss residents, which asked respondents to suppose that they had terminal cancer and was designed to elicit WTP for a life-prolonging novel cancer drug. Attributes in the DCE included survival, quality of life, and ‘hope’ (chance of being cured). Individual WTP – using out-of-pocket costs – and societal WTP – based on social health insurance – were both estimated. The overall finding is that the hypotheses are on the whole true, at least in part. But the fact is that different people have different preferences – the authors note that “preferences with regard to end-of-life treatment are very heterogeneous”. The findings provide evidence to explain the prevailing high level of expenditure in end of life (cancer) care. But the questions remain of what we can or should do about it, if anything.

Valuation of preference-based measures: can existing preference data be used to generate better estimates? Health and Quality of Life Outcomes [PubMed] Published 5th June 2018

The EuroQol website lists EQ-5D-3L valuation studies for 27 countries. As the EQ-5D-5L comes into use, we’re going to see a lot of new valuation studies in the pipeline. But what if we could use data from one country’s valuation to inform another’s? The idea is that a valuation study in one country may be able to ‘borrow strength’ from another country’s valuation data. The author of this article has developed a Bayesian non-parametric model to achieve this and has previously applied it to UK and US EQ-5D valuations. But what about situations in which few data are available in the country of interest, and where the country’s cultural characteristics are substantially different. This study reports on an analysis to generate an SF-6D value set for Hong Kong, firstly using the Hong Kong values only, and secondly using the UK value set as a prior. As expected, the model which uses the UK data provided better predictions. And some of the differences in the valuation of health states are quite substantial (i.e. more than 0.1). Clearly, this could be a useful methodology, especially for small countries. But more research is needed into the implications of adopting the approach more widely.

Can a smoking ban save your heart? Health Economics [PubMed] Published 4th June 2018

Here we have another Swiss study, relating to the country’s public-place smoking bans. Exposure to tobacco smoke can have an acute and rapid impact on health to the extent that we would expect an immediate reduction in the risk of acute myocardial infarction (AMI) if a smoking ban reduces the number of people exposed. Studies have already looked at this effect, and found it to be large, but mostly with simple pre-/post- designs that don’t consider important confounding factors or prevailing trends. This study tests the hypothesis in a quasi-experimental setting, taking advantage of the fact that the 26 Swiss cantons implemented smoking bans at different times between 2007 and 2010. The authors analyse individual-level data from Swiss hospitals, estimating the impact of the smoking ban on AMI incidence, with area and time fixed effects, area-specific time trends, and unemployment. The findings show a large and robust effect of the smoking ban(s) for men, with a reduction in AMI incidence of about 11%. For women, the effect is weaker, with an average reduction of around 2%. The evidence also shows that men in low-education regions experienced the greatest benefit. What makes this an especially nice paper is that the authors bring in other data sources to help explain their findings. Panel survey data are used to demonstrate that non-smokers are likely to be the group benefitting most from smoking bans and that people working in public places and people with less education are most exposed to environmental tobacco smoke. These findings might not be generalisable to other settings. Other countries implemented more gradual policy changes and Switzerland had a particularly high baseline smoking rate. But the findings suggest that smoking bans are associated with population health benefits (and the associated cost savings) and could also help tackle health inequalities.

Credits

Method of the month: Eye-tracking

Once a month we discuss a particular research method that may be of interest to people working in health economics. We’ll consider widely used key methodologies, as well as more novel approaches. Our reviews are not designed to be comprehensive but provide an introduction to the method, its underlying principles, some applied examples, and where to find out more. If you’d like to write a post for this series, get in touch. This month’s method is eye-tracking.

Principles

Eye-tracking methods can be used to analyse how individuals acquire information and how they make decisions. The method has been extensively used by psychologists in a variety of applications, from identifying cases of dyslexia in children to testing aviation pilots’ awareness. It was made popular by Keith Rayner, but its growing use reflects changes in the availability and affordability of technology. The textbook ‘Eye Tracking: A Comprehensive Guide to Methods and Measures’ provides a great introduction and complements the course offered by Lund University.

Eye-tracking analyses typically depend on the ‘eye-mind hypothesis’ which states “there is no appreciable lag between what is fixated on and what is processed”. In addition to fixing their gaze, individuals make rapid eye movements called ‘saccades’ when they are searching for information or items of interest. There is also research into ‘pupilometery’ which relates pupil size to cognitive burden, where ‘hard’ tasks are hypothesised to cause dilation.

There is a growing interest in using quantitative elicitation methods to understand individuals’ preferences for healthcare goods or services. Quantitative methods, such as the standard gamble, time trade-off, contingent valuation or discrete choice experiments, often employ surveys which are increasingly self-completed and administered online. These valuation methods are often underpinned by economic theories either for utility (random utility theory, expected utility theory) or, in the case of attribute-based approaches, Lancaster’s Theory of consumer demand. If respondents do not answer in line with the supporting theories, the valuations derived from the survey data may be biased. Therefore, various approaches have been employed to understand whether people complete surveys in line with the analysts’ expectations – from restricted models to test for attribute non-attendance to qualitative ‘think-aloud’ interviews. Eye-tracking offers an alternative method of testing these hypotheses.

Implementation

The research question will dictate the eye-tracking study design, if it’s a reading study or a survey then accuracy will be key. If the experiment seeks to understand how participants respond to large visual stimuli, then it may be preferable to have comfortable equipment which can be used in a setting the participant is familiar with.

Equipment

In its most basic form, eye-tracking research has involved researcher-individual observation of a participants’ eyes and manual notes on pupil dilation. However, more sophisticated methods have since developed in line with changes to, and availability of, technology. In the 1950s, magnetic search coils were used to track people’s eye movements which involved placing two coils on the eye, with one circling the iris on a contact lens. Nowadays, most eye-tracking involves less invasive equipment, commonly with a camera recording data on a computer and complex algorithms to calculate the location of the individual’s gaze.

To track eyes, almost all modern devices record the corneal reflection on a camera positioned towards the individual’s pupil. The corneal reflection is a glint, usually in the iris, which allows the machine to calculate the direction of the gaze using the distance from 1) the camera to the eye and; 2) the eye to the screen. From the corneal reflection, the X and Y (horizontal and vertical) coordinates, which provide the location of current focus on the screen, are then recorded. The number of times this is logged a second is referred to as the speed (or ‘frequency’) of the tracker. As the eye moves from one position to another, the magnitude of the movement is measured in visual degrees (θ), rather than millimetres, as studies may involve moving stimulus and so the distance between eye and object could change.

Popular manufacturers include Tobii, SensoMotoric Instruments (SMI) and SR Research. Eye-trackers are usually distinguished by their speed and, as a general rule, a ‘good’ eye-tracker has a high frequency and high-resolution camera. A higher frequency allows a more accurate estimation of the fixation duration, as the start of the fixation is revealed earlier and the end revealed later. There is some consensus that a sampling frequency of 500 Hz is sufficiently powerful to accurately determine fixations and saccades. Another determinant of a ‘good’ eye-tracking device is its ‘latency’, which is the time taken for the computer to make a recording. A substantial volume of processing from the headset to screen to recording is required and, for some devices, there is a measurable delay in this process.

There are three broad categories of modern eye-tracking devices.

Head-mounted

Head-mounted eye-trackers, such as smart glasses or helmet cameras, offer participants some freedom but are harder to calibrate and can be cumbersome to wear. These eye-trackers are often used to understand how objects are attended to in a dynamic situation, for example, whilst the participant is engaged in a shopping activity.

Remote

Remote eye-trackers let the participant move freely but in front of a screen, with algorithms used to detect non-eye movements [PDF]. However, the additional calculations to distinguish head and eye movements are a burden to the processing capacity of the computer and, generally, result in a lower frequency and, as a consequence, have decreased precision.

Head-supported towers

Head-supported towers involve the use of a forehead and chin-rest. Whilst being contactless, these can be uncomfortable and unnatural for some participants. These devices are also often immobile, due to their heavy processing power, and require stability of the head because of their high frequency. However, head-supported towers are the most accurate and precise equipment available for researchers. For studies where the individual is not required to move and the stimuli are stationary (such as a survey), a head-supported tower eye-tracker provides the best quality data. Head-supported towers also offer the most accurate recording of pupil size.

Data collection

Data collection will likely occur in a university lab if a head-supported tower tracker is chosen. Head-mounted and remote trackers are generally mobile and can, therefore, travel to people of interest.

Tracking devices can either record both eyes (binocular) or a single eye (monocular). When both eyes are recorded, an average of the horizontal and vertical coordinates from each eye are taken. However, most people generally have an ‘active’ and ‘lazy’ eye and literature suggests that the active, dominant eye should only be tracked. If a participant performs poorly in the calibration, then an alternative eye should be tried.

It is crucial that the eye-tracker is calibrated for each individual to ensure the eye-tracker is recording correctly. The calibration procedure involves collecting fixation data from simple points on the screen in order to ascertain the true gaze position of the individual before the experiment begins. The points are often shown as dots or crosses which move around the screen whilst fixation data are collected. A test of the calibration can be conducted by re-running the sequence and comparing the secondary fixations to the tracker’s prediction based on the first calibration data.

The calibration should involve points in all corners of the screen to ensure that the tracker is able to record in all areas. In the corners and edges, the corneal reflection can disappear, which therefore invalidates the computer’s calculations as well as resulting in missing data. Similarly, for individuals with visual aids (glasses, contact lenses) or heavy eye-makeup, the far corners can often induce another reflection which may confuse the recording and create anomalous data.

If a respondent is completing a survey, between-page calibration called ‘drift correction’ can also be completed. In this procedure, a small dot is presented in the centre of the screen and the next page appears once the participant has focussed on the spot. If there has been too much movement, the experiment will not progress and the tracker must be recalibrated.

Data analysis

Saccades are easily identifiable as the eye moves quickly in response to or in search of visual ‘stimuli’ or objects of interest. Saccadic behaviour rarely indicates information processing as the movements are so rapid that the brain is unable to consciously realise everything that is scanned, a process known as ‘saccadic suppression’. Instead, saccades most often represent a search for information. Saccades are distinctly different to ‘micro-saccades’, which are involuntary movements whilst an individual is attempting to fixate, and the involuntary movements which occur when an individual blinks. Blinks are quite easily identifiable from regular saccades as they are immediately followed by a missing pupil image on the camera as the eyelid closes.

What constitutes a fixation varies from study to study and is dependent on the stimulus presented. For example, a familiar picture may be processed quicker than text, and a new diagram may be somewhere in between. Although complex algorithms exist for the identification of fixations in eye-tracking data, most studies define a threshold for a fixation as less than one degree of movement (a measure of distance) for between 50 to 200 milliseconds. Aggregation of the total time spent fixating, including recurrent fixations, is defined as the ‘dwell time’ to a stimulus.

Eye-tracking data provide a highly detailed record of all the locations that a user has looked at, so reducing these data to a level that can be easily analysed is challenging. One common approach in the analysis of eye-tracking data involves segmenting coordinates to defined regions or ‘areas of interest’ (AOI). AOI can be defined either prior to the experiment or post-experimentally once eye-movement data have been collected.

Another approach to reducing the data is the generation of a ‘scan path’ describing the overall sequence of movements in terms of both saccades and fixations of a respondent, either imposed on a background image of the stimulus or as a colour-coded heat map.

Pupil size can be more difficult to interpret and analyse. Measurement of the pupil differs by equipment, some use an ellipse whereas others count the number of black pixels on the camera image of the eye. Pupil dilation can be calculated as the difference in pupil size, however, analysing this as a percentage increase can cause inflated estimates when the baseline pupil size is small. Pupil size can also rarely be compared across studies as it highly affected by equipment set-up and setting luminosity.

Software

Many eye-trackers come with manufacturer written software for programming the experiment (such as the EyeLink Experiment Builder). However, access to eye-trackers and related software may be restricted. PsychoPy is an open-source software, written in Python, for eye-tracking (and other neuroscience) experiments. Similarly, data can be analysed either in specialist software such as EyeLink’s Data Viewer (which is useful for creating scan paths) or the data can be exported to other statistical programmes such as Matlab, Stata or R.

Applications

Orquin & Loose review how eye-tracking methods have been used in decision-making generally. In health, there are a few examples of survey-based choice experiments which have employed eye-tracking methods to understand more about respondents’ decision-making. Spinks & Mortimer used a remote eye-tracker to identify attribute non-attendance. Krucien et al combined eye-tracking and choice data to model information processing; and Ryan extended this analysis to focus on presentation biases in a subsequent publication. Vass et al use eye-tracking to understand how respondents complete choice experiments and if this differed with the presentation of risk. A forthcoming publication by Rigby et al. will provide an overview of approaches, including eye-tracking, to capturing decision-making in health care choice experiments.

Credit