Meeting round-up: R for Cost-Effectiveness Analysis Workshop 2019

I have switched to using R for my cost-effectiveness models, but I know that I am not using it to its full potential. As a fledgling R user, I was keen to hear about other people’s experiences. I’m in the process of updating one of my models and know that I could be coding things better. But with so many packages and ways to code, the options seem infinite and I struggle to know where to begin. In an attempt to remedy this, I attended the Workshop on R for trial and model-based cost-effectiveness analysis hosted at UCL. I was not disappointed.

The day showcased speakers with varying levels of coding expertise doing a wide range of cool things in R. We started with examples of implementing decision trees using the CEdecisiontree package and cohort Markov models. We also got to hear about a population model using the HEEMOD package, and the purrr package was suggested for probabilistic sensitivity analyses. These talks highlighted how, compared to Excel, R can be reusable, faster, transparent, iterative, and open source.

The open source nature of R, however, has its drawbacks. One of the more interesting conversations that was woven in throughout the day was around the challenges. Can we can trust open-source software? When will NICE begin accepting models coded in R? How important is it that we have models in something like Excel that people can intuitively understand? I’ve not experienced problems choosing to use R for my work; for me, it’s always been around getting the support and information I need to get things done efficiently. The steep learning curve seems to be a major hurdle for many people. I had hoped to attend the short course introduction that was held the day before the workshop, but I was not fast enough to secure my spot as the course sold out within 36 hours. Never fear, the short course will be held again next year in Bristol.

To get around some of the aforementioned barriers to using R, James O’Mahony presented work on an open-source simplified screening model that his team is developing for teaching. An Excel interface with VBA code writes the parameter values in a file that can be imported into R, which has a single file of short model code. Beautiful graphs show the impact of important parameters on the efficiency frontier. He said that they would love to have people look at the code and give suggestions as they want to keep it simple but there is a nonlinear relationship between additional features and complexity.

And then we moved on to more specific topics, such as setting up a community for R users in the NHS, packages for survival curves, and how to build packages in R. I found Gianluca Baio’s presentation on what is a package and why we should be using them really helpful. I realised that I hadn’t really thought about what a package was before (a bundle of code, data, documentation and tests that is easy to share with others) or that it was something that I could or (as he argued) should be thinking about doing for myself as a time-saving tool even if I’m not sharing with others. It’s no longer difficult to build a package when you use packages like devtools and roxygen2 and tools like rstudio and github. He pointed out that packages can be stored on github if you’re not keen to share with the wider world via CRAN.

Another talk that I found particularly helpful was on R methods to prepare routine healthcare data for disease modelling. Claire Simons from The University of Oxford outlined her experiences of using R and ended her talk with a plethora of useful tips. These included using the data.table package for big data sets as it saves time when merging, use meaningful file names to avoid confusion later, and investing in doing things properly from the start as this will save time later. She also suggested using code profiling to identify which code takes the most time. Finally, she reminded us that we should be constantly learning about R: read books on R and writing algorithms and talk to other people who are using R (programmers and other people, not just health economists).

For those who agree that the future is R, check out resources from the Decision Analysis in R for Technologies in Health, a hackathon at Imperial on 6-7th November, hosted by Nathan Green, or join the ISPOR Open Source Models Special Interest Group.

Overall, the workshop structure allowed for a lot of great discussions in a relaxed atmosphere and I look forward to attending the next one.

Thesis Thursday: Rebecca Addo

On the third Thursday of every month, we speak to a recent graduate about their thesis and their studies. This month’s guest is Dr Rebecca Addo who has a PhD from the University of Technology Sydney. If you would like to suggest a candidate for an upcoming Thesis Thursday, get in touch.

Title
The feasibility of health technology assessment (HTA) in the Ghanaian health system
Supervisors
Jane Hall, Stephen Goodall, Marion Haas
Repository link
http://hdl.handle.net/10453/133353

Why is now the right time to research the feasibility of HTA in Ghana?

In recent years, Ghana has been struggling to financially sustain the National Health Insurance Scheme (NHIS), through which it aims to attain universal health coverage (UHC). As a result, a number of payment methods have been explored, including capitation, but costs to the NHIS continue to escalate. The search for a more efficient NHIS funding resulted in stakeholders visiting the then NICE International, learnt of HTA, and expressed an interest in pursuing it. This interest was strengthened by the World Health Organization 2014 resolution, which encouraged its member states to adopt health interventions and technology assessments in support of UHC. In 2016, a pilot HTA study was conducted with support from international bodies that demonstrated potential cost savings with HTA. Subsequently, the Ghana National Medicines Policy, 2017, made provisions for the use of HTA in the selection of medicines. What remains uncertain is how the policy will be implemented, considering that the limited use of HTA in developing countries has been attributed to a lack of human capacity to undertake it, quality data, and limited resources to support it. With Ghana making progress towards the formal adoption of HTA for health decision-making, it is important to examine its feasibility considering the available national capacity and the health system’s particular characteristics, and to make recommendations on how Ghana can proceed, so that the anticipated positive changes can be realised.

What determines ‘feasibility’ in this context?

The usefulness of HTA to any health system is highly dependent on its availability, the quality of assessment, and the human capacity to conduct country specific appraisals. Thus ‘feasibility’ in this context is determined by the existing health resources and systems that could support the adoption and use of HTA in Ghana. Health resources include human capacity with the needed technical skills to conduct and contribute to HTA, funding for the HTA processes, and the available data, which is of good quality and easily accessible. In addition, potential users of HTA should have knowledge in HTA and be able to interpret its findings. Without these building blocks, HTA in itself cannot be successfully used in Ghana. The systems to consider are health system characteristics such as existing health decision-making processes, and political and social structures. Knowledge of this would aid with planning, design, and introduction of an HTA process that suits the Ghanaian health system’s decision-making context, which would promote its use.

How is HTA perceived by stakeholders in Ghana?

Whilst the majority of Ghanaian stakeholders who participated in my study understood HTA as a decision making tool, others saw it as using technologies such as telemedicine and mobile phone devices for healthcare delivery. Their prior understanding of HTA and its uses drove these differences. In terms of its potential use in the Ghanaian health system, most stakeholders acknowledged the benefits the health system stood to gain should HTA be adopted. They however perceived some barriers to the successful implementation of HTA and made some recommendations to address them. Perceived barriers included lack of knowledge of HTA by potential users, lack of human resource capacity to conduct it, lack of funds to support the conduct, and existing ways of making decisions. Factors perceived to promote HTA use were allocating funds for HTA activities, educating stakeholders on HTA and involving them in the planning, and introduction of HTA for health decision-making in Ghana. Also, stakeholders recommended that data be collated and managed for HTA, and for local Ghanaians to be trained to conduct HTA but rely on experts from other countries where possible.

Was it especially challenging to conduct an economic evaluation in the Ghanaian context?

Yes. Conducting a Ghanaian specific economic evaluation was very challenging, especially, in getting the appropriate data. There were no country-specific utility and clinical efficacy data, hence, I had to rely on data from elsewhere, which needed to the transformed to be context specific. The most challenging aspect was with getting appropriate clinical data due to the differences between clinical trial settings and the Ghanaian setting. Applicability issues that were addressed included differences in clinical treatment algorithm, alternative treatments, and epidemiology of disease. Cultural acceptance of available treatment for the study population also defined the appropriate comparator for the evaluation and consequently the clinical data that could be considered. This resulted in having to draw on data from two separate arms of two clinical trials for one of the models I built for my economic evaluation. To ensure applicability of data from other countries to Ghana, the data identified were transformed to be context specific with data input from Ghana either not available or not easily accessible. Therefore, clinical experts were relied upon for such inputs, adding to the limitations of the economic evaluation.

Can HTA processes from other countries be applied in Ghana?

Every health system is unique in its entirety, therefore processes used in one cannot be adopted and applied to the other. The same applies to HTA in Ghana. As part of my thesis, I reviewed a number of HTA organisations across the world to assess if one could be adopted in Ghana. The review revealed that HTA processes vary with each health system in terms of the context under which they were established, the scope or focus of HTA, outcomes, and links to funding decisions and their uses. The establishment of most of these HTA organisations was driven by country specific needs such as curbing the rising costs of healthcare and reducing variations in the availability of quality treatment and care. The available resources, such as human and data, and the health systems characteristics also influenced the HTA processes. Therefore it is not advisable for Ghana to simply adopt and use a model of HTA process from other countries. Rather, Ghana must pursue a country specific HTA process that is informed by relevant country data.

What would be your recommended ‘next step’ for HTA in Ghana?

Firstly, to ensure the acceptance, use and diffusion of HTA in Ghana, stakeholders of health should be educated on HTA and a legal framework stipulating its focus and conduct, and mandating its use, to be adopted.

Secondly, in the short-to-medium term, Ghana can leverage on ongoing collaborations with other countries and foreign organisations, such as the International Decision Support initiative (IDSi), to develop local capacity for HTA. In the long-term, it will be necessary for policy makers to explore the human resource capacity available for HTA in Ghana to guide the development of a human resource plan for HTA.

Thirdly, Ghana has to develop a country-specific methodological guideline or adapt an existing one for the conduct and reporting of economic evaluation studies in Ghana. Subsequently, guidelines for conducting HTA should be developed.

Lastly, to support HTA conduct, Ghana must create a national data repository including a manual on health resource use and their corresponding unit prices. The creation of an HTA standing panel of clinical experts and other stakeholders who could be relied upon to supply inputs for HTA when needed is also recommended. This is very important in the Ghanaian setting where availability and access to data is limited.

Rachel Houten’s journal round-up for 22nd April 2019

Every Monday our authors provide a round-up of some of the most recently published peer reviewed articles from the field. We don’t cover everything, or even what’s most important – just a few papers that have interested the author. Visit our Resources page for links to more journals or follow the HealthEconBot. If you’d like to write one of our weekly journal round-ups, get in touch.

To HTA or not to HTA: identifying the factors influencing the rapid review outcome in Ireland. Value in Health [PubMed] Published 6th March 2019

National health services are constantly under pressure to provide access to new medicines as soon as marketing authorisation is granted. The NCPE in the Republic of Ireland has a rapid review process for selecting medicines that require a full health technology assessment (HTA), and the rest, approximately 45%, are able to be reimbursed without such an in-depth analysis.

Formal criteria do not exist. However, it has previously been suggested that the robustness of clinical evidence of at least equivalence; a drug that costs the same or less; an annual (or estimated) budget impact of less than €0.75 million to €1 million; and the ability of the current health systems to restrict usage are some of what is considered when making the decision.

The authors of this paper used the allocation over the past eight years to explore the factors that drive the decision to embark on a full HTA. They found, unsurprisingly, that first-in-class medicines are more likely to require an HTA as too are those with orphan status. Interestingly, the clinical area influenced the requirement for a full HTA, but the authors consider all of these factors to indicate that high-cost drugs are more likely to require a full assessment. Drug cost information is not publicly available and so the authors used the data available on the Scottish Medicine Consortium website as a surrogate for costs in Ireland. In doing so, they were able to establish a relationship between the cost per person for each drug and the likelihood of the drug having a full HTA, further supporting the idea that more expensive drugs are more likely to require HTA. On the face of it, this seems eminently sensible. However, my concern is that, in a system that is designed to deliberately measure cost per unit of health care (usually QALYs), there is the potential for lower-cost but ineffective drugs to become commonplace while more expensive medicines are subject to more rigor.

The paper provides some insight into what drives a decision to undertake a full HTA in Ireland. The NICE fast-track appraisal system operates as an opt-in system where manufacturers can ask to follow this shorter appraisal route if their drug is likely to produce an ICER of £10,000 or less. As my day job is for an Evidence Review Group (opinions my own), how things are done elsewhere – unsurprisingly – captured my attention. The desire to speed up the HTA process is obvious but the most appropriate mechanisms in which to do so are far from it. Whether or not the same decision is ultimately made is what concerns me.

NHS joint working with industry is out of public sight. BMJ [PubMed] Published 27th March 2019

This paper suggests that ‘joint working arrangements’ – a government-supported initiative between pharmaceutical companies and the NHS – are not being implemented according to guidelines on transparency. These arrangements are designed to promote collaborative research between the NHS and industry and help advance NHS provision of services.

The authors used freedom of information requests to obtain details on how many trusts were involved in joint working arrangements in 2016 and 2017. The declarations of payments made by drug companies are disclosed but the corresponding information from trusts is less readily accessible, and in some cases access to any details was prevented. Theoretically, the joint working arrangements are supposed to be void of any commercial influence on what is prescribed, but my thoughts are echoed in this paper when it asks “what’s in it for the private sector?” The sheer fact that some NHS trusts were unwilling to provide the BMJ with the information requested due to ‘commercial interest’ rings huge alarm bells.

I’m not completely cynical of these arrangements in principle, though, and the paper cites a couple of projects that involved building new facilities for age-related macular generation, which likely offer benefits to patients, and possibly much faster than could have been achieved with NHS funding alone. Some of the arrangements intend to push the implementation of national guidance, which, as a small cog in the guidance generation machine, I unashamedly (and predictably) think is a good thing.

Does it matter to us? As economists, it means that any work based on national practice and costs is likely to be unrepresentative of what actually happens. This, however, has always been the case to some extent, with variations in local service provision and the negotiation power of trusts with large volumes of patients. A national register of the arrangements would have the potential to feed into economic analysis, even if just as a statement of awareness.

Can the NHS survive without getting into bed with industry? Probably not. I think the paper does a good job of presenting the arguments on all sides and pushing for increasing availability of what is happening.

Estimating joint health condition utility values. Value in Health [PubMed] Published 22nd February 2019

I’m really interested in how this area is developing. Multi-morbidity is the norm, especially as we age. Single condition models are criticised for their lack of representation of patients in the real world. Appropriately estimating the quality of life of people with several chronic conditions, when only individual condition data are available, is incredibly difficult.

In this paper, parametric and non-parametric methods were tested on a dataset from a large primary care patient survey in the UK. The multiplicative approach was the best performing for two conditions. When more than two conditions were considered, the linear index (which incorporates additive, multiplicative, and minimum models with the use of linear regression and parameter weights derived from the underlying data) achieved the best results.

Including long-term mental health within the co-morbidities for which utility was estimated produced biased estimates. The authors discuss some possible explanations for this, including the fact that the anxiety and depression question in the EQ-5D is the only one which directly maps to an individual condition, and that mental health may have a causal effect on physical health. This is a fascinating finding, which has left me somewhat scratching my head as to how this oddity could be addressed and if separate methods of estimation will need to be used for any population with multi-morbidity including mental health conditions.

It did make me wonder if more precise EQ-5D data could be helpful to uncover the true interrelationships between joint health conditions and quality of life. The EQ-5D asks patients to think about their health state ‘today’. Although the primary care dataset used includes 16 chronic health conditions, it doesn’t, as far as I know, contain any information on the symptoms apparent on the day of quality of life assessment, which could be flaring or absent at any given time. This is a common problem with the EQ-5D and I don’t think a readily available data source of this type exists, so it’s a thought on ideals. Unsurprisingly, the more joint health conditions to be considered, the larger the error in terms of estimation from individual conditions. This may be due to the increasing likelihood of overlap in the symptoms experienced across conditions and thus a violation of the assumption that quality of life for an individual condition is independent of any other condition.

Whether the methodology remains robust for populations outside of the UK or for other measures of utility would need to be tested, and the authors are keen to highlight the need for caution before running away and using the methods verbatim. The paper does present a nice summary of the evidence to date in this area, what the authors did, and what it adds to the topic, so worth a read.

Credits