This event was originally scheduled for June 30th in London. However, due to COVID-19 restrictions, the event took place online in October. The workshop was the best online event that I have attended during this stay-at-home period, thanks to the high standard of presentations, pro-active organisers, and lively text-based chat happening alongside the sessions.
The event was organised by the R for HTA consortium, the lead organisers and chairs for the meeting were Gianluca Baio and Howard Thom. A broad range of topics related to using R in the health technology assessment process was covered. A key element of the format was that each talk included a demonstration of the R code, which was also available to download, giving the experience of real hands-on learning. The final day concluded with a fascinating panel discussion on the value and challenges of using R in HTA. Recordings of the talks are available on Youtube and slides are available at the R for HTA website.
Sessions were pitched at a variety of levels of R experience. At the more introductory end of the scale Iryna Schlackow presented an entertaining and informative review of good coding practices in R using the popular Tidyverse packages. It’s well worth checking out even if you already know the basics of coding in R. Mi Jun Keng took a systematic look at some standard ways to improve the efficiency of calculations in R, which was very useful to help understand the somewhat counterintuitive apply function family.
More suited to intermediate and advanced R users, there were sessions on patient heterogeneity, propensity scores, the hesim package and the SpeedyMarkov package. Introducing patient heterogeneity into a model can be a challenge. Joe Moss introduced how models developed in R can be adapted to achieve this with relative ease. A conceptual overview of how to do so, together with a nice worked example in chronic kidney disease, was presented. As a user of propensity scores, I know how time consuming this type of analysis can be. Best practices when using propensity score methods include a number of preparatory steps and diagnostic checks and the work can multiply when sensitivity analysis is considered. Kevin Deighton demostrated a template to make this easy and convenient.
Sam Abbott introduced the SpeedyMarkov package and explained some of its inner workings. A key insight was that R’s relatively poor efficiency in implementing loop structures can be overcome by using complementary C++ code. Despite being more in-depth, this topic was well explained. The hesim package was introduced by Devin Incerti. This talked packed in a lot of detailed discussion of the many features of this speacialist health economic modelling R package, including data analysis for multi-state models, semi-markov structures (with time-dependent transition probabilities), and production of cost-effectiveness analysis results. The talk is a very useful resource if you are considering using the hesim package.
A variety of other applied topics were covered in the remaining sessions. There was a nice worked example of a cost-effectiveness analysis developed in R using a SIR-type infectious disease model from Josephine Walker. R-based cost-effectiveness analysis may be especially of interest for infectious disease applications because of the popularity of R for the underlying disease models. Seamus Kent introduced the idea of the OMOP common data model. This talk summarised current efforts and I got the impression that this will be something that will affect many projects in a positive way in the future.
Experiences in model conversion from Excel to R were presented by Yiqiao Xin (CoI declaration: I was part of the team on this project). The talk reflected on the process of converting a complex cardiovascular disease model. Issues in model conversion such as whether to use base R or specialised packages were considered, as well as how experienced modellers but inexperienced R users can adapt to the R environment.
A new use of Shiny was suggested by Philip Cooney, as a visual aid for expert elicitation of model parameters. R Shiny is well suited for this purpose because it is possible to visualise the effect of changing a parameter that is being elicited and which feeds through to outcomes. I can see this being helpful when the elicitation is trying to get at more abstract parameters. Felicity Lamrock demostrated an innovative method to calculate confidence intervals for transition probabilities using the MSM package. This is the type of problem where an intermediate R user like myself would struggle without this type of example for guidance.
Finally to the panel discussion, Value and challenges of using R in HTA, with Francois Maignen, Venediktos Kapetanakis, Liz Fenwick, and Andy Briggs. The panel discussion was wide-ranging. There were reflections from Francois Maignen on regulator’s reservations about accepting R-based models and from Venediktos Kapetanakis and Liz Fenwick on successes and challenges using R with clients in HTA consultancy. The advantages and disadvantages of R compared to Excel and other options was a key topic. Two pithy statements from Andy Briggs stimulated further discussion: “transparency is in the eye of the beholder” and “Excel and R are just calculators”.
To summarise the key points, Excel and R are just alternative tools for the same purpose. As health economists, we should be focused on using the best methods rather than the best calculator. As long as the calculator is suitable for programming the model in question then that is sufficient. Putative advantages such as convenience and transparency are subjective and often depend more on past experience than anything intrinsic to the software.
My own opinion on this is that we should consider how R might improve applied modelling by thinking about how the move to R influences methodological choices. Many types of complexity are more easily encoded in R. For example – as mentioned by other participants in response to my question – treatment sequencing models, investigation of patient heterogeneity, PSA diagnostics, and more. Models with greater complexity or flexibility can add value for informing decisions but this often comes at a price, requiring more of the modeller’s time and creating a higher barrier for a decision maker trying to understand the analysis.
Only if we think the methodological possibilities that R makes more accessible are truly desirable, and important, will we have the motivation to invest our time to accelerate the movement to R. To put it another way, we should move to R for better reasons than feeling embarrassment among our scientific colleagues about using Excel.