Meeting round-up: Society for Medical Decision Making 17th Biennial European Conference

The Society for Medical Decision Making (SMDM) held their 17th European Conference between 10th and 12th June at the Stadsgehoorzaal in Leiden, the Netherlands. The meeting was chaired by Anne Stiggelbout and Ewout Steyerberg who, along with Uwe Siebert, welcomed us (early) on Monday morning. Some delegates arrived on Sunday for short courses on a range of topics, from modelling in R and causal inference to the psychology of decision making.

Although based in the US, SMDM holds biennial meetings in Europe which are generally attended by delegates from around the world. Around 300 delegates were in attendance at this meeting, travelling from Toronto to Tehran.

The meeting was ‘Patients Included’ and we were introduced to around 10 patients and caregivers on the first morning. They confidently asked questions and gave comments after the presentations and the plenary, sharing their real-world experience to provide context to findings.

There were five ‘oral abstract’ sessions each comprising six presentations in 15 minute slots (10 minutes long with 5 minutes for audience questions). The sessions covered empirical research relating to physician and patient decision-making, and quantitative valuation and evaluation. Popular applied areas were prostate cancer, breast cancer and precision medicine.

Running in parallel to the oral presentations, workshops were dealing with methodological issues relating to health economics, shared decision-making and psychology.

Four poster sessions, conveniently surrounding the refreshment table, attracted delegates in the morning, breaks and lunch. SMDM provides some of the best poster sessions: posters are always of high quality which means poster sessions are always well attended.

One of the highlights of the meeting was the plenary presentation by Sir David Spiegelhalter who spoke about the challenges of communicating benefits and harms (often probabilities) impartially. Sir David gave examples from the UK’s national breast screening programme to show how presenting information can change people’s interpretation of risk. He also drew on examples of ‘nudges’ which may involve providing information in a persuasive rather than informing way in order to manipulate behaviour. Sir David gave us examples of materials which had been redesigned to improve both patients’ and clinicians’ understanding of the information of benefits and harms. The session concluded with a short video about how Ugandan primary school children have reading comic strips to help interpret information and find facts about the benefits and harms of healthcare interventions.

The European SMDM meeting was thoroughly enjoyable and very interesting. The standard of oral and poster presentations was very high, and the environment was very friendly and conducive to networking.

The next North American meeting is in Montreal (October 2018) and the next European meeting will be in 2020 (location to be confirmed).


Call for a model registry

Decision models can be very useful. They can also be very bad. ISPOR and SMDM recently got together to form the Modeling Good Research Practices Task Force and have produced a set of reports. One of the reports, by David Eddy and colleagues, considers Model Transparency and Validation. The authors discuss the importance of transparency and validation in ensuring model credibility. They provide an excellent set of best practices that all modellers should follow. Go and read it.

The problem

The apparent effectiveness of a health care intervention can be distorted in many ways. Ben Goldacre‘s new book does a pretty good job of covering these. When it comes to cost-effectiveness the opportunities possibilities for distortion increase exponentially. A recent paper investigated FDA actions against health economic promotions. Many companies seek to promote products based on their economic value of “saving money” or “lowering costs”. The most common type of economic violation was an implied claim of cost savings due to work productivity or functioning. One can imagine how an unjustified figure of this nature could go unrecognised in a model.

There will always be intentionally misleading models, with which regulators will have to deal. However, most researchers’ models will have validity issues in spite of good intentions. If Eddy et al’s best practices were adhered to by good researchers then bad models could be easily identified and amended. The authors argue that, to ensure transparency, each model should be accompanied by both technical and non-technical documentation available to interested readers. However, a feasible process of ensuring transparency in this way does not currently exist.

The solution

The solution is to establish a model registry and database. It is unrealistic to expect journal publishers to accept extensive appendices, to which additions would need to be made as a model is developed.

Setting up a registry and online database would be very simple. Inclusion in the registry should require a minimum submission to the database: Eddy et al’s proposed non-technical documentation. Further information and files could be optional. Each model, whether single-application or multi-application, would be assigned a registration number and a corresponding database entry and webpage. The webpage could include all of the technical and non-technical documentation required by Eddy et al’s best practices. It could also include relevant citations, manuals and code. The site should provide a means for feedback and discussion. As models are updated and improved, or used in other applications, details can be added about subsequent versions. I’m not convinced of the argument presented by Eddy et al regarding intellectual property. This is meant to be science; one wouldn’t expect a health researcher to keep their trial design secret. Regardless, a registry would further protect intellectual property rights. Copyright is an automatic right in most countries, and plagiarism or use without appropriate attribution could easily be recognised and acted upon in a transparent system. The registry would provide a means of ensuring appropriate attribution of intellectual property.

There will be costs associated with maintaining such a registry and database (web hosting, doi fees), but most would be achieved by the contribution of researchers’ time. Academics are well accustomed to pro bono work and this shouldn’t represent a serious barrier. The primary challenge would be in encouraging researchers to submit their models to the registry. Model registration could not be a pre-requisite for publication in the same way as trial registration, so researchers would have to see other incentives. I would see the primary benefit as increased citations. If people are allowed to use existing models this will lead to further citation of the original, which could be facilitated by the provision of a Digital Object Identifier and would promote model validation.

An open and transparent registry would enable the research community to investigate the validity of models, which may be used to inform policy decisions. A registry would also help prevent the duplication of efforts. So, I call on relevant organisations (ISPOR, SMDM, OHE, CRD etc) to establish a registry of this nature. If you don’t do it soon I’ll have to do it myself.

DOI: 10.6084/m9.figshare.1246218