'Vicarious thinking' was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results
AffiliationDivision of Cancer Sciences, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Cancer Research Centre, Wilmslow Road, Manchester
MetadataShow full item record
AbstractObjective: The objectives of this nested study were to (1) assess whether changes in scores between rounds altered the final degree of consensus achieved in three Delphi surveys conducted as part of COS development projects (anal, gastric, and prostate cancer), and (2) explore participants' reasons for changing scores between rounds. Study design and setting: All Delphi surveys were conducted online using DelphiManager software and included healthcare professionals and participating patients. Participants were invited to give a free-text reason whenever they changed their score across an important threshold on a 1-9 Likert scale (1-3 not important, 4-5 important, 7-9 critically important). Reasons for score change were coded by four researchers independently using an inductive-iterative approach. Results: In all three Delphi surveys, the number of outcomes reaching criteria for consensus was greater in R2 than R1. Twelve themes and 23 subthemes emerged from 2298 discrete reasons given for score change. The most common reasons for the change were "time to reflect" (482 responses, 23%) and vicarious thinking (424, 21%), with 68% (291) of vicarious thinking attributed to seeing other participant's scores. Conclusion: Our findings support conducting a Delphi survey over the use of a single questionnaire where building consensus is the objective. Time to reflect and vicarious thinking, facilitated by seeing other participant's scores, were important drivers of score change. How results are presented to participants between rounds and the duration of and time between rounds in a Delphi survey may, therefore, influence the results and should be clearly reported.
CitationFish R, MacLennan S, Alkhaffaf B, Williamson PR. "Vicarious thinking" was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results. J Clin Epidemiol. 2020;128:118-29.
JournalJournal of Clinical Epidemiology
- Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development.
- Authors: Brookes ST, Macefield RC, Williamson PR, McNair AG, Potter S, Blencowe NS, Strong S, Blazeby JM
- Issue date: 2016 Aug 17
- Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial.
- Authors: Brookes ST, Chalmers KA, Avery KNL, Coulman K, Blazeby JM, ROMIO study group.
- Issue date: 2018 Jan 25
- Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance.
- Authors: Biggane AM, Williamson PR, Ravaud P, Young B
- Issue date: 2019 Nov 14
- Core outcome set for research studies evaluating treatments for twin-twin transfusion syndrome.
- Authors: Perry H, Duffy JMN, Reed K, Baschat A, Deprest J, Hecher K, Lewi L, Lopriore E, Oepkes D, Khalil A, International Collaboration to Harmonise Outcomes for Twin-Twin Transfusion Syndrome (CHOOSE).
- Issue date: 2019 Aug
- Core Outcome Set for Actinic Keratosis Clinical Trials.
- Authors: Reynolds KA, Schlessinger DI, Vasic J, Iyengar S, Qaseem Y, Behshad R, DeHoratius DM, Denes P, Drucker AM, Dzubow LM, Etzkorn JR, Harwood C, Kim JYS, Lee EH, Lissner GS, Marghoob AA, Matin RN, Mattox A, Mittal BB, Thomas JR, Zhou XA, Zloty D, Schmitt J, Kirkham J, Poon E, Sobanko JF, Cartee TV, Maher IA, Alam M
- Issue date: 2020 Mar 1