Inter-rater agreement in glioma segmentations on longitudinal MRI
van Duijn, RJM
van Herk, Marcel
de Witt, Hamer PC
de Munck, JC
AffiliationDepartment of Radiology and Nuclear Medicine, Amsterdam UMC, location VUmc, De Boelelaan 1117, 1081 HZ Amsterdam
MetadataShow full item record
AbstractBACKGROUND: Tumor segmentation of glioma on MRI is a technique to monitor, quantify and report disease progression. Manual MRI segmentation is the gold standard but very labor intensive. At present the quality of this gold standard is not known for different stages of the disease, and prior work has mainly focused on treatment-naive glioblastoma. In this paper we studied the inter-rater agreement of manual MRI segmentation of glioblastoma and WHO grade II-III glioma for novices and experts at three stages of disease. We also studied the impact of inter-observer variation on extent of resection and growth rate. METHODS: In 20 patients with WHO grade IV glioblastoma and 20 patients with WHO grade II-III glioma (defined as non-glioblastoma) both the enhancing and non-enhancing tumor elements were segmented on MRI, using specialized software, by four novices and four experts before surgery, after surgery and at time of tumor progression. We used the generalized conformity index (GCI) and the intra-class correlation coefficient (ICC) of tumor volume as main outcome measures for inter-rater agreement. RESULTS: For glioblastoma, segmentations by experts and novices were comparable. The inter-rater agreement of enhancing tumor elements was excellent before surgery (GCI 0.79, ICC 0.99) poor after surgery (GCI 0.32, ICC 0.92), and good at progression (GCI 0.65, ICC 0.91). For non-glioblastoma, the inter-rater agreement was generally higher between experts than between novices. The inter-rater agreement was excellent between experts before surgery (GCI 0.77, ICC 0.92), was reasonable after surgery (GCI 0.48, ICC 0.84), and good at progression (GCI 0.60, ICC 0.80). The inter-rater agreement was good between novices before surgery (GCI 0.66, ICC 0.73), was poor after surgery (GCI 0.33, ICC 0.55), and poor at progression (GCI 0.36, ICC 0.73). Further analysis showed that the lower inter-rater agreement of segmentation on postoperative MRI could only partly be explained by the smaller volumes and fragmentation of residual tumor. The median interquartile range of extent of resection between raters was 8.3% and of growth rate was 0.22?mm/year. CONCLUSION: Manual tumor segmentations on MRI have reasonable agreement for use in spatial and volumetric analysis. Agreement in spatial overlap is of concern with segmentation after surgery for glioblastoma and with segmentation of non-glioblastoma by non-experts.
CitationVisser M, Muller DMJ, van Duijn RJM, Smits M, Verburg N, Hendriks EJ, et al. Inter-rater agreement in glioma segmentations on longitudinal MRI. Neuroimage Clin. 2019 Feb 22;22:101727.
- Reliability of Semi-Automated Segmentations in Glioblastoma.
- Authors: Huber T, Alber G, Bette S, Boeckh-Behrens T, Gempt J, Ringel F, Alberts E, Zimmer C, Bauer JS
- Issue date: 2017 Jun
- Intra-rater variability in low-grade glioma segmentation.
- Authors: Bø HK, Solheim O, Jakola AS, Kvistad KA, Reinertsen I, Berntsen EM
- Issue date: 2017 Jan
- Multi-modal glioblastoma segmentation: man versus machine.
- Authors: Porz N, Bauer S, Pica A, Schucht P, Beck J, Verma RK, Slotboom J, Reyes M, Wiest R
- Issue date: 2014
- Glioblastoma Segmentation: Comparison of Three Different Software Packages.
- Authors: Fyllingen EH, Stensjøen AL, Berntsen EM, Solheim O, Reinertsen I
- Issue date: 2016
- Volumetric glioma quantification: comparison of manual and semi-automatic tumor segmentation for the quantification of tumor growth.
- Authors: Odland A, Server A, Saxhaug C, Breivik B, Groote R, Vardal J, Larsson C, Bjørnerud A
- Issue date: 2015 Nov