Assessing the generalisability of radiomics features for predicting sticky saliva and xerostomia

Abstract

Purpose or Objective Few studies reporting radiomics-based models for prediction of clinical outcomes are externally validated and fewer are replicated. While core to the scientific approach, reproducibility of experimental results, is often challenging for such studies because of the complexity of the methods. Recently, on a cohort of patients with head and neck cancer (HNC), van Dijk et al identified radiomics features that improve prediction of moderate-to-severe sticky saliva (SS12m) and xerostomia (Xer12m) at 12 months after radiotherapy, compared to models only based on dose and clinical parameters. In this replication study, we assessed the generalisability of these findings using a different cohort of HNC patients. Materials and Methods The methods described by van Dijk et al were applied to a cohort of 109 HNC patients treated with 50-70Gy in 20-35fx using TomoTherapy. Xerostomia and sticky saliva scores were collected at baseline and 12 months after RT (EORTC QLQ-HN35). For each patient, a planning CT (Toshiba Aquilion/LB) was acquired and parotid and submandibular glands (SMG) contoured. Imaging features identified by van Dijk et al as associated with the clinical outcomes of interest were calculated on each slice of the contoured structure on planning CTs. Specifically, van Dijk et al found Short Run Emphasis (SRE) and maximum CT intensity (maxHU) to improve prediction of Xer12m and SS12m respectively, compared to models solely using baseline toxicity and mean dose to the salivary glands. We evaluated, on our cohort, the predictive performance of the variables identified by van Dijk et al. However, inherent differences were present between the two approaches (Table 1). In an attempt to determine the impact these differences had on the performance of the models, tests were run on subgroups of patients with varying proportions having: 1) intact salivary glands, 2) excluded CT slices with dental implants, and 3) consistent fractionation schedules. Results None of the univariate associations between radiomic features identified by van Dijk et al with the outcome of interest could be replicated on our cohort (Table 2). The addition of SRE to the standard model did not improve Xer12m prediction on any of the subgroups of patients tested. For patients with both SMG intact, the addition of the feature maxHU was found to improve the AUC from 0.53 to 0.66. Conclusion While none of univariate associations identified by van Dijk et al as being statistically significant could be replicated, the addition of maxHU improved SS12m prediction on patients with both SMG intact. These variations, and the limited generalisability of the findings, may be explained by the number of differences in the imaging characteristics of the two studies and subsequent methodological implementation. This highlights the importance of external validation as well as high quality reporting guidelines and standardisation protocols to ensure generalisability, replication and ultimately clinical implementation.