The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT

“…and it has served as an invaluable resource to the protein structure prediction community over the last 30 years as it has progressed through several generations of computational biochemistry [29], with the latest being dominated by deep learning methods such as DeepMind’s ”AlphaFold” [26].”

“05 and corrections for multiple hypothesis testing were made using the Holm-Bonferroni procedure [23].”

“All models were trained with a combination of Dice and cross-entropy losses [24].”

“Increasingly, the so-called “radiome” is revealing itself as a powerful quantitative predictor of clinically-meaningful outcomes in cancer [15].”

“Researchers often exploit this property in conjunction with a variety of nonparametric dimensional reduction techniques [49, 38] in order to visualize the clusters of input samples which appear to be represented by the network in similar ways.”

“In surgical oncology, a number of “nephrometry” scores such as R.E.N.A.L. [30] and PADUA [13] have been developed which synthesize various manually-extracted radiomic features to produce scores which have been shown to robustly correlate with perioperative and oncologic outcomes.”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“One of the models in the final ensemble used by this method was initialized with weights of a model pretrained on the publicly available Liver Tumor Segmentation (LiTS) dataset [5].”

“Existing health care disparities are well-documented [3, 10, 11], and there is a significant concern that the proliferation of predictive models in medicine will serve to exacerbate these disparities [9].”

“In surgical oncology, a number of “nephrometry” scores such as R.E.N.A.L. [30] and PADUA [13] have been developed which synthesize various manually-extracted radiomic features to produce scores which have been shown to robustly correlate with perioperative and oncologic outcomes.”

“This imbalance is consistent between the training and test sets, and is, in fact, a well established phenomenon in the epidemiology of renal cell carcinoma [7].”

“With the utilization of cross-sectional imaging now as high as it’s ever been, kidney tumors are now most often discovered incidentally, rather than on the basis of symptoms [37, 8].”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“Existing health care disparities are well-documented [3, 10, 11], and there is a significant concern that the proliferation of predictive models in medicine will serve to exacerbate these disparities [9].”

“Modern deep learning approaches have achieved impressive performance on a wide variety of semantic segmentation tasks [18], but their need for large and high-quality training datasets has hindered their development in problems such as kidney cancer for which little annotated data is publicly…”

“The challenge described in this report, “The 2021 Kidney Tumor Segmentation Challenge” or “KiTS21” is the second challenge in the “KiTS” series after its 2019 iteration, “KiTS19” [21].”

“Which teams used which of these methods is presented in tabular form in figure 9.”

“…and it has served as an invaluable resource to the protein structure prediction community over the last 30 years as it has progressed through several generations of computational biochemistry [29], with the latest being dominated by deep learning methods such as DeepMind’s ”AlphaFold” [26].”

“One of the first machine learning competitions to use this format was the Critical Assessment of protein Structure Prediction or “CASP” competition which has been followed by a sequel CASP event every even-numbered year since 1994 [39].”

“There is growing evidence that large numbers of renal tumors are either benign or indolent, especially when they are small and incidentally discovered, and they might therefore be best managed with surveillance rather than intervention [42, 50, 25].”

“With the utilization of cross-sectional imaging now as high as it’s ever been, kidney tumors are now most often discovered incidentally, rather than on the basis of symptoms [37, 8].”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“Many excellent challenges have been organized in the medical image analysis community over the last two decades, and the semantic segmentation of cross-sectional images is one of the most popular subjects [36].”

“It is therefore generally recommended that data from a separate institution should be used for external validation [6].”

“Existing health care disparities are well-documented [3, 10, 11], and there is a significant concern that the proliferation of predictive models in medicine will serve to exacerbate these disparities [9].”

“Best viewed in color. radiomic features have shown exciting potential for predicting histologic subtype [4], nuclear grade [40], somatic tumor mutations [27], and even cancer-specific and overall survival [17].”

“To address this, the Sørensen-Dice metric was supplemented with the Surface Dice metric, as described in [41].”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“There is growing evidence that large numbers of renal tumors are either benign or indolent, especially when they are small and incidentally discovered, and they might therefore be best managed with surveillance rather than intervention [42, 50, 25].”

“Best viewed in color. radiomic features have shown exciting potential for predicting histologic subtype [4], nuclear grade [40], somatic tumor mutations [27], and even cancer-specific and overall survival [17].”

“There is growing evidence that large numbers of renal tumors are either benign or indolent, especially when they are small and incidentally discovered, and they might therefore be best managed with surveillance rather than intervention [42, 50, 25].”

“…that renal mass biopsy has the potential to resolve this treatment decision dilemma, but others argue that its relative lack of sensitivity hinders its ability to convince physicians and patients that their disease won’t progress [48], and ultimately, its utilization remains relatively low [32].”

“There has recently been some discussion about the need for greater clarity about how annotations are produced for medical image analysis research [43].”

“Generated with the ChallengeR package [51].”

“…that renal mass biopsy has the potential to resolve this treatment decision dilemma, but others argue that its relative lack of sensitivity hinders its ability to convince physicians and patients that their disease won’t progress [48], and ultimately, its utilization remains relatively low [32].”

“However, metastatic renal cancer remains highly lethal, so the rare instances in which a small and/or incidentally-discovered renal tumor progresses to metastatic disease are unacceptable, and their risk must be weighed against overtreatment and its associated cost and morbidity [46].”

“These segmentation challenges have asked participants to segment things such as specific anatomical structures like bones and organs [33, 19, 28, 47, 35], organs at risk in radiation therapy planning [31], and, like KiTS, lesions and the organs they affect [1, 5, 2].”

“Best viewed in color. radiomic features have shown exciting potential for predicting histologic subtype [4], nuclear grade [40], somatic tumor mutations [27], and even cancer-specific and overall survival [17].”

“At the top, a podium plot [12] shows the aggregate metric values for each team for each bootstrap sample of the test set, with lines connecting each sample iteration.”

“As one might imagine, manually delineating each region of interest is a very time consuming activity, and is subject to significant interobserver variability [20] to which, ra-diomics algorithms are sometimes quite sensitive [34].”

“As one might imagine, manually delineating each region of interest is a very time consuming activity, and is subject to significant interobserver variability [20] to which, ra-diomics algorithms are sometimes quite sensitive [34].”

“Best viewed in color. radiomic features have shown exciting potential for predicting histologic subtype [4], nuclear grade [40], somatic tumor mutations [27], and even cancer-specific and overall survival [17].”

“However, metastatic renal cancer remains highly lethal, so the rare instances in which a small and/or incidentallydiscovered renal tumor progresses to metastatic disease are unacceptable, and their risk must be weighed against overtreatment and its associated cost and morbidity [46].”

“This submission [52] was made by Zhongchen Zhao, Huai Chen, and Lisheng Wang from the Shanghai Jiao Tong University, China.”

“Machine learning competitions, or “challenges” as they are often called, have become a mainstay in the medical image analysis research community [45].”

“For more details please refer to our paper [14].”

“Of these, the R.E.N.A.L. score has recently been approximated in terms of segmentation-based radiomic features, and was shown to be noninferior to human-derived scores at predicting clinical outcomes [22], and the others are sure to follow in short order.”

“This submission [16] was made by Alex Golts, Daniel Khapun, Daniel Shats, Yoel Shoshan and Flora Gilboa-Solomon of IBM Research - Israel.”

“Following the similar trend as CNN, hybrid models are often trained and validated on open-source datasets, e.g., BraTS [32], Figshare [33], KiTS21 [34], TCIA [35], Synapse, with limited samples of high dimensional medical images, and obtained current state-of-the-art performance for all major…”

“Note that , some public datasets are not mutually exclusive, e.g., KiTS23andKiTS21 12 , we thus only collectthe latest version, to avoid redundancy and potential data leakage in the train-test split.”

“This trend was influenced by the constraints of early datasets such as MSD [1], LiTS [2], and KiTS [10, 11].”

“The results proposed in this paper show superiority to the results provided by the participant involved in KiTS23 challenge [5] only by comparing tumour dice metrics and more comprehensive evaluation (involving calculation of dice and surface dice metrics) would be included in future research.”

“To evaluate the effectiveness of scaling up datasets, we conduct ablation studies on fi ve segmentation datasets, i.e., LiTS 61 (liver tumors), MSD07 10 (pancreas tumors), KiTS23 66 , MSD06 10 (lung tumors), and CV19-20 69 (COVID-19 infection).”

“Additionally, it is worth mentioning that nnUNet models are often restricted to specific tasks, tends to underperform on complex datasets, and require longer training and inference times while ONCOPILOT offers an all-in-one solution.”

“We evaluated our approach on the KiTS23 dataset [13], demonstrating its effectiveness in identifying ambiguous regions and out-of-distribution data.”

“…reward R total within each group to capture the relative quality of the rollouts O i : Land-man et al., 2015; Ji et al., 2022; Ma et al., 2022; Heller et al., 2023; Armato III et al., 2011), MRI (Bernard et al., 2018; Ji et al., 2022; Buda et al., 2019), X-Ray (Jaeger et al., 2014; Chowdhury…”

“…especially relevant for tumor segmentation, where different lesion appearances and annotation targets have been reported across tumor benchmarks [3, 8, 23], and where different cases may call for different interactions: A compact lesion may be corrected with a few clicks, an elongated lesion may…”

“We consider six fully annotated 3D medical segmentation datasets from the ULS23 Segmentation Challenge (de Grauw et al. 2025), each covering a different anatomical region or organ system: KiTS21 (Heller et al. 2023), LiTS (Bilic et al. 2023), NIH-LN ABD (Roth et al. 2014), LIDC-IDRI (Armato III et al. 2011), MDSC-Colo…”

“The Abdomen-SSL dataset was created by extracting 20 slices around the center of each volume in 4 abdominal CT datasets [23, 24, 25], cropping around the subject, resizing to 224x224 and copying the channel over, resulting in ∼ 21 , 000 whole abdomen images.”