CoChat
Sign Up
Cited by 8,421Radiomics and Machine Learning in Medical ImagingImage Retrieval and Classification TechniquesMedical Imaging Techniques and Applications

Radiomics: Images Are More than Pictures, They Are Data

Open access
Gillies, Robert J., Kinahan, Paul E., Hricak, Hedvig·2016·Radiology
DOI 10.1148/radiol.2015151169PMC PMC4734157
Open this paper in CoChat — free account

Abstract

In the past decade, the field of medical image analysis has grown exponentially, with an increased number of pattern recognition tools and an increase in data set sizes. These advances have facilitated the development of processes for high-throughput extraction of quantitative features that result in the conversion of images into mineable data and the subsequent analysis of these data for decision support; this practice is termed radiomics. This is in contrast to the traditional practice of treating medical images as pictures intended solely for visual interpretation. Radiomic data contain first-, second-, and higher-order statistics. These data are combined with other patient data and are mined with sophisticated bioinformatics tools to develop models that may potentially improve diagnostic, prognostic, and predictive accuracy. Because radiomics analyses are intended to be conducted with standard of care images, it is conceivable that conversion of digital images to mineable data will eventually become routine practice. This report describes the process of radiomics, its challenges, and its potential power to facilitate better clinical decision making, particularly in the care of patients with cancer.

AI snapshot

Written from the paper’s full text.

TL;DR

Radiomics converts digital medical images into high-dimensional, mineable quantitative data and mines those data, often with clinical and genomic covariates, to support diagnosis, prognosis, and treatment decisions. This review describes the radiomics workflow, highlights oncology examples, and outlines major challenges including reproducibility, standardization, and data sharing.

Key findings

  • Radiomics is defined as the conversion of images to higher-dimensional data and subsequent mining for decision support; it differs from CAD systems by extracting many quantitative features rather than delivering a single detection/diagnosis answer.
  • Feature extraction has expanded from 182 texture plus 22 semantic features for lung cancer CT to sets of 442, 662, and 522 features; a covariance matrix example used 219 features extracted from non–small cell lung cancer tumors in 235 patients.
  • Selected clinical examples report that Haralick texture analysis in 147 men with biopsy-proven prostate cancer differentiated benign from cancerous tissue, and follow-up work distinguished Gleason score 6 from 7 or more with 93% accuracy and 4+3 from 3+4 with 92% accuracy; in 49 prostate cancer patients, texture analysis improved AUC from 0.82–0.85 to 0.92–0.96.
  • Curation of a 143-patient non–small cell lung cancer dataset required an initial 285 candidates (approximately 50% attrition) and 430 hours of processing, or approximately 3 hours per patient; the paper also notes histopathology error rates can be as high as 23%.

Methods

  • Narrative review/report in *Radiology*; it describes the radiomics process—image acquisition, volume-of-interest identification, segmentation, feature extraction/qualification, database building, and classifier modeling—rather than presenting a primary study.
  • Evidence is drawn from selected published radiomics and radiogenomics studies, mostly in oncology, using CT, MR imaging, and PET; examples include prostate, lung, glioma, hepatocellular carcinoma, breast, and bladder cancer.
  • Main analysis is conceptual synthesis and illustrative reporting of selected quantitative results; no systematic search strategy, meta-analysis, or formal risk-of-bias assessment is described.

Limitations

  • Stated in the paper: reproducibility problems, overfitting, poor study design (especially mixing hypothesis generation with hypothesis testing), lack of validation standards, incomplete reporting, and unrecognized confounding variables, particularly in retrospective databases; independent validation and prospective multicenter trials are needed.
  • Stated in the paper: segmentation is the most critical, challenging, and contentious component, with high interoperator variability in manual contours; data sharing faces cultural, administrative, regulatory, personal, and infrastructure barriers.
  • Evident from the text: as a narrative review, the cited examples are selected rather than systematically identified, and the paper does not report a formal review protocol or quantitative synthesis.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1148/radiol.2015151169

Cites

75 of 75 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 80 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    When images with different acquisition parameters are combined, the combinations reside in spatially explicit regions of the tumors.

    “Notably, when this is performed, the combinations reside in spatially explicit regions of the tumors (Fig 2).”
    Quote in the paperSubstantial claim
  • Premisecitation

    Genomic heterogeneity within tumors and across metastatic tumor sites in the same patient is the major cause of treatment failure and emergence of therapy resistance.

    “Genomic heterogeneity within tumors and across metastatic tumor sites in the same patient is the major cause of treatment failure and emergence of therapy resistance (19).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [19] source

      Not checked in the free run.

  • Premisecitation

    It is estimated that the error rate of cancer histopathology can be as high as 23%.

    “This is important, as it is estimated that the error rate of cancer histopathology can be as high as 23% (20–23).”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [20] source

      Not checked in the free run.

    • cites [21] source

      Not checked in the free run.

    • cites [22] source

      Not checked in the free run.

    • cites [23] source

      Not checked in the free run.

  • Interpretationreasoning

    Correlation of radiomic data with genomic or other -omic data could provide confirmatory information to support histopathologic findings.

    “It also could provide confirmatory information to support histopathologic findings.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    The combination of this subset of radiomic features with genomic data may increase diagnostic, prognostic, and predictive power.

    “The combination of this subset of radiomic features with genomic data may increase diagnostic, prognostic, and predictive power.”
    Quote in the paperSubstantial claim
  • May combine two claims. 48
  • Classified as probably not a substantial claim. 63

  • May combine two claims.

    Findingreasoning

    Curation of a data set of 143 patients required an initial cohort of 285 patients (approximately 50% attrition) and required 430 hours of processing, or approximately 3 hours of processing per patient.

    “Thus, the curation of a data set of 143 patients required an initial cohort of 285 patients (approximately 50% attrition) and required 430 hours of processing, or approximately 3 hours of processing per patient.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Figure 3 is a covariance matrix of 219 features extracted from CT scans in 143 patients with non-small cell lung cancer.

    “Figure 3 is a covariance matrix of 219 features extracted from CT scans in 143 patients with non–small cell lung cancer.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Research has shown the capacity of radiomics analyses to help distinguish prostate cancer from benign prostate tissue or add information about prostate cancer aggressiveness.

    “For example, research has already shown the capacity of radiomics analyses to help distinguish prostate cancer from benign prostate tissue or add information about prostate cancer aggressiveness (6).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    A subset of radiomic features is not significantly related to gene expression or mutational data and hence has the potential to provide additional, independent information.

    “Second, a subset of radiomic features is not significantly related to gene expression or mutational data and, hence, has the potential to provide additional, independent information.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Precision medicine requires not only in vitro biomarkers and companion diagnostics but also spatially and temporally resolved in vivo biomarkers of tumor biology.

    “Thus, precision medicine requires not only in vitro biomarkers and companion diagnostics but also spatially and temporally resolved in vivo biomarkers of tumor biology.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CAD systems are usually standalone systems designated by the Food and Drug Administration for use in either the detection or diagnosis of disease.

    “CADcomputer-aided diagnosis and detection systems are usually standalone systems that are designated by the Food and Drug Administration for use in either the detection or diagnosis of disease (1).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Institute of Medicine I, 2012

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisereasoning

    Unlike CAD, radiomics is explicitly a process designed to extract a large number of quantitative features from digital images, place these data in shared databases, and subsequently mine the data for hypothesis generation, testing, or both.

    “Unlike CADcomputer-aided diagnosis and detection systems, which are directed toward delivering a single answer (ie, presence of a lesion or cancer), radiomics is explicitly a process designed to extract a large number of quantitative features from digital images, place these data in shared databases, and subsequently mine the data for hypothesis generation, testing, or both.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Correlation of radiomic data with genomic or other -omic data could inform not only the decision about whether to test for certain gene alterations in biopsy samples but also the choice of biopsy sites.

    “However, correlation of radiomic data with genomic or other -omic data could inform not only the decision about whether to test for certain gene alterations in biopsy samples but also the choice of biopsy sites.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Quantitative image features can be correlated with clinical outcomes data and used for evidence-based clinical decision support.

    “These features, in conjunction with the other information, can be correlated with clinical outcomes data and used for evidence-based clinical decision support (Fig 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Radiomics is the conversion of digital medical images into mineable high-dimensional data.

    “The conversion of digital medical images into mineable high-dimensional data, a process that is known as radiomics, is motivated by the concept that biomedical images contain information that reflects underlying pathophysiology and that these relationships can be revealed via quantitative image analyses.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Biomedical images contain information that reflects underlying pathophysiology and these relationships can be revealed via quantitative image analyses.

  • May combine two claims.

    Interpretationreasoning

    Radiomics can provide important information regarding the sample genomics and can be used for cross-validation.

    “Thus, radiomics can provide important information regarding the sample genomics and can be used for cross-validation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    In targeted therapies, most responses are not durable and benefit is generally measured in months, not years.

    “In this emerging era of targeted therapies, it is notable that most responses are not durable and that benefit is generally measured in months, not years.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [16] source

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

    • cites [18] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisereasoning

    Radiomics is designed to develop decision support tools and involves combining radiomic data with other patient characteristics to increase the power of decision support models.

    “Radiomics is designed to develop decision support tools; therefore, it involves combining radiomic data with other patient characteristics, as available, to increase the power of the decision support models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Most clinically relevant solid tumors are highly heterogeneous at the phenotypic, physiologic, and genomic levels and continue to evolve over time.

    “It is now appreciated that most clinically relevant solid tumors are highly heterogeneous at the phenotypic, physiologic, and genomic levels (13–15) and that they continue to evolve over time.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [13] source

      Not checked in the free run.

    • cites [14] source

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Early successes of CAD have been greatest in breast cancer imaging.

    “Early successes of CADcomputer-aided diagnosis and detection have been greatest in breast cancer imaging (2,3).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [2] source

      Not checked in the free run.

    • cites [3] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Quantitative image features based on intensity, shape, size or volume, and texture offer information on tumor phenotype and microenvironment that is distinct from that provided by clinical reports, laboratory test results, and genomic or proteomic assays.

    “Quantitative image features based on intensity, shape, size or volume, and texture offer information on tumor phenotype and microenvironment (or habitat) that is distinct from that provided by clinical reports, laboratory test results, and genomic or proteomic assays.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Radiomic and radiogenomic analyses can be used to identify correlations, not causes; thus, they are not expected to enable definitive assessment of genetic or other contents of tissue through imaging alone.

    “It should be emphasized that radiomic and radiogenomic analyses can be used to identify correlations, not causes; thus, they are not expected to enable definitive assessment of genetic or other contents of tissue through imaging alone.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    Habitats change after treatment, and the pattern of change has been observed to be predictive of response.

    “Furthermore, these habitats change after treatment (eg, treatment with radiation and temazolamide), and the pattern of change has been observed to be predictive of response.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    Attempting to filter the data at input would be inefficient and would presuppose knowledge regarding the value of the features in classifier models before they were tested.

    “This is driven by the knowledge that attempting to filter the data at input would be inefficient and would presuppose knowledge regarding the value of the features in classifier models before they were tested.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    There is a movement to capture semantic data with the aid of computers to achieve higher interreader agreement, faster throughput, and lower variance.

    “This approach continues to have high value, and there is a movement to capture such semantic data with the aid of computers to achieve higher interreader agreement, faster throughput, and lower variance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Segmentation of normal structures, such as skeletal elements and organs, can now be achieved with full automation.

    “Segmentation of normal structures, such as skeletal elements and organs, can now be achieved with full automation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    Identification of tissue volumes of prognostic value is the core of the practice of radiology in oncology.

    “Identification of tissue volumes of prognostic value is the core of the practice of radiology in oncology.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Robust models accommodate patient features beyond imaging.

    “Ideally, robust models accommodate patient features beyond imaging.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The most systematic approach is to first identify features that may be redundant, that is, those that are highly correlated with one another.

    “The most systematic approach is to first identify features that may be redundant (ie, those that are highly correlated with one another).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationown data

    In the future, capturing images and other data prospectively and with higher quality and standards should reduce data attrition and make the process more efficient.

    “In the future, capturing images and other data prospectively and with higher quality and standards should reduce data attrition and make the process more efficient.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    More recently, feature sets have been expanded to 662 features that also contain Laplace transforms of Gaussian fits and 522 features that include texture and fractal dimension features.

    “More recently, this has been expanded to 662 features that also contain Laplace transforms of Gaussian fits (46) and 522 features that include texture and fractal dimension features (48).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [46] Grossmann P, Sci Transl Med

      Not checked in the free run.

    • cites [48] Pickup L, 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    There is a pressing need to capture such data and to share data across institutions to accumulate sufficient numbers for statistical power.

    “Hence, there is a pressing need to capture such data and to share data across institutions to accumulate sufficient numbers for statistical power.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    Supervised and nonsupervised machine learning approaches use a priori knowledge through training sets but are agnostic in that they make no assumptions about the meaning of the individual features.

    “Although these approaches use a priori knowledge through training sets, they are agnostic in that they make no assumptions about the meaning of the individual features.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The best models are those that can accommodate additional clinical or genomic covariates, and this increases the need for large high-quality data sets.

    “Furthermore, the best models are those that can accommodate additional clinical or genomic covariates, and this increases the need for large high-quality data sets.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    Any disease, especially cancer, requires operator input because of inter- and intrasubject morphologic and contrast heterogeneity at the initial examination.

    “However, any disease, especially cancer, requires operator input because of inter- and intrasubject morphologic and contrast heterogeneity at the initial examination.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Any disease, especially cancer, requires operator input because of inter- and intrasubject morphologic and contrast heterogeneity at the initial examination.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Semantic and agnostic features are the two types of features extracted in radiomics.

    “In practice, “semantic” and “agnostic” features are the two types of features extracted in radiomics.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Clusters of highly correlated features can be collapsed into one representative feature, usually the one with the largest intersubject variability or highest dynamic range.

    “Clusters of highly correlated features can be collapsed into one representative feature, usually the one with the largest intersubject variability or highest dynamic range.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The heart of radiomics is the extraction of high-dimension feature data to quantitatively describe attributes of volumes of interest.

    “The heart of radiomics is the extraction of high-dimension feature data to quantitatively describe attributes of volumes of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A further level of prioritization described by Aerts et al is to rank order features within separate categories representing different agnostic and semantic classes of features.

    “A further level of prioritization, described by Aerts et al (47), is to rank order features within separate categories representing different agnostic and semantic classes of features (eg, size, shape, and first-, second-, and higher-order textures).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Aerts HJ, 2014, Nat Commun source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Overall survival is a common endpoint for many studies, but this includes death from all causes, which may not be related to the disease being studied.

    “For example, overall survival is a common endpoint for many studies, but this includes death from all causes, which may not be related to the disease being studied.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Overall survival includes death from all causes, which may not be related to the disease being studied.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    A reasonable rule of thumb is that 10 samples (patients) are needed for each feature in a model based on binary classifiers.

    “It has been our experience that a reasonable rule of thumb is that 10 samples (patients) are needed for each feature in a model based on binary classifiers.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: A reasonable rule of thumb is that 10 samples (patients) are needed for each feature in a model based on binary classifiers.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Segmentation is challenging because many tumors have indistinct borders.

    “It is challenging because many tumors have indistinct borders.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The power of the model is entirely dependent on the size and quality of the data within the database.

    “As mentioned previously, the power of the model is entirely dependent on the size and quality of the data within the database.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    More exact endpoints include progression-free survival and disease-free survival or recurrence; however, these data are not readily available and require a dedicated abstraction effort with chart review.

    “More exact endpoints include progression-free survival and disease-free survival or recurrence; however, these data are not readily available and require a dedicated abstraction effort with chart review.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Texture analysis of images was first introduced in 1973 by Haralick et al.

    “Texture analysis of images was first introduced in 1973 by Haralick et al (41).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    It is not sufficient to report a number or a set of numbers derived from images; instead, error bars must also be provided, as is done with every other credible laboratory measurement.

    “In other words, it is not sufficient to report a number or a set of numbers derived from images; instead, we must also be able to provide error bars, as is done with every other credible laboratory measurement.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Modern CT, MR imaging, and combined PET/CT units allow for wide variations in acquisition and image reconstruction protocols, and standardization of these protocols across medical imaging centers is typically lacking.
    • Rests on an assumption: Variations in acquisition and image reconstruction parameters can introduce changes that are not due to underlying biologic effects.

  • May combine two claims.

    Interpretationreasoning

    With such large complexity, there is a danger of overfitting analyses, and hence, dimensionality must be reduced by prioritizing the features.

    “With such large complexity, there is a danger of overfitting analyses, and hence, dimensionality must be reduced by prioritizing the features (49,50).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [49]

      Not checked in the free run.

    • cites [50] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    The basic philosophy of radiomics is to capture as much data as possible at the front end and use downstream database mining to identify the features with the highest prognostic value.

    “The basic philosophy, which has its foundation in process engineering, is to capture as much data as possible at the front end and use downstream database mining to identify the features with the highest prognostic value.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    In the final analysis, the value of feature sets is determined by their contribution to classifier models created through database mining.

    “In the final analysis, the value of feature sets is determined by their contribution to classifier models created through database mining.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The best models are those that are tailored to a specific medical context and, hence, start out with a well-defined endpoint.

    “While both of these approaches have merit, the best models are those that are tailored to a specific medical context and, hence, start out with a well-defined endpoint.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Errors in histopathology are due to both sampling errors and observer variability; thus, there is a great need for additional quantitative diagnostic information.

    “Errors in histopathology are due to both sampling errors and observer variability; thus, there is a great need for additional quantitative diagnostic information.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning

    Radiomics can be performed with as few as 100 patients, although larger data sets provide more power.

    “Radiomics can be performed with as few as 100 patients, although larger data sets provide more power.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Radiomics can be performed with as few as 100 patients.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    This was followed by a 442-member feature set that also contained wavelets.

    “This was followed by a 442-member feature set that also contained wavelets (47).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Aerts HJ, 2014, Nat Commun source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductreasoning

    As the patients in the curation example were not filtered for medical or demographic issues, there was no selection bias.

    “As these patients were not filtered for medical or demographic issues, there was no selection bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    In practice, not all information is available for all patients; hence, models should also be designed to accommodate sparse data.

    “In practice, not all information is available for all patients; hence, models should also be designed to accommodate sparse data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Contrast enhancement at MR imaging is often heterogeneous, with complex patterns.

    “It is well known in the radiology community that contrast enhancement at MR imaging is often heterogeneous, with complex patterns.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Data sharing is a major initiative of the QIN, whose members are committed to depositing well-curated data sets into The Cancer Imaging Archive for public and private data mining efforts.

    “Such data sharing is a major initiative of the QINQuantitative Imaging Network, whose members are committed to depositing well-curated data sets into The Cancer Imaging Archive for public and private data mining efforts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The Vignati et al study may have been underpowered.

    “Although this study may have been underpowered”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In the first of these studies, combinations of 28 imaging traits could be used to reconstruct 78% of the global gene expression profiles, which in turn were linked to cell proliferation, liver synthetic function, and patient prognosis.

    “They found that combinations of 28 imaging traits could be used to reconstruct 78% of the global gene expression profiles, which in turn were linked to cell proliferation, liver synthetic function, and patient prognosis (40).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In the peripheral zone of the prostate, all five features assessed (entropy, inertia, energy, correlation, and homogeneity) differed significantly between benign and cancerous tissue on both types of images; in the transition zone, significant differences were found for all five features on ADC maps and for two features (inertia and correlation) on T2-weighted images.

    “In the peripheral zone of the prostate, all five features assessed (entropy, inertia, energy, correlation, and homogeneity) differed significantly between benign and cancerous tissue on both types of images; however, in the transition zone, significant differences were found for all five features on ADCapparent diffusion coefficient maps and for two features (inertia and correlation) on T2-weight”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In Gevaert et al, the feature set was filtered for reproducibility, yielding 18 features assessed in three distinct habitats.

    “The feature set was then filtered for reproducibility, yielding 18 features that were assessed in three distinct habitats.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In a follow-up study, these features were used to automatically compute Gleason grade and were found to enable discrimination between cancers with a Gleason score of 6 (3+3) and those with a Gleason score of 7 or more with 93% accuracy.

    “In a follow-up study, these features were used to automatically compute Gleason grade and were found to enable discrimination between cancers with a Gleason score of 6 (3+3) and those with a Gleason score of 7 of more with 93% accuracy.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [53] Fehr D, Proc Natl Acad Sci USA

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Testing of the core technology of texture analysis is among the areas in which standards are lacking.

    “Testing of the core technology of texture analysis also is among the areas in which standards are lacking.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    No guidelines yet exist for the reporting of quantitative imaging results, let alone for the reporting of much more complex radiomics results.

    “While there are guidelines for reporting, none yet exist for the reporting of quantitative imaging results, let alone for the reporting of much more complex radiomics results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The most rigorous biomarker qualification requires a prospective multicenter trial wherein the biomarker is one of the primary endpoints.

    “Furthermore, the most rigorous biomarker qualification requires a prospective multicenter trial wherein the biomarker is one of the primary endpoints (62,63).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [62] source

      Not checked in the free run.

    • cites [63] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The standards for statistical testing and raw data access have been adopted by the National Institutes of Health.

    “The standards have been adopted by the National Institutes of Health (8,70).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [8]

      Not checked in the free run.

    • cites [70] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    Collection of high-quality image data requires sophisticated content expertise to identify and circumscribe (with computer assistance) and annotate (with a standardized and mineable lexicon) the volumes of interest.

    “Collection of high-quality image data requires sophisticated content expertise to identity and circumscribe (with computer assistance) and annotate (with a standardized and mineable lexicon) the volumes of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Radiomics has been initiated in oncology studies, but it is potentially applicable to all diseases.

    “Radiomics has been initiated in oncology studies, but it is potentially applicable to all diseases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Although relationships between radiomics and outcome are defined with populations, they can be applied to individual patients.

    “Although relationships between radiomics and outcome are defined with populations, they can be applied to individual patients.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Communities like the Children's Oncology Group have established a history and culture of data sharing and therefore are in a prime position to expand their efforts to include radiomic image analysis.

    “Notably, communities like the Children’s Oncology Group have established a history and culture of data sharing (73) and therefore are in a prime position to expand their efforts to include radiomic image analysis.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Many published test-retest subject or patient image studies have small sample sizes.

    “For example, while the value of test-retest subject or patient image studies is well recognized, many of the published studies have small sample sizes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Radiomic analysis promises to increase precision in diagnosis, assessment of prognosis, and prediction of therapy response.

    “Radiomic analysis promises to increase precision in diagnosis, assessment of prognosis, and prediction of therapy response.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Once images are archived, they are rarely reaccessed.

    “Furthermore, once images are archived, they are rarely reaccessed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisereasoning

    Multisite trials are required to interrogate separate cohorts of patients and to create databases with sufficient size for statistical power.

    “Multisite trials are required to interrogate separate cohorts of patients and to create databases with sufficient size for statistical power.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Most large medical centers are now investing in appropriate electronic medical record systems to make patient data accessible in a mineable form.

    “This is well recognized, and most large medical centers are now investing in appropriate electronic medical record systems to make patient data accessible in a mineable form.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    They are beginning to be solved through sharing of oncology metadata and some sharing of image files has been enabled by The Cancer Imaging Archive.

    “They are beginning to be solved through sharing of oncology metadata (www.oriencancer.org or the Children’s Oncology Group), and some sharing of image files has been enabled by The Cancer Imaging Archive (77).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [77] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Radiomics is a new field, and there are substantial challenges to its implementation in a clinical setting.

    “Radiomics is a new field, and there are substantial challenges to its implementation in a clinical setting.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Such capabilities are nearly at hand, as most picture archiving and communication systems have the capability to coregister current images with prior images and perform user-interactive segmentation.

    “Such capabilities are nearly at hand, as most picture archiving and communication systems have the capability to coregister current images with prior images and perform user-interactive segmentation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    These challenges of data sharing have been met and solved in the basic sciences of gene expression, sequencing, and protein structure databases.

    “These challenges have been met and solved in the basic sciences of gene expression, sequencing, and protein structure databases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In current general practice, radiologic examinations are qualitatively evaluated, and the generated reports often do not use a standard lexicon, despite efforts to embrace a uniform lexicon such as RadLex.

    “In current general practice, radiologic examinations are qualitatively evaluated, and the generated reports often do not use a standard lexicon, despite a number of efforts to embrace a uniform lexicon, such as RadLex® (74).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [74] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Superb reporting guidelines for clinical studies have been developed by many organizations, with an excellent overview provided by the Equator network.

    “Superb reporting guidelines for clinical studies have been developed by many organizations. An excellent overview is provided by the Equator network, which promotes the quality and transparency of health research (65).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Data sharing is a common challenge in all biomedical research, and it must overcome cultural, administrative, regulatory, and personal issues.

    “Data sharing is a common challenge in all biomedical research, and it must overcome cultural, administrative, regulatory, and personal issues (72).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [72] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Although massive image repositories exist, they are virtually inaccessible for curation because of the limitations described previously.

    “Although massive image repositories exist, they are virtually inaccessible for curation because of the limitations described previously.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Radiomics can be performed with tomographic images from CT, MR imaging, and PET studies.

    “Radiomics can be performed with tomographic images from CT, MR imaging, and PET studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    Differences in image acquisition and reconstruction are covariates that must be incorporated in the mining of quantitative image data and therefore increase the amount of data required for model building by an order of magnitude.

    “Differences in image acquisition and reconstruction are covariates that must be incorporated in the mining of quantitative image data and therefore increase the amount of data required for model building by an order of magnitude.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Differences in image acquisition and reconstruction are covariates that must be incorporated in mining quantitative image data.

  • Classified as probably not a substantial claim.

    Premisereasoning

    While standards exist or are being developed in many areas, there are still gaps.

    “While standards exist or are being developed in many of the areas already mentioned in this article, there are still gaps.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Currently, radiomic data are typically not incorporated as part of the electronic medical record data stream; however, this is changing with the adoption of structured radiology reporting.

    “Currently, radiomic data are typically not incorporated as part of this data stream; however, this is changing with the adoption of structured radiology reporting.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Editors from more than 30 high-impact-factor biomedical journals have united to impose common standards for statistical testing and to improve access to raw data.

    “These issues have become serious enough that editors from more than 30 high-impact-factor biomedical journals have united to impose common standards for statistical testing and to improve access to raw data (68,69).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [68] 2014, Nature

      Not checked in the free run.

    • cites [69] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Kuo et al's results suggested that radiogenomic analyses could be used to guide the selection of therapy for individual tumors.

    “Their results suggested that radiogenomic analyses could be used to guide the selection of therapy for individual tumors.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [55] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Data sharing in radiomics is especially daunting because shared data must include images and sharing must be in compliance with HIPAA, as a substantial amount of personal health information is needed to build models of sufficient complexity.

    “Data sharing in radiomics is especially daunting because shared data must include images and sharing must be in compliance with the Health Insurance Portability and Accountability Act, as a substantial amount of personal health information is needed to build models of sufficient complexity.”
    Quote in the paperSubstantial claim
85 of 85

  • The authors state this as an assumption.

    Biomedical images contain information that reflects underlying pathophysiology and these relationships can be revealed via quantitative image analyses.

    “The conversion of digital medical images into mineable high-dimensional data, a process that is known as radiomics, is motivated by the concept that biomedical images contain information that reflects underlying pathophysiology and that these relationships can be revealed via quantitative image analyses.”

  • The paper relies on this without support.

    Virtually all patients with cancer undergo imaging at some point and often multiple times during their care.

    “The value of radiogenomics stems from the fact that while virtually all patients with cancer undergo imaging at some point and often multiple times during their care, not all of them have their disease genomically profiled.”

  • The paper relies on this without support.

    Not all patients with cancer have their disease genomically profiled.

    “The value of radiogenomics stems from the fact that while virtually all patients with cancer undergo imaging at some point and often multiple times during their care, not all of them have their disease genomically profiled.”

  • The paper relies on this without support.

    When genomic profiling is performed, it is done one time at one location and is susceptible to sampling error.

    “Furthermore, when genomic profiling is performed, it is done one time at one location and is susceptible to sampling error.”

  • The paper relies on this without support.

    Digital radiologic images are obtained for almost every patient with cancer.

    “A major strength of a radiomics approach for cancer is that digital radiologic images are obtained for almost every patient with cancer, and all of these images are potential sources for radiomics databases (Table 1).”

  • The paper relies on this without support.

    Most patients with cancer will undergo multiple CT, MR imaging, and PET examinations.

    “Most of these patients will undergo multiple CT, MR imaging, and PET examinations.”

  • The paper relies on this without support.

    Modern CT, MR imaging, and combined PET/CT units allow for wide variations in acquisition and image reconstruction protocols, and standardization of these protocols across medical imaging centers is typically lacking.

    “Modern CT, MR imaging, and combined PET/CT units allow for wide variations in acquisition and image reconstruction protocols, and standardization of these protocols across medical imaging centers is typically lacking.”

  • The paper relies on this without support.

    Variations in acquisition and image reconstruction parameters can introduce changes that are not due to underlying biologic effects.

    “However, when images are analyzed numerically to extract meaningful data, variations in acquisition and image reconstruction parameters can introduce changes that are not due to underlying biologic effects.”

  • The paper relies on this without support.

    A consensus is emerging that truth is elusive and that optimum reproducible segmentation is achievable with computer-aided edge detection followed by manual curation.

    “However, a consensus is emerging that truth is elusive and that optimum reproducible segmentation is achievable with computer-aided edge detection followed by manual curation.”

  • The paper relies on this without support.

    Any disease, especially cancer, requires operator input because of inter- and intrasubject morphologic and contrast heterogeneity at the initial examination.

    “However, any disease, especially cancer, requires operator input because of inter- and intrasubject morphologic and contrast heterogeneity at the initial examination.”

  • The paper relies on this without support.

    The power of the predictive classifier model is dependent on having sufficient data.

    “In radiomics and elsewhere, the power of the predictive classifier model is dependent on having sufficient data.”

  • The paper relies on this without support.

    A reasonable rule of thumb is that 10 samples (patients) are needed for each feature in a model based on binary classifiers.

    “It has been our experience that a reasonable rule of thumb is that 10 samples (patients) are needed for each feature in a model based on binary classifiers.”

  • The paper relies on this without support.

    Radiomics can be performed with as few as 100 patients.

    “Radiomics can be performed with as few as 100 patients, although larger data sets provide more power.”

  • The paper relies on this without support.

    Overall survival includes death from all causes, which may not be related to the disease being studied.

    “For example, overall survival is a common endpoint for many studies, but this includes death from all causes, which may not be related to the disease being studied.”

  • The paper relies on this without support.

    The slow progress of molecular biology–based systemic diagnostic techniques and therapies is attributable to technical complexity, poor study design, overfitting, lack of standards, incomplete reporting, and unrecognized confounding.

    “That slow progress can be attributed to a number of causes, including technical complexity, poor study design (in particular, mixing hypothesis generation with hypothesis testing) and overfitting of data, lack of standards for validating results, incomplete reporting of results, and unrecognized confounding variables in the databases used, particularly if data are derived retrospectively.”

  • The paper relies on this without support.

    The quality of classifier models is limited by the size of the data sets used to create them.

    “As discussed previously, the quality of classifier models is limited by the size of the data sets used to create them.”

  • The paper relies on this without support.

    Differences in image acquisition and reconstruction are covariates that must be incorporated in mining quantitative image data.

    “Differences in image acquisition and reconstruction are covariates that must be incorporated in the mining of quantitative image data and therefore increase the amount of data required for model building by an order of magnitude.”
17 of 17

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In oncology, imaging information can indicate histological type of tumor, staging, presence of mutations, chance of treatment response, risk of recurrence, and expected survival.

    “Those changes have been especially prominent in oncology, showing that the information provided by imaging studies can go far beyond determining whether a lesion is benign or malignant, now being able to indicate the histological type of the tumor, staging, presence of mutations, chance of treatment response, risk of recurrence, and expected survival(7-9).”

    From Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine (2019) · cites this paper as [7]

  • Radiomics associates quantitative characteristics of images with patient data and clinical outcomes, allowing diagnosis and providing information regarding prognosis and treatment response.

    “Radiomics has been described as an extension of CAD that associates the quantitative characteristics (features) of images with patient data and clinical outcomes, not only allowing the diagnosis to be made but also providing information regarding the prognosis and treatment response(7,25).”

    From Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine (2019) · cites this paper as [7]

  • Many studies using radiomics in different types of imaging studies to evaluate different diseases exist; most are related to oncology, such as lung and kidney neoplasms on CT and PET/CT, and prostate cancer, breast cancer, glioma, and hepatocellular carcinoma on MRI.

    “In the medical literature, there are already many studies using radiomics in different types of imaging studies to evaluate different diseases. Most of those studies are related to oncology, such as the study of lung and kidney neoplasms on CT and positron-emission tomography/CT scans, as well as prostate cancer, breast cancer, glioma, and hepatocellular carcinoma on MRI(7,14,36).”

    From Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine (2019) · cites this paper as [7]

  • CAD tools were initially developed to aid interpretation of radiological findings and early identification of diseases, especially breast and lung cancer.

    “Tools of CAD were initially developed to aid in the interpretation of radiological findings and early identification of diseases, especially breast and lung cancer(7,10).”

    From Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine (2019) · cites this paper as [7]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Figure 3 gives two patient counts

    Minor

    The text describes the Figure 3 matrix as based on 143 patients, while the figure caption says the features were extracted from tumors in 235 patients.

1 of 1

Cites

Flagged first, then by citations

1 of the 76 references couldn’t be matched to a known paper.

Research works printed but not linked: 7

  1. Office of the Press Secretary, The White House Web site
  2. Shankar LK, 2006, J Nucl Med, 47, 1059
  3. Kinahan P, 2014, J Nucl Med, 55, 1520
  4. Grossmann P, Sci Transl Med
  5. Pickup L, 2015, Radiological Society of North America Scientific Assembly and Annual Meeting Program, SSM06
  6. Fehr D, Proc Natl Acad Sci USA
  7. 2014, Nature, 515, 7

Cited by

Sorted by citations

Related

Authors
Gillies, R. J., Kinahan, P. E., & Hricak, H.
Affiliations
Memorial Sloan Kettering Cancer Center (US)
Corresponding author
Gillies, Robert J.
ORCID
Venue
Radiology
Publisher
Radiological Society of North America (RSNA)
Published
2016-02
Volume
278
Issue
2
Pages
563-577
ISSN
  • 0033-8419 (ISSN-L)
  • 1527-1315
DOI
10.1148/radiol.2015151169
PMC
PMC4734157
OpenAlex
W2174661749
Semantic Scholar
dcd99d49af33bd14e9e0750bcf854e7b306c808a
Cited by
8,421
References
76
Citations by year
2026: 916 · 2025: 1020 · 2024: 1098 · 2023: 1089 · 2022: 1116 · 2021: 1084 · 2020: 837 · 2019: 587 · 2018: 400 · 2017: 199 · 2016: 62 · 2015: 2 · 2013: 2
FWCI
179.12
Type
journal article
Language
en
Open access
Locations
  • Radiology (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Cancer Institute (R01CA187532, R01CA190105, U54CA143970, U01CA143062, P30CA008748)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
Keywords
radiomics, medical image analysis, quantitative imaging features, higher-order statistics, clinical decision support, cancer diagnosis, cancer prognosis, predictive modeling, multimodal data integration, bioinformatics
MeSH terms
Algorithms, Biopsy, Decision Making, Diagnosis, Computer-Assisted, Diagnostic Imaging, Humans, Image Interpretation, Computer-Assisted, Medical Informatics, Neoplasms, Prognosis, Radiology Information Systems, Genomics, Data Mining

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review