OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models
Open accessAbstract
OnSIDES database: Extracting adverse drug events from drug labels using natural language processing models
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
24 of 56 cited sources checked · 8 supported · 0 flagged · accuracy not yet measured
- Premisereasoning
No currently up-to-date, computable databases exist for on-label ADEs.
“However, as SIDER was most recently updated in 2015, no currently up-to-date, computable databases exist for on-label ADEs.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [13] source
The Introduction (Table 1; page not provided) lists SIDER 4’s release year as 2015, and the paper describes SIDER as a machine-readable ADR database. It does not establish that 2015 was SIDER’s last update or that no up-to-date, computable database for on-label ADEs currently exists. (LLM, conf=0.88)
Experimental — accuracy not yet measured
- Premisecitation
SIDER was most recently updated in 2015.
“However, as SIDER was most recently updated in 2015, no currently up-to-date, computable databases exist for on-label ADEs.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [13] source
- Interpretationcitation
This analysis recapitulates general trends in enriched occurrence of specific classes of ADEs suggested by prior relevant research efforts.
“This analysis recapitulates general trends in the enriched occurrence of specific classes of ADEs suggested by prior relevant research efforts11,27.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [11] source
- cites [27] source
The Methods and Results describe a reference set of selected drug–event pairs, including positive associations, rather than measuring how often classes of adverse drug events occur. The Discussion considers frequent and rare events but does not report enrichment of any specific event class. I searched for event-class frequency and enrichment in the indexed copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
- Premisecitation
Previous studies suggest significant international differences in ADEs listed on drug labels depending on the ADE and therapeutic area, due to differences in drug safety regulation and drug approvals.
“Previous studies have suggested that there is a significant inter-national difference in ADE listed on drug labels depending on the ADE in question and therapeutic area given the differences in drug safety regulation and drug approvals46.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [46] source
The Introduction (p. 1) and Conclusion (p. 9) discuss SHAP and interpreting model predictions, not adverse drug events on labels. Searches for international ADE-label differences and for drug safety regulation or approvals found no passage addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
In FDA SPL labels, ADE information is mainly found in the Adverse Reactions, Boxed Warnings, and Warnings and Precautions sections.
“In these labels, the ADE related information is primarily described in the “Adverse Reactions” (AR), “Boxed Warnings” (BW), and “Warnings and Precautions” (WP) sections.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [36] source
The MEDICUS EXTENSION section (page not provided) mentions FDA drug labels linked to DailyMed, but does not describe where adverse drug event information appears within those labels. Searches for the three named label sections and for adverse drug events in FDA structured product labels found no supporting passage in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
Previous efforts in constructing KidSIDES focused on extracting ADE signals from spontaneous reporter databases while controlling for developmental stages.
“Our previous efforts in constructing a database for pediatric ADEs, KidSIDES, focused on extracting ADE signals from spontaneous reporter databases while controlling for developmental stages11.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [11] source
- Findingreasoning
OnSIDES is a compiled machine-friendly database of drug-ADE pairs generated with the PubMedBERT method.
“Here, we present OnSIDES (ON-label SIDE effectS resource), a compiled, machine-friendly database of drug-ADE pairs generated with this method.”
Quote in the paperSubstantial claim - Findingreasoning
Because the method starts from the MedDRA vocabulary, it cannot identify ADEs expressed in non-standard terms.
“as we use the MedDRA vocabulary as our starting point, a key limitation to this method is that we are unable to identify ADEs that are described in non-standard terms.”
Quote in the paperNot a substantial claim - Findingown data
EU drug labels have the fewest number of unique ADEs across all drug classes.
“While we observe similar distributions in the number of ADEs across drug classes for all three data sources, we find that the EU drug labels have the fewest number of unique ADEs across all drug classes.”
Quote in the paperSubstantial claim - Findingown data
Replacing the candidate ADE term with the common token EVENT and prepending the term and section title to the input string gave the best performance for both ClinicalBERT and PubMedBERT.
“we find that substituting the term with the common token and prepending the term and section title to the input string resulted in the best performance for both the ClinicalBERT and PubMedBERT models.”
Quote in the paperSubstantial claim - Findingown data
25,453 drug labels include potential pediatric-specific ADE mentions.
“We extract 25,453 drug labels that include potential pediatric-specific ADE mentions.”
Quote in the paperSubstantial claim - Findingown data
Similar distribution of unique drugs across all SOCs was observed.
“Again, we observe a similar distribution of unique drugs across all SOCs.”
Quote in the paperNot a substantial claim - Findingreasoning
KidSIDES had more drugs in each MedDRA SOC category.
“Again, KidSIDES had more drugs in each category, highlighting the limited description and subsequent extraction of pediatric-specific ADEs in drug label text.”
Quote in the paperSubstantial claim - Findingown data
Across the models, test-set AUROC of 0.620-0.745 per ADE was achieved.
“Across these models, we achieve a test-set AUROC of 0.620–0.745 per ADE (Figure 5).”
Quote in the paperSubstantial claim- Rests on an assumption: The 10 severe ADEs selected for ChemBERT are representative of severe ADEs.
- Findingreasoning
OnSIDES-PED includes information from preclinical trial descriptions, such as ADEs from in-vivo juvenile rat experiments, that may not occur in human pediatric patients.
“it includes any information extracted from pre-clinical trial descriptions, such as ADEs identified through in-vivo experiments conducted in juvenile rats, that may not occur in a human pediatric patient.”
Quote in the paperNot a substantial claim - Findingown data
Similar distributions in the number of ADEs across drug classes were observed for all three data sources.
“While we observe similar distributions in the number of ADEs across drug classes for all three data sources, we find that the EU drug labels have the fewest number of unique ADEs across all drug classes.”
Quote in the paperNot a substantial claim - Findingown data
Exact string matching alone achieved an F1 score of 0.890 and an AUROC of 0.880 on a 100-drug-label test set using PubMedBERT.
“the exact string matching-only extraction achieved a F1 score of 0.890 and an AUROC of 0.880”
Quote in the paperSubstantial claim - Findingown data
The input string length that performed best for the BERT models was 125 words.
“We find that the 125 word input string performed best, closely followed by the 60 word input string (Supplementary Figure 2C).”
Quote in the paperSubstantial claim - Findingown data
ADEs within the SOC for social circumstances and surgical/medical procedures had the least common drug-ADE pairs across the databases.
“Generally, the ADEs within the SOC for social circumstances and surgical/medical procedures had the least common drug-ADE pairs across the databases.”
Quote in the paperNot a substantial claim - Findingnone given
OnSIDES-PED is a database of drug label-extracted ADEs specific to pediatric patients intended to be utilized alongside OnSIDES and KidSIDES.
“OnSIDES-PED is a database of drug label-extracted ADEs specific to pediatric patients intended to be utilized alongside the OnSIDES and KidSIDES databases.”
Quote in the paperSubstantial claim - Findingown data
1,026 common drugs and 3,181 side effects were identified between OnSIDES and SIDER after mapping to standardized vocabularies.
“we identified 1,026 common drugs and 3,181 side effects as mapped to standardized vocabularies.”
Quote in the paperSubstantial claim - Findingown data
PubMedBERT achieves comparatively better performance than ClinicalBERT for ADE extraction.
“We conclude that while these models achieve similar performance, the PubMedBERT model achieves comparatively better performance, and as such, we use this model to generate the OnSIDES database.”
Quote in the paperSubstantial claim - Findingreasoning
OnSIDES is a large-scale, machine-readable database of adverse drug events created by fine-tuning PubMedBERT to extract ADE terms from drug labels.
“As such, we present OnSIDES, a large-scale, machine-readable ADE database built by fine-tuning a NLP model, PubMedBERT, to accurately extract ADE terms from drug labels.”
Quote in the paperSubstantial claim - Findingown data
Some pediatric ADE descriptions consisted only of a statement like 'this medication has not been assessed for safety in pediatric patients'.
“some pediatric ADE descriptions consisted only of a statement similar to “this medication has not been assessed for safety in pediatric patients”.”
Quote in the paperNot a substantial claim - Findingown data
Prepending the drug label section title improves model performance more than the baseline input.
“we find that prepending the drug label section title most significantly improves the performance of the model compared to the baseline input”
Quote in the paperSubstantial claim - Findingreasoning
OnSIDES-INTL is restricted to currently active drug labels, which may give an incomplete view of possible ADEs.
“For the OnSIDES-INTL database, we are limited to the currently active drug labels, which may be an incomplete picture of the possible ADEs that may occur.”
Quote in the paperNot a substantial claim - Findingown data
There is a logistic relationship between both the proportion of shared drug indications and shared targets and the shared side-effect profiles of drug pairs.
“We find that there is a logistic relationship between both the proportion of shared drug indications and shared targets and the shared side-effect profiles of drug pairs (Figure 4A).”
Quote in the paperSubstantial claim- Rests on an assumption: Drug target and indication data from DrugBank, OpenTargets, and MEDI are accurate.
- Findingreasoning
OnSIDES-PED excludes general ADEs that affect pediatric patients when those ADEs are listed in the general AR section.
“this does not include any general ADEs that pediatric patients may be affected by that are listed in the general AR section, as the pediatric mentions in this section are difficult to systematically separate from those that are non-pediatric specific.”
Quote in the paperSubstantial claim - Findingown data
The fine-tuned PubMedBERT model achieves overall accuracy 0.64, precision 0.61, and AUROC 0.66 in extracting pediatric-specific ADE terms from pediatric sections.
“Here, we find that the model achieves an overall accuracy of 0.64, precision of 0.61, and AUROC of 0.66 in extracting pediatric-specific ADE terms from the aforementioned sections, suggesting the moderate flexibility of adapting this method to specific conditions.”
Quote in the paperSubstantial claim- Rests on an assumption: The manually annotated 200 drug labels for pediatric and international tasks are representative of all labels.
- Findingreasoning
KidSIDES had more ADEs in each ATC class of drugs when compared to OnSIDES/OnSIDES-PED.
“As hypothesized by the spontaneous-reporter derived nature of KidSIDES, we found that there were more ADEs in each ATC class of drugs when compared to OnSIDES/OnSIDES-PED.”
Quote in the paperSubstantial claim - Findingown data
ADEs 4 and 6 drop significantly when scaffold stratification is applied.
“In particular, ADEs 4 and 6 in particular drop significantly when scaffold stratification is applied, suggesting that their initial performance with random-split was confounded by train/test information leakage.”
Quote in the paperSubstantial claim- Rests on an assumption: The 10 severe ADEs selected for ChemBERT are representative of severe ADEs.
- Findingown data
The combined extraction method gave only a marginal performance gain over exact string matching alone.
“As there is only a marginal improvement in performance”
Quote in the paperSubstantial claim - Findingown data
Pediatric drug labels contain ADE descriptions related to delayed elimination of amoxicillin due to incompletely developed renal function in neonates and young infants.
“Of note, we found specific occurrences of ADE descriptions in the pediatric drug labels related to the delayed elimination of amoxicillin due to the incompletely developed renal function in neonates and young infants, and the increased susceptibility to suicidal ideation and behaviour for adolescents when taking antidepressants.”
Quote in the paperNot a substantial claim - Findingown data
The best-performing proportion of words placed after the candidate ADE term was 87.5%.
“Additionally, the proportion of string before and after the potential ADE term with which the model achieved peak performance was when 87.5% of words came after the term, followed by 75% after the term (Supplementary Figure 2D).”
Quote in the paperNot a substantial claim - Findingown data
Substituting the candidate term with a common token EVENT, with the term and section source title prepended to the string, shows improved performance.
“Next, we show that substituting the candidate term with a common token “EVENT”, with the term and section source title prepended to the string shows improved performance (Figure 1B).”
Quote in the paperNot a substantial claim - Findingown data
Pediatric drug labels contain ADE descriptions related to increased susceptibility to suicidal ideation and behaviour for adolescents when taking antidepressants.
“Of note, we found specific occurrences of ADE descriptions in the pediatric drug labels related to the delayed elimination of amoxicillin due to the incompletely developed renal function in neonates and young infants, and the increased susceptibility to suicidal ideation and behaviour for adolescents when taking antidepressants.”
Quote in the paperNot a substantial claim - Findingown data
Neither Chemprop nor ChemBERTa perform consistently better across all 10 ADEs.
“We find that neither Chemprop nor ChemBERTa perform consistently better across all 10 ADEs.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 10 severe ADEs selected for ChemBERT are representative of severe ADEs.
- Findingown data
The EU dataset has the fewest number of drugs with an ADE listed within an SOC in most cases.
“However, the EU dataset has the fewest number of drugs with an ADE listed within an SOC in most cases.”
Quote in the paperNot a substantial claim - Findingown data
The fine-tuned models achieve high performance in accurately predicting and extracting true ADE terms from drug label text.
“The evaluations and benchmarking analyses suggest that we are able to achieve high performance in accurately predicting and extracting “true” ADE terms from drug label”
Quote in the paperSubstantial claim - Findingown data
Scaffold-stratifying reduced performance, resulting in a best model test-set AUROC of 0.548-0.719 per ADR.
“Scaffold-stratifying reduced performance, resulting in a best model test-set AUROC of 0.548–0.719 per ADR.”
Quote in the paperSubstantial claim- Rests on an assumption: The 10 severe ADEs selected for ChemBERT are representative of severe ADEs.
- Findingown data
The number of listed ADEs per drug is clearly correlated between different data sources.
“Despite apparent bias and variation between drugs, the number of listed ADEs per drug is clearly correlated between different data sources.”
Quote in the paperSubstantial claim - Findingnone given
The authors applied the method to all publicly available FDA SPLs to construct OnSIDES database of adverse reactions and boxed warnings.
“We apply this method to all publicly available FDA SPLs to construct the OnSIDES (ON label SIDE effectS) database of both adverse reactions and boxed warnings.”
Quote in the paperSubstantial claim - Findingown data
The fine-tuned general NLP model can accurately pull and normalize drug safety information from label text in multiple formats.
“In this study, we show that we are able to accurately extract and standardize drug safety information from drug label data structured in various text formats by fine-tuning a generally trained NLP model.”
Quote in the paperSubstantial claim - Findingown data
Extending the method to international labels incorporates ADE data for an additional 1,777 unique drug ingredient combinations not found in OnSIDES.
“By extending this method to international drug labels, we incorporate ADE data for an additional 1,777 unique drug ingredient combinations not found in OnSIDES.”
Quote in the paperSubstantial claim - Findingown data
Combining exact string matching with DeepCADRME achieved an F1 score of 0.891 and an AUROC of 0.900 on a 100-drug-label test set using PubMedBERT.
“the combined method of extraction achieved a F1 score of 0.891 and an AUROC of 0.900.”
Quote in the paperNot a substantial claim - Findingown data
The majority of drug-ADE pairs were not exactly shared between sources.
“Overall, we found that the majority of the drug-ADE pairs were not exactly shared between sources.”
Quote in the paperNot a substantial claim - Premisecitation
Lack of broad-scale studies may be attributed in part to heterogeneity and complexity of ADEs and lack of standardized publicly available data.
“The lack of broad scale studies may be attributed in part to the heterogeneity and complexity of the phenomenon itself, and the lack of standardized data publicly available5–6.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [6] source
- Premisecitation
Recent advances in natural language processing, such as BERT models, enable accurate extraction of relevant information from unstructured biomedical text.
“Recent advances in Natural Language Processing methods, such as BERT models, present an opportunity to accurately extract relevant information from unstructured biomedical text.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] source
- Premisecitation
Drug labels are considered the gold standard primary resource for ADE information that physicians and patients reference.
“Drug labels are considered the gold standard primary resource for ADE information that both physicians and patients reference, with many international regulatory agencies including the FDA (Food and Drug Administration) and the EMA (European Medicines Agency) maintaining publicly accessible online drug label repositories12.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [12] source
- Interpretationown data
While considerable variation exists, results show considerable similarity between different data sources.
“Overall, while considerable variation exists, these results also show considerable similarity between different data sources.”
Quote in the paperSubstantial claim - Premisecitation
Specific populations are known to be at higher risk for ADEs when taking certain medications.
“However, specific populations are known to be at higher risk for ADEs when taking certain medications.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationown data
OnSIDES-based HLT prediction models achieve performance on-par with previous work predicting the less-granular SOCs.
“While we find that our OnSIDES-based HLT prediction models achieve performance on-par with previous work predicting the less-granular SOCs, we hypothesize this relatively small dataset may not be large enough to achieve optimal performance in training deep learning models for this particular task.”
Quote in the paperNot a substantial claim - Interpretationown data
The initial performance of ADEs 4 and 6 with random-split was confounded by train/test information leakage.
“In particular, ADEs 4 and 6 in particular drop significantly when scaffold stratification is applied, suggesting that their initial performance with random-split was confounded by train/test information leakage.”
Quote in the paperSubstantial claim - Interpretationown data
The varying degree of overlap provides insight into cross-comparability between data sources for different drug classes and ADEs.
“This varying degree of overlap provides insight into the cross-comparability between data sources for different drug classes and ADEs.”
Quote in the paperSubstantial claim - Interpretationown data
The similarities between side-effect profiles found in drugs in the OnSIDES database may be used to predict novel drug targets of a given drug.
“This suggests that the similarities between side-effect profiles found in drugs in the OnSIDES database may be used to predict novel drug targets of a given drug.”
Quote in the paperSubstantial claim - Interpretationown data
The dataset's bias may affect model performance on drug labels or ADEs that are less represented in the dataset.
“may influence model performance on drug labels / ADEs that are less represented in this dataset.”
Quote in the paperNot a substantial claim - Study design and conductown data
The annotated dataset contains potential biases.
“Of note, there are potential biases contained in this dataset, which should be taken into consideration when utilizing this dataset and interpreting the results described as part of this study.”
Quote in the paperNot a substantial claim - Interpretationown data
The method has moderate flexibility in adapting to specific conditions.
“Here, we find that the model achieves an overall accuracy of 0.64, precision of 0.61, and AUROC of 0.66 in extracting pediatric-specific ADE terms from the aforementioned sections, suggesting the moderate flexibility of adapting this method to specific conditions.”
Quote in the paperNot a substantial claim - Study design and conductown data
The extraction method's performance was validated by manually reviewing and annotating 200 randomly sampled drug labels each from the EU, UK, and Japan.
“in order to validate the performance of the developed extraction method, we manually reviewed and annotated 200 randomly sampled drug labels each from the EU, UK, and Japan.”
Quote in the paperSubstantial claim - Interpretationown data
OnSIDES can be applied across many pharmacovigilance and related uses.
“Through example drug repurposing and chemoinformatics studies, we show that OnSIDES can be used in a wide variety of pharmacovigilance and related applications.”
Quote in the paperNot a substantial claim - Interpretationown data
A substantial amount of context following the term improves the model's predictive performance on this task.
“suggesting that for this task a significant amount of post-term context aids the model’s predictive performance.”
Quote in the paperSubstantial claim - Premisecitation
The risk of serious ADEs is greater for pediatric patients.
“For example, while the risk of serious ADEs is greater for pediatric patients, our knowledge is limited for this population, with few RCTs being conducted on these populations given the restricted population and ethical concerns24.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- May combine two claims. 43
- Classified as probably not a substantial claim. 28
- The quote only partly states this claim. 1
May combine two claims.
PremisecitationSIDER is one of the most widely utilized ADE databases derived from drug label data, constructed using dictionary-based named entity recognition to extract adverse event terms from FDA Structured Product Labeling drug labels.
“One of the most widely utilized ADE databases derived from drug label data is SIDER, constructed using a dictionary-based named entity recognition (NER) method to extract adverse event terms from FDA Structured Product Labeling (SPLs) drug labels13.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [13] source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA number of publicly available ADE databases exist, constructed from sources including drug labels, electronic health records, research literature, social media, and pharmacovigilance programs.
“A number of publicly available ADE databases exist, which have been constructed by extracting, stratifying or detecting signals from a variety of sources, including drug labels, electronic health records, research literature, social media, and adverse event reporting pharmacovigilance programs7–11.”
Quote in the paperNot a substantial claim3 of 5 sources checked- cites [7] source
The Materials and Methods ‘Data source’ section describes SIDER as mined from drug labels and identifies FDA and Canadian adverse-event reporting databases. The paper also makes its own ADE databases publicly available, but it does not describe databases constructed from research literature or social media; it uses electronic medical records to validate findings, not as a source for those databases. Page numbers are unavailable in our copy. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites [8] source
The Background and Summary identifies electronic health records, spontaneous reporting systems, social media, and literature mining as sources of drug-safety evidence. The Usage Notes describe a harmonized evidence base combining product labels, published literature, observational data, and spontaneous reports; the Methods and Data Records describe a publicly available FAERS-derived dataset. The paper does not establish that publicly available ADE databases have been constructed from each listed source, particularly social media. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [9] source
- cites [10] source
- cites [11]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEfforts have fine-tuned BERT models to classify drug label text blocks into predefined sections and outperformed baseline models.
“For example, a number of efforts have fine-tuned BERT models to classify drug label text blocks into predefined sections of the label and outperformed baseline models for this task22–23.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [22] source
Section 2.3.5 (p. 9) describes classifying PubMed abstracts by cancer hallmark, and Section 2.4 (p. 12) describes BERT-based document classification. Neither the supplied passages nor searches for drug-label text blocks and label-section classification found a passage addressing that task or its performance against baselines. (LLM, conf=0.75)
Experimental — accuracy not yet measured - cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationcitationThese research efforts present promising results in extracting true ADE mentions, indicating suitability of NLP methods for this task.
“These research efforts present promising results in extracting true ADE mentions from drug label text, indicating the suitability of using NLP methods for this task.”
Quote in the paperSubstantial claim2 of 4 sources checked- cites [20] source
- cites [21] source
- cites [22] source
The Named Entity Recognition section (p. 7) describes NLP models for detecting drug and disease entities, and the results table (p. 17) reports performance on biomedical extraction tasks. These support the broader suitability of NLP for such tasks, but the cited paper does not report extracting true adverse drug event mentions from drug label text; its described datasets use PubMed abstracts. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe TAC 2017 task was conducted to encourage development of computational methods to recognize ADE mentions from drug labels and their relationships.
“Of note, the “Adverse Drug Reaction Extraction from Drug Labels” task of the Text Analysis Conference (TAC) 2017 was conducted to encourage the development of computational methods to recognize mentions of ADEs from drug labels and their relationships.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [18]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [19] source
The Background & Summary identifies the TAC 2017 task as an evaluation of adverse-reaction extraction from drug labels, and Technical Validation describes its mention-level annotations and training and test sets. Our copy does not say that the task aimed to recognize relationships between ADE mentions, so that part of the claim is unsupported. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationNumerous biomedical models such as BioBERT, MedBERT, and BiomedBERT have been developed by training on relevant text corpora.
“In biomedical informatics, numerous models utilizing this approach, such as BioBERT, MedBERT, and BiomedBERT, have been developed by training models on various relevant text corpora, such as EHR data and biomedical academic literature15–17.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [15]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [16]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [17] source
The Introduction identifies Med-BERT as a pretrained model trained on structured EHR diagnosis data. It explicitly says those data are ICD codes, unlike the free text used to train most BERT variants, so describing Med-BERT as trained on a text corpus is inaccurate. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA number of NLP methods have been proposed to extract ADE terms from drug label text utilizing the TAC 2017 dataset.
“A number of NLP methods have been proposed to extract ADE terms from drug label text utilizing this dataset20–21.”
Quote in the paperNot a substantial claim1 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe TAC 2017 organizers published a manually annotated gold-standard reference dataset for extracting ADEs from 200 FDA SPLs.
“As part of this task, the organizers published a manually annotated, gold-standard reference dataset for extracting adverse drug events from 200 FDA SPLs18–19.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [18]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [19] source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationEU, UK, and Japan represent some of the largest pharmaceutical markets outside the US and have different drug regulatory processes independent of the US.
“These nations/regions represent some of the largest pharmaceutical markets in the world outside of the US, and have different drug regulatory processes independent to that of the US25–26.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [25]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [26] source
The Summary (page unavailable) describes the US and EU as major locations for drug investigation and approval and says their regulatory systems differ. The European Regulation of Drugs section describes EU approval routes distinct from the FDA’s. The paper does not establish the claimed market standing of the UK or Japan or describe Japan’s regulatory process; it discusses the UK within the EU framework rather than as a separate regulatory system. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSignificant advances in chemoinformatics and AI to drug discovery and repurposing have increased research into in-silico driven development of new medications.
“Significant advances in chemoinformatics and the application of artificial intelligence to drug compound discovery and the repurposing of existing drugs have greatly increased research into in-silico driven development of new, effective medications for a variety of indications30.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [30] source
The Introduction (page unavailable) describes growing drug-discovery data resources, predictive models that may aid drug repurposing, and a need for computer-aided drug discovery. The model-development sections (page unavailable) also discuss cheminformatics and in-silico approaches, but the paper does not establish that advances in AI and repurposing have greatly increased research into new, effective medications. (LLM, conf=0.79)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDrugBank mainly contains drug indications extracted from drug labels, while MEDI uses public internet sources for drug indications.
“DrugBank primarily contains drug indications extracted from drug labels, while MEDI utilizes public internet sources to drug indications, and is a well utilized resource of drug indications.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [48] source
In “New and enhanced interface features” (page unavailable), the paper says DrugBank’s new linked indications were manually curated from drug labels, but specifies that these were available only for currently marketed products. It does not address MEDI, its use of public internet sources, or how widely MEDI is used. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [49] source
The Methods > Data sources section (page unavailable) says MEDI draws indications from four public resources, including the MedlinePlus website and Wikipedia. The paper does not identify DrugBank as a source or say that DrugBank's indications primarily come from drug labels; its discussion of label-derived indications concerns SIDER instead. It also does not establish that MEDI is well utilized. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChemBERTa performed better than Chemprop on several other MoleculeNet benchmark tasks.
“However, ChemBERTa was found to perform better than Chemprop on a number of other MoleculeNet benchmark tasks51”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [51] source
The Results and discussion section and Table 3 in the Conclusions compare other models on MoleculeNet tasks, but neither reports a ChemBERTa–Chemprop comparison. Searches for both model names and their comparative performance found no passage addressing the claimed result. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe SPL format includes a broad range of information such as indications, clinical pharmacology, contraindications, warnings, and ADEs.
“The SPL format incorporates a broad range of information including the drug’s indications, clinical pharmacology, contraindications, warnings, and ADEs36.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [36] source
The MEDICUS EXTENSION section mentions FDA drug labels linked to DailyMed, but does not describe the SPL format or what information it contains. Searches for SPL and the listed label contents found no passage addressing the claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
Findingown dataOnSIDES contains 3,610,120 ADEs and 114,322 Boxed Warning ADEs from 47,211 drug labels, covering 3,233 unique drug ingredient combinations and 4,510 unique ADEs.
“OnSIDES contains over 3.6 million drug-ADE pairs for 3,233 unique drug ingredient combinations extracted from 47,211 labels.”
- Also stated in: § Generating the OnSIDES Database · plain
“As of the February 2024 release, the OnSIDES database contains 3,610,120 ADEs and 114,322 Boxed Warning ADEs from 47,211 drug labels, listing 3,233 unique drug ingredient combinations (1,873 single-ingredient drugs, 1,360 multi-ingredient drugs) and 4,510 unique ADEs.”
Quote in the paperSubstantial claim- Also stated in: § Generating the OnSIDES Database · plain
May combine two claims.
Findingown dataThe model achieved F1 0.90, AUROC 0.92, and AUPR 0.95 at extracting ADEs from the Adverse Reactions section.
“We achieve an F1 score of 0.90, AUROC of 0.92, and AUPR of 0.95 at extracting ADEs from the labels’ “Adverse Reactions” section.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenThe method was used to extract pediatric-specific ADEs, serious ADEs from Boxed Warnings, and ADEs from UK, EU, and Japan labels to build OnSIDES-INTL.
“We further utilize this method to extract pediatric-specific ADEs, serious ADEs from labels’ “Boxed Warnings” section, and ADEs from drug labels of other major nations—the UK, EU, and Japan—to build a complementary OnSIDES-INTL database.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataSIDER contains 1,430 unique drugs and 4,251 unique side effects.
“SIDER contains 1,430 unique drugs and 4,251 unique side effects.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAn OnSIDES-trained linear regression model performs moderately in predicting whether a given drug has a specific drug target based on its side-effect profile (AUROC = 0.667) and outperforms a SIDER-trained model (AUROC = 0.590).
“Additionally, we find that an OnSIDES-trained linear regression model performs moderately in predicting whether a given drug has a specific drug target based on its side-effect profile (AUROC = 0.667) and outperforms a SIDER-trained model (AUROC = 0.590) (Figure 4B).”
Quote in the paperSubstantial claim- Rests on an assumption: Drug target and indication data from DrugBank, OpenTargets, and MEDI are accurate.
May combine two claims.
Findingown dataThere was notable overlap in sensory organ drugs between US and UK, and antineoplastic and immunomodulating agent drugs among all data sources.
“However, there was some notable overlap in sensory organ drugs between the US and UK, and also antineoplastic and immunomodulating agent drugs among all data sources.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOnSIDES-INTL contains 108,958, 705,917, and 237,827 unique drug-ADE pairs from 1,048, 9,003, and 6,541 drug labels, with 929, 1,889, and 1,441 unique drug ingredient combinations and 3,570, 4,848, and 3,296 unique ADEs from European, British, and Japanese labels respectively.
“By doing so, we construct additional databases with 108,958, 705,917, and 237,827 unique drug-ADE pairs from 1,048, 9,003, and 6,541 drug labels, with 929, 1,889, and 1,441 unique drug ingredient combinations and 3,570, 4,848, and 3,296 unique ADEs from European, British, and Japanese drug labels respectively.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOnSIDES contains 1,485 unique single-ingredient drugs and 4,345 unique side effects.
“OnSIDES contains 1,485 unique single-ingredient drugs and 4,345 unique side effects”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOnSIDES performs best in predicting existing therapeutic indications identified in both DrugBank and MEDI (AUROC = 0.77) and better than SIDER (AUROC = 0.715).
“We further find that OnSIDES performs best in terms of predicting the existing therapeutic indications that are identified in both DrugBank and MEDI (AUROC = 0.77) and better than SIDER in these predictions also (AUROC = 0.715), suggesting these models perform well in predicting indications of high confidence (Figure 4D).”
Quote in the paperSubstantial claim- Rests on an assumption: Drug target and indication data from DrugBank, OpenTargets, and MEDI are accurate.
May combine two claims.
Findingown dataSOCs identified across the fewest drug labels were product issues, pregnancy and related conditions, and congenital, familial, and genetic disorders while general disorders and administration site conditions, gastrointestinal, skin and subcutaneous tissue, nervous system disorders, and vascular disorders were identified on the largest number of drug labels.
“Overall, the SOCs identified across the fewest number of drug labels were “product issues”, “pregnancy and related conditions”, and “congenital, familial, and genetic disorders” while “general disorders and administration site conditions”, “gastrointestinal”, “skin and subcutaneous tissue”, “nervous system disorders”, and “vascular disorders” were identified on the largest number of drug labels.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe annotated dataset has many ADEs in vascular disorders, investigations, and skin and subcutaneous tissue disorders, and few in endocrine disorders, social circumstances, and congenital, familial, and genetic disorders.
“we find a significant proportion of annotated ADEs that fall into the “vascular disorders”, “investigations”, and “skin and subcutaneous tissue disorders” categories, and relatively few “endocrine disorders”, “social circumstances”, and “congenital, familial, and genetic disorders” (Supplementary Figure 1B).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe antineoplastics and immunomodulators drug class had the largest number of ADEs while antiparasitics and insecticides consistently had the fewest across all data sources.
“We also find that the “antineoplastics and immunomodulators” drug class had the largest number of ADEs while the “antiparasitics and insecticides” class drugs consistently have the fewest identified across all data sources.”
Quote in the paperNot a substantial claimThe quote only partly states this claim.
FindingreasoningThe method was extended to extract ADE data from EU, UK, and Japanese drug labels to create additional computable resources.
“we leverage the method to expand the database to incorporate ADE data found in drug labels of other nations/regions.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenFDA SPL labels mainly present ADEs as free text, while labels from other regulatory agencies present the ADE section as structured lists, tables, or a mix.
“While the FDA SPL drug labels primarily list the ADEs in a free text format, drug labels from other regulatory agencies format the ADE section in structured lists, tables, or a combination of the three formats.”
Quote in the paperSubstantial claimMay combine two claims.
FindingreasoningThe extraction method makes errors: some side effects are missed and some predicted adverse events are wrong.
“computational extraction methods are prone to errors, with some side effects not extracted accurately, and some predicted adverse events being incorrect.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataGeneral ADE descriptions were standardized across drug labels and label types, but pediatric ADE descriptions were often nested in general ADE sections or absent due to lack of information.
“While general ADE descriptions were found to be standardized across drug labels / label types, pediatric ADE descriptions were often found nested in the general ADE sections, or did not exist anywhere in the label text due to the lack of information.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOptimal input string length is 125 words, with 87.5% of the words coming after the candidate term.
“We first determine the optimal input text string format, concluding that optimal input string length is 125 words, with 87.5% of the words coming after the candidate term (Figure 1A).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataDrugs tend to have the highest degree in US data, followed by UK, while Japanese data tends to assign the lowest degree.
“Drugs tend to have the highest degree (number of ADEs) in the US data, followed by the UK data, while the Japanese data tends to assign the lowest degree.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWith the optimal parameter combination, PubMedBERT achieved an F1 score of 0.874 and an AUROC of 0.899.
“we achieve F1 Scores of 0.861 and 0.874, and AUROC Scores of 0.896 and 0.899, respectively for the ClinicalBERT and PubMedBERT models.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn comparison between OnSIDES and OnSIDES-PED, there was enrichment of a large number of Skin and Subcutaneous Tissue and Gastrointestinal ADEs, and very few Social Circumstances and Pregnancy, Puerperium and Perinatal Conditions ADEs enriched.
“In the comparison between OnSIDES and OnSIDES-PED, we observed an enrichment of a large number of “Skin and Subcutaneous Tissue” and “Gastrointestinal” ADEs, and very few “Social Circumstances” and “Pregnancy, Puerperium and Perinatal Conditions” ADEs enriched.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOnSIDES-PED contains 359,213 drug label-ADE pairs from 1,161 unique drug ingredient combinations, 1,526 unique ADEs, and 20,014 drug labels derived from special populations section.
“We use this adapted method to construct a large pediatric-specific ADE database of 359,213 drug label-ADE pairs from 1,161 unique drug ingredient combinations, 1,526 unique ADEs, and 20,014 drug labels derived from the drug labels’ special populations section.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe fine-tuned PubMedBERT model outperforms all TAC 2017 models and the SIDER database.
“We find that our model outperforms all TAC 2017 models and the SIDER database as presented in (Supplementary Figure 3B).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBetween OnSIDES-PED and KidSIDES, there was enrichment of ADEs in Nervous System and General and Administration Site Conditions MedDRA SOC classes, and a relative lack of ADEs reported in Ear and Labyrinth and Social Circumstances classes in KidSIDES.
“Between OnSIDES-PED and KidSIDES, we observed an enrichment of ADEs in the “Nervous System” and “General and Administration Site Conditions” MedDRA SOC classes, and a relative lack of ADEs reported in the “Ear and Labyrinth” and “Social Circumstances” classes in KidSIDES.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe OnSIDES-trained model was able to predict drug indications from each database, and outperforms a SIDER-trained model.
“We find that the OnSIDES-trained model was able to predict drug indications from each database, and outperforms a SIDER-trained model (Figure 4C).”
Quote in the paperSubstantial claim- Rests on an assumption: Drug target and indication data from DrugBank, OpenTargets, and MEDI are accurate.
May combine two claims.
Findingown dataThe PubMedBERT-based model gives a modest performance improvement over the ClinicalBERT-based model and performs more consistently across the two evaluations.
“The PubMedBERT-based model achieves a modest performance improvement over the ClinicalBERT-based model, and notably it performs more consistently across the two different evaluations.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe annotated dataset overrepresents antineoplastic and immunomodulating agents and underrepresents dermatologicals, systemic hormonal preparations, and antiparasitic products.
“we find that there is a significant representation of drugs that are “antineoplastic and immunomodulating agents”, and very few “dermatologicals”, “systemic hormonal preparations”, and “antiparasitic products” (Supplementary Figure 1A).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWith the optimal parameter combination, ClinicalBERT achieved an F1 score of 0.861 and an AUROC of 0.896.
“we achieve F1 Scores of 0.861 and 0.874, and AUROC Scores of 0.896 and 0.899, respectively for the ClinicalBERT and PubMedBERT models.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataUS drug labels tend to have more listed ADEs than other nations, although considerable variation exists between drugs.
“These results show that US drug labels tend to have more listed ADEs than that of other nations, although considerable variation exists between drugs.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMEDI has 1,257 drugs with indications for OnSIDES and 1,155 for SIDER.
“There are 1,474 and 1,347 drugs with indications listed in DrugBank, and 1,257 and 1,155 drugs with indications listed in MEDI for OnSIDES and SIDER respectively.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDrugBank has 1,474 drugs with indications for OnSIDES and 1,347 for SIDER.
“There are 1,474 and 1,347 drugs with indications listed in DrugBank, and 1,257 and 1,155 drugs with indications listed in MEDI for OnSIDES and SIDER respectively.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenPediatric mentions in the general AR section are difficult to systematically separate from non-pediatric-specific mentions.
“as the pediatric mentions in this section are difficult to systematically separate from those that are non-pediatric specific.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSignificant advancements in NLP methods, including BERT models, have been made.
“In recent years, significant advancements have been made in Natural Language Processing (NLP) methods, including the development of BERT (Bidirectional Encoder Representations from Transformers) models14.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14] source
May combine two claims.
PremisecitationFew machine-readable databases of adverse drug events exist, which limits broad systematic study of drug safety.
“However, few machine-readable databases of ADEs exist, limiting our capacity to study drug safety on a broader, systematic scale.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [6] source
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningOnSIDES can be utilised as a comprehensive resource to study and enhance drug safety.
“OnSIDES can be utilised as a comprehensive resource to study and enhance drug safety.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisenone givenDrug label repositories are collections of raw text or PDF files and are difficult for secondary research because they are not standardized or optimized for machine readability.
“However, these repositories are a collection of individual raw text or PDF files, and are difficult to utilize for secondary research purposes as they are not standardized or optimized for machine readability.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningTo comprehensively study ADEs, high-quality machine-readable databases encompassing the breadth of drugs currently used are required.
“As such, to comprehensively study ADEs, we require high-quality, machine-readable databases encompassing the breadth of drugs currently used.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataMore context helps performance, but an overly long input string may dilute relevant context with unrelated information.
“This suggests that more context aids performance, but too long of an input string may dilute the relevant context with unrelated information.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningSome known ADEs may not appear on drug labels and may be found in other sources like adverse event reporting systems.
“there may be known ADEs that are not yet reflected on drug labels that may appear in other sources, such as adverse event reporting systems.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationKnowledge of ADEs in pediatric patients is limited, with few RCTs conducted on these populations due to restricted population and ethical concerns.
“For example, while the risk of serious ADEs is greater for pediatric patients, our knowledge is limited for this population, with few RCTs being conducted on these populations given the restricted population and ethical concerns24.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationnone givenTo the best of the authors' knowledge, OnSIDES is the largest and only current up-to-date drug label-derived ADE database.
“To the best of our knowledge, OnSIDES is the largest and only, current, up-to-date, drug label-derived ADE database.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationPubMedBERT and ClinicalBERT were selected because they are trained on biomedical language similar to drug label text and have performed well on relevant NLP tasks.
“We chose to consider and evaluate these models as they are trained on biomedical language similar to drug label text, and have been demonstrated to perform well on relevant NLP tasks37.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [37]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductown dataThe manually annotated dataset was used to evaluate the model's accuracy in the generated datasets.
“We use this manually annotated dataset to evaluate the accuracy of our model in the generated datasets.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationCampillios et al. previously suggested the similarity of two drugs' side effect profiles is positively correlated with the probability that these drugs share a common target.
“The work of Campillios et al. previously suggested the similarity of two drugs’ side effect profiles is positively correlated with the probability that these drugs share a common target28.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [28] source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationYang and Agarwal showed side-effect profile similarities can also predict shared indications.
“In a similar manner, Yang and Agarwal showed side-effect profile similarities can also predict shared indications29.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationown dataThe limited description and subsequent extraction of pediatric-specific ADEs in drug label text is highlighted by KidSIDES having more drugs in each category.
“Again, KidSIDES had more drugs in each category, highlighting the limited description and subsequent extraction of pediatric-specific ADEs in drug label text.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataOnSIDES could be used as a potential resource for drug repurposing applications, identifying existing drugs that may be used for novel therapeutic indications, and to identify potential drug off-targets.
“This suggests that OnSIDES could be used as a potential resource for drug repurposing applications, identifying existing drugs that may be used for novel therapeutic indications, and to identify potential drug off-targets.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
PremisereasoningFurther challenges lie in applying these methods to systematically construct a comprehensive and readily accessible database to aid further pan-medication study of ADEs.
“However, further challenges lie in applying these methods to systematically construct a comprehensive and readily accessible database to aid further pan-medication study of ADEs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe accuracy of machine translation for Japanese labels is a limitation of OnSIDES-INTL.
“Another limitation is the machine translation accuracy of the Japanese labels.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChemprop performed better than ChemBERTa for ClinTox classification.
“Ahmad et al.32 found that Chemprop performed better than ChemBERTa for ClinTox classification, a drug-toxicity classification task.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ahmad et al.32
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe EMA, MHRA, and PMDA are the regulatory bodies overseeing pharmaceutical product approval in the EU, UK, and Japan, respectively.
“In each country/region, the EMA (European Medicines Agency), MHRA (Medicines and Healthcare products Regulatory Agency), and PMDA (Pharmaceuticals and Medical Devices Agency) are the respective regulatory bodies overseeing pharmaceutical product approval38–40.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [38]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [39]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [40]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAll entities submitting drug approval applications to the FDA must submit documentation in SPL format.
“All entities that submit drug approval applications to the FDA are required to submit documentation in this format35.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [35]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataEnrichment in the number of adverse events per adverse event type for each drug class was evaluated using Fisher's exact test with a Bonferroni adjustment.
“we evaluated enrichment in the number of adverse events per adverse event type for each drug class using Fisher’s exact test, as listed in the axis labels in Figure 3C. We used a Bonferroni adjustment to p-values from these tests due to testing all drug classes in parallel.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationChemBERTa is a transformer-based RoBERTa model that takes SMILES strings as input and can be fine-tuned to predict chemical and physical targets.
“ChemBERTa is a transformer-based model of RoBERTa architecture52 that takes SMILES strings as input and can be fine-tuned to predict various chemical and physical targets”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe SPL drug label format is an XML-based HL7 clinical and administrative standard document markup format used by the FDA.
“The SPL drug label format is a XML-based HL7 clinical and administrative standard document markup format utilized by the FDA to uniformly present relevant pharmaceutical information text34.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [34]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe SHAP method is a game theory-based approach that explains model predictions by assigning importance values to individual features.
“The SHAP method is a game theory-based approach that explains model predictions by assigning individual features an importance value for a specific prediction, allowing for a better understanding of how individual features impact model decisions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [45]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
The paper relies on this without support.
Fewer broad-scale studies of ADEs exist compared to studies of individual medications.
“While many drug safety studies are conducted on individual medications during clinical trials and through postmarketing surveillance programs, far fewer studies have studied the occurrence of ADEs more broadly.”
The paper relies on this without support.
The MedDRA vocabulary terms used as a reference dictionary cover all potential ADE terms in drug labels.
“we first extract potential ADE terms from the drug label using MedDRA vocabulary terms as a reference dictionary”
The paper relies on this without support.
GPT-4 machine translation of Japanese free text into English preserves ADE term meaning sufficiently for extraction.
“The free text sections are machine translated using GPT-4 into English, and then we use the fine-tuned model to predict ADE terms.”
The paper relies on this without support.
Exact string matching against MedDRA-J for Japanese tabular sections correctly identifies ADE mentions.
“we use MedDRA-J, the Japanese MedDRA vocabulary dictionary, to perform exact string matching for the tabular sections.”
The paper relies on this without support.
The manually annotated 200 drug labels for pediatric and international tasks are representative of all labels.
“we manually annotate 200 randomly sampled drug labels to evaluate the fine-tuned PubMedBERT model’s performance.”
The paper relies on this without support.
The 10 severe ADEs selected for ChemBERT are representative of severe ADEs.
“We predict whether these drugs are known to cause any of the 10 severe ADEs selected, which were mapped to MedDRA HLTs (High-Level Terms).”
The paper relies on this without support.
Drug target and indication data from DrugBank, OpenTargets, and MEDI are accurate.
“Drug target and indication data for these drugs are mapped from the DrugBank, OpenTargets, and MEDI databases”
The paper relies on this without support.
Drug labels are the gold standard source of ADE information.
“Drug labels continue to be the gold standard for ADE information.”
The paper relies on this without support.
No actively maintained comprehensive ADE databases exist.
“However, no actively maintained comprehensive ADE databases exist, making it difficult to leverage computational tools to study ADEs broadly.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Target-prediction model is called both logistic and linear regression
MinorThe methods describe logistic regression for predicting shared drug targets, whereas the results call the OnSIDES target-prediction model linear regression. These name different regression methods.
“Then, we use z-score normalized Tanimoto coefficient scores to predict the shared drug targets and indications using logistic regression models.”
“Additionally, we find that an OnSIDES-trained linear regression model performs moderately in predicting whether a given drug has a specific drug target based on its side-effect profile (AUROC = 0.667)”
FDA label counts differ for the February 2024 release
MinorThe results say the February 2024 database draws from 47,211 drug labels, but the methods say 46,686 FDA labels were downloaded as of the same month. The text does not explain the additional 525 labels.
“As of the February 2024 release, the OnSIDES database contains 3,610,120 ADEs and 114,322 Boxed Warning ADEs from 47,211 drug labels”
“To develop the model to extract ADE mentions and construct the OnSIDES database, we download 46,686 FDA SPL drug labels from NIH NLM DailyMed available as of February 2024.”
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
DrugBank 5.0: a major update to the DrugBank database for 2018
Wishart, D. S., Feunang, Y. D., Guo, A. C., Lo, E. J., Marcu, A., Grant, J. R., Sajed, T., Johnson, D., Li, C., Sayeeda, Z., et al. (2018) · Nucleic Acids Research · doi:10.1093/nar/gkx1037 · Cited by 8,404 · see citing works
Toward understanding the origin and evolution of cellular organisms
Kanehisa, M. (2019) · Protein Science · doi:10.1002/pro.3715 · Cited by 5,509 · see citing works
MoleculeNet: a benchmark for molecular machine learning
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., & Pande, V. (2018) · Chemical Science · doi:10.1039/c7sc02664a · Cited by 3,107 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
Analyzing Learned Molecular Representations for Property Prediction
Yang, K., Swanson, K., Jin, W., Coley, C., Eiden, P., Gao, H., Guzman-Perez, A., Hopper, T., Kelley, B., Mathea, M., et al. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00237 · Cited by 1,923 · see citing works
Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations?
Bajusz, D., Rácz, A., & Héberger, K. (2015) · Journal of Cheminformatics · doi:10.1186/s13321-015-0069-3 · Cited by 1,646 · see citing works
The SIDER database of drugs and side effects
Kuhn, M., Letunic, I., Jensen, L. J., & Bork, P. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1075 · Cited by 1,451 · see citing works
Drug target identification using side-effect similarity
Campillos, M., Kuhn, M., Gavin, A.-C., Jensen, L. J., & Bork, P. (2008) · Science · doi:10.1126/science.1158140 · Cited by 1,248 · see citing works
Data-driven prediction of drug effects and interactions
Tatonetti, N. P., Ye, P. P., Daneshjou, R., & Altman, R. B. (2012) · Science Translational Medicine · doi:10.1126/scitranslmed.3003377 · Cited by 936 · see citing works
Self-citationRasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property PredictionPreprint
Chithrananda, S., Grand, G., & Ramsundar, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09885 · Cited by 881 · see citing works
Literacy and misunderstanding prescription drug labels
Davis, T. C., Wolf, M. S., Bass, P. F., Thompson, J. A., Tilson, H. H., Neuberger, M., & Parker, R. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-12-200612190-00144 · Cited by 628 · see citing works
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Huang, K., Altosaar, J., & Ranganath, R. (2019) · arXiv · doi:10.48550/arxiv.1904.05342 · Cited by 618 · see citing works
Coleman, J. J., & Pontefract, S. K. (2016) · Clinical Medicine · doi:10.7861/clinmedicine.16-5-481 · Cited by 399 · see citing works
Cost of Prescription Drug–Related Morbidity and Mortality
Watanabe, J. H., McInnis, T., & Hirsch, J. D. (2018) · Annals of Pharmacotherapy · doi:10.1177/1060028018765159 · Cited by 366 · see citing works
Revisiting Few-sample BERT Fine-tuningPreprint
Zhang, T., Wu, F., Katiyar, A., Weinberger, K. Q., & Artzi, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05987 · Cited by 295 · see citing works
A curated and standardized adverse drug event resource to accelerate drug safety research
Banda, J. M., Evans, L., Vanguri, R. S., Tatonetti, N. P., Ryan, P. B., & Shah, N. H. (2016) · Scientific Data · doi:10.1038/sdata.2016.26 · Cited by 246 · see citing works
Self-citationVan Norman, G. A. (2016) · JACC: Basic to Translational Science · doi:10.1016/j.jacbts.2016.06.003 · Cited by 223 · see citing works
Towards reproducible computational drug discovery
Schaduangrat, N., Lampa, S., Simeon, S., Gleeson, M. P., Spjuth, O., & Nantasenamat, C. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-0408-x · Cited by 208 · see citing works
ChemBERTa-2: Towards Chemical Foundation ModelsPreprint
Ahmad, W., Simon, E. P., Chithrananda, S., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.01712 · Cited by 148 · see citing works
Adverse drug events: database construction and in silico prediction
Cheng, F., Li, W., Wang, X., Zhou, Y., Wu, Z., Shen, J., & Tang, Y. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci4000079 · Cited by 124 · see citing works
Development and evaluation of an ensemble resource linking medications to their indications
Wei, W.-Q., Cronin, R. M., Xu, H., Lasko, T. A., Bastarache, L., & Denny, J. C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001431 · Cited by 106 · see citing works
A dataset of 200 structured product labels annotated for adverse drug reactions
Demner-Fushman, D., Shooshan, S. E., Rodriguez, L., Aronson, A. R., Lang, F., Rogers, W., Roberts, K., & Tonning, J. (2018) · Scientific Data · doi:10.1038/sdata.2018.1 · Cited by 68 · see citing works
Using machine learning to identify adverse drug effects posing increased risk to women
Chandak, P., & Tatonetti, N. P. (2020) · Patterns · doi:10.1016/j.patter.2020.100108 · Cited by 51 · see citing works
Self-citationOverview of the TAC 2017 Adverse Reaction Extraction from Drug Labels Track.
Roberts, K. E., Demner‐Fushman, D., & Tonning, J. M. (2017) · Theory and applications of categories · Cited by 43 · see citing works
Douros, A., Ades, M., & Renoux, C. (2018) · CNS Drugs · doi:10.1007/s40263-018-0507-7 · Cited by 42 · see citing works
Osokogu, O. U., Fregonese, F., Ferrajolo, C., Verhamme, K., de Bie, S., Jong, G. ’., Catapano, M., Weibel, D., Kaguelidou, F., Bramer, W. M., et al. (2015) · Drug Safety · doi:10.1007/s40264-015-0265-0 · Cited by 31 · see citing works
Safety information in drug labeling: a comparison of the USA, the UK, and Japan
Shimazawa, R., & Ikeda, M. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3408 · Cited by 28 · see citing works
Tiftikci, M., Özgür, A., He, Y., & Hur, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3195-5 · Cited by 23 · see citing works
DeepCADRME: a deep neural model for complex adverse drug reaction mentions extraction
El-allaly, E.-D., Sarrouti, M., En-Nahnahi, N., & Ouatik El Alaoui, S. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.12.013 · Cited by 19 · see citing works
Automatic Extraction of Adverse Drug Reactions from Summary of Product Characteristics
Shen, Z., & Spruit, M. R. (2021) · Applied Sciences · doi:10.3390/app11062663 · Cited by 17 · see citing works
Giangreco, N. P., & Tatonetti, N. P. (2022) · Med · doi:10.1016/j.medj.2022.06.001 · Cited by 16 · see citing works
Self-citationGiangreco, N. P., & Tatonetti, N. P. (2021) · medRxiv · doi:10.1101/2021.07.15.21260602 · Cited by 15
Self-citationClinically relevant pretraining is all you need
Walk, O. J. B. D., Sun, T., Perotte, A., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab086 · Cited by 14 · see citing works
The European Federation of Pharmaceutical Industries and Associations
(2013) · Pharmaceutical Medicine · doi:10.1093/med/9780199609147.003.0119 · Cited by 13 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
5 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- 2018, PLoS One, 13
- 2012, PLoS One, 7
- 2019, PLoS Comput. Biol., 15
- 2019, Bioinformatics, 36, 1234
- 2011, PLoS One, 6
Cited by
Sorted by citations
Zhang, J., Li, H., Zhang, Y., Huang, J., Ren, L., Zhang, C., Zou, Q., & Zhang, Y. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf533 · Cited by 125 · see citing works
“Beyond recording acute and chronic toxicity effects on diverse organisms (algae, benthic organisms, fish, and birds), these databases include parameters like environmental persistence, bioaccumulation, and transformation, thus serving as foundational tools for ecological risk assessments, pollution…”
Genomics of drug target prioritization for complex diseases
Chen, R. K., Duffy, Á., & Do, R. (2025) · Nature Reviews Genetics · doi:10.1038/s41576-025-00904-4 · Cited by 22 · see citing works
“Outcome databases such as OnSIDES, which extracted over 3.6 million drug–adverse event pairs from FDA labels 173 , may facilitate model training.”
InfluentialGu, S. C., Welton, T., Sun, Q.-Y., Wu, Y., Tan, E. K., & Zhou, Z. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf350 · Cited by 6 · see citing works
“In parallel, LLMs such as BioGPT and PubMedBERT, trained on extensiv e biomedical corpora, are increasingly being explored for hypothesis generation, literature-based gene annotation, and functional context extraction [49, 50].”
Reinshagen, J., Seashore-Ludlow, B. A., Gadiya, Y., et al. (2025) · Database · doi:10.1093/database/baaf081 · Cited by 5 · see citing works
“We aim to integrate in future versions adverse effect and toxicity data from repositories such as the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) [ 48 ], VigiBase [ 49 ], OnSIDES [ 50 ], or PubChem.”
Causal knowledge graph analysis identifies adverse drug effects
Toonsi, S., Schofield, P. N., & Hoehndorf, R. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf661 · Cited by 6 · see citing works
Modeling the genomic architecture of adiposity and anthropometrics across the lifespan
Arehart, C. H., Lin, M. H., Gibson, R. A., Anderson, H. D., Aquilante, C. L., Arbogast, K. L., Arehart, C. H., Brooks, I. M., Brunetti, T. M., Brutus-Lestin, J., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62730-w · Cited by 4 · see citing works
“Notably, while 392 DEPICT-prioritized genes were common to BMI and F4, 45 genes were unique to only F4 (Supplementary Data 13, 15, and 36 – 37; Supplementary Fig.”
Ricci, P., Luca, M. D., Ferraro, G., et al. (2025) · Pharmacophore · doi:10.51847/8pe7suiyyy · Cited by 2 · see citing works
Large Language Models for Pharmaceutical Knowledge Management: A Critical Review
Torres, A., & Fernandez, M. (2025) · Pharmacophore · doi:10.51847/xphozuyfsy · Cited by 2 · see citing works
ALERT: An AI-enabled decision support approach for early risk monitoring from heterogeneous data
Bartal, A., Pliskin, N., Jagodnik, K. M., et al. (2026) · Decision Support Systems · doi:10.1016/j.dss.2026.114684 · Cited by 1 · see citing works
Jeong, E., Shendre, A., Su, Y., Guo, X., Li, L., & Chen, Y. (2025) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/phar.70058
Савосина, П. И., Poroikov, V., & Druzhilovskiy, D. S. (2025) · Safety and Risk of Pharmacotherapy · doi:10.30895/2312-7821-2025-535
SAMARA, A. A., & Khalil, A. A. (2026) · Med · doi:10.1016/j.medj.2025.100969
Asano, H., Tian, Y., Hatabu, A., Fukuzawa, K., Takaya, D., & Ikeda, K. (2026) · Research Square · doi:10.21203/rs.3.rs-8923363/v1
Disentangling the contribution of disease genes to drug therapeutic and side effectsPreprint
Lalagkas, P. N., & Melamed, R. D. (2026) · medRxiv · doi:10.64898/2026.05.04.26352378
SIDEKICK: A Semantically Integrated Resource for Drug Effects, Indications, and Contraindications
Ashhad, M., Mashkova, O., Henao, R., & Hoehndorf, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25159-6_14
Retrieval-Augmented Language Models for Checking Safety Consistency in Pharmaceutical Product Labels
Dupont, C., & Martin, J. (2025) · Pharmacophore · doi:10.51847/fzeuaxeluk
Wen, L., Xu, Z., Wu, X., Xiang, J., & Zhang, X. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27198685
- Authors
- Tanaka, Y., Chen, H. Y., Belloni, P., Gisladottir, U., Kefeli, J., Patterson, J., Srinivasan, A., Zietz, M., Sirdeshmukh, G., Berkowitz, J., LaRow Brown, K., & Tatonetti, N. P.
- Affiliations
- Columbia University Irving Medical Center (US)
- University of Padua (IT)
- Cedars-Sinai Medical Center (US)
- Corresponding author
- Tatonetti, Nicholas P.
- ORCID
- Venue
- Med
- Publisher
- Elsevier BV
- Published
- 2025-07
- Volume
- 6
- Issue
- 7
- Pages
- 100642
- ISSN
- 2666-6340 (ISSN-L)
- PMC
- PMC12256195
- OpenAlex
- W4409150806
- Cited by
- 17
- References
- 45
- Citations by year
- 2026: 6 · 2025: 11
- FWCI
- 8.63
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 375 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Med (acceptedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Padua Research Archive (University of Padova) (submittedVersion)
- Funders
- National Library of Medicine (T15LM007079)
- National Institutes of Health
- NIH National Institute of General Medical Sciences (R35GM131905, T32GM145440)
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Keywords
- adverse drug reactions, drug labels, drug safety, natural language processing, BERT, PubMedBERT, black box warning, drug target prediction
- MeSH terms
- Drug Labeling, Humans, Natural Language Processing, United States, United States Food and Drug Administration, Databases, Factual, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review