Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
The second track of the CEGS N-GRID 2016 natural language processing shared tasks focused on predicting symptom severity from neuropsychiatric clinical records. For the first time, initial psychiatric evaluation records have been collected, de-identified, annotated and shared with the scientific community. One-hundred-ten researchers organized in twenty-four teams participated in this track and submitted sixty-five system runs for evaluation. The top ten teams each achieved an inverse normalized macro-averaged mean absolute error score over 0.80. The top performing system employed an ensemble of six different machine learning-based classifiers to achieve a score 0.86. The task resulted to be generally easy with the exception of two specific classes of records: records with very few but crucial positive valence signals, and records describing patients predominantly affected by negative rather than positive valence. Those cases proved to be very challenging for most of the systems. Further research is required to consider the task solved. Overall, the results of this track demonstrate the effectiveness of data-driven approaches to the task of symptom severity classification.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mining electronic health records: towards better research applications and clinical care
Jensen, P. B., Jensen, L. J., & Brunak, S. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3208 · Cited by 1,680 · cited 2×
“Organizing clinical shared tasks is arduous due to the sensitive nature of the data involved: acquiring, de-identifying, annotating and sharing clinical records are all strictly regulated tasks [1].”
BackgroundEvaluating the state of the art in coreference resolution for electronic medical records
Uzuner, O., Bodnari, A., Shen, S., Forbush, T., Pestian, J., & South, B. R. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000784 · Cited by 164
Cited by
Machine learning and natural language processing in mental health: systematic review
Le Glaz, A., Haralambous, Y., Kim-Dufor, D.-H., Lenca, P., Billot, R., Ryan, T. C., Marsh, J., DeVylder, J., Walter, M., Berrouiguet, S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/15708 · Cited by 576
“A total of 8 studies were transnosographic [59,67-71,”
Clinical text data in machine learning: systematic review
Spasic, I., & Nenadic, G. (2020) · JMIR Medical Informatics · doi:10.2196/17984 · Cited by 358
“In addition to the NLP community challenges such as i2b2, ShARe/CLEF eHealth, and CEGS N-GRID, most commonly used data sources were MIMIC and VHA.”
Background2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records
Henry, S., Buchan, K., Filannino, M., Stubbs, A., & Uzuner, O. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz166 · Cited by 303
Self-citationA scoping review of publicly available language tasks in clinical natural language processing
Gao, Y., Dligach, D., Christensen, L., Tesch, S., Laffin, R., Xu, D., Miller, T., Uzuner, O., Churpek, M. M., & Afshar, M. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac127 · Cited by 28
Metadata
- Authors
- Filannino, M., Stubbs, A., & Uzuner, Ö.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2017-11
- Volume
- 75
- Pages
- S62-S70
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W2608185370
- Cited by
- 42
- References
- 2
- FWCI
- 9.39
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2. https://app.cochat.ai/papers/10.1016/j.jbi.2017.04.017
- The paper
- Filannino, M., Stubbs, A., & Uzuner, Ö. (2017). Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2. Journal of Biomedical Informatics. https://doi.org/10.1016/j.jbi.2017.04.017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.