The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents
Open accessAbstract
We describe our entry for the Systematic Review Information Extraction track of the 2018 Text Analysis Conference. Our solution is an end-to-end, deep learning, sequence tagging model based on the BI-LSTM-CRF architecture. However, we use interleaved, alternating LSTM layers with highway connections instead of the more traditional approach, where last hidden states of both directions are concatenated to create an input to the next layer. We also make extensive use of pre-trained word embeddings, namely GloVe and ELMo. Thanks to a number of regularization techniques, we were able to achieve relatively large capacity of the model (31.3M+ of trainable parameters) for the size of training set (100 documents, less than 200K tokens). The system's official score was 60.9% (micro-F1) and it ranked first for the Task 1. Additionally, after rectifying an obvious mistake in the submission format, the system scored 67.35%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Dextr employs a Bidirectional Long ShortTerm Memory - Conditional Random Field (BI-LSTM-CRF) neural network architecture (Nowak and Kunstman 2019); (Walker et al. 2022).”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Nowak and Kunstman 2019
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Research works printed but not linked: 13
- Yarin Gal and Zoubin Ghahramani. 2016. A theoretically grounded application of dropout in recurrent neural networks. In NIPS .
- Luheng He, Kenton Lee, Mike Lewis, and Luke S. Zettlemoyer. 2017. Deep semantic role labeling: What works and what's next. In ACL .
- Sepp Hochreiter and J¨ urgen Schmidhuber. 1997. Long shortterm memory. Neural Comput. , 9(8):1735-1780.
- Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR , arxiv:abs/1508.01991.
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR , abs/1412.6980. · arXiv
- Tibor Kiss and Jan Strunk. 2006. Unsupervised multilingual sentence boundary detection. Computational Linguistics , 32(4):485-525.
- John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning , ICML '01, pages 282-289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP) , pages 1532-1543.
- Kirk Roberts, Dina Demner-Fushman, and Joseph M. Tonning. 2017. Overview of the TAC 2017 adverse reaction extraction from drug labels track. In Proceedings of the 2017 Text Analysis Conference, TAC 2017, Gaithersburg, Maryland, USA, November 13-14, 2017 . NIST.
- Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. , 15(1):1929-1958.
- Rupesh Kumar Srivastava, Klaus Greff, and J¨ urgen Schmidhuber. 2015. Highway networks. CoRR , abs/1505.00387. · arXiv
- Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. CoRR , abs/1409.2329. · arXiv
- Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 1127-1137. Association for Computational Linguistics.
Cited by
Sorted by citations
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246
Walker, V. R., Schmitt, C. P., Wolfe, M. S., Nowak, A. J., Kulesza, K., Williams, A. R., Shin, R., Cohen, J., Burch, D., Stout, M. D., et al. (2022) · Environment International · doi:10.1016/j.envint.2021.107025 · Cited by 28
Self-citation
Metadata
- Authors
- Nowak, A., & Kunstman, P.
- ORCID
- Nowak, Artur
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1901.02081
- OpenAlex
- W2910788328
- Cited by
- 4
- References
- 1
- Citations by year
- 2024: 3 · 2021: 1
- FWCI
- 0.02
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- information extraction, systematic reviews, environmental agents, sequence labeling, BiLSTM-CRF, highway connection, pre-trained word embeddings, gloves, ELMo, deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents. https://app.cochat.ai/papers/10.48550/arxiv.1901.02081
- The paper
- Nowak, A., & Kunstman, P. (2019). Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents. arXiv. https://doi.org/10.48550/arxiv.1901.02081
