Message Understanding Conference-6: a brief history
Open accessAbstract
We have recently completed the sixth in a series of "Message Understanding Conferences" which are designed to promote and evaluate research in information extraction. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted. We describe some of the motivations for the new format and briefly discuss some of the results of the evaluations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Still being checked.
Cited by
Sorted by citations
A survey of named entity recognition and classification
Nadeau, D., & Sekine, S. (2007) · Lingvisticae Investigationes · doi:10.1075/li.30.1.03nad · Cited by 2,507 · see citing works
A Survey on Deep Learning for Named Entity Recognition
Li, J., Sun, A., Han, J., & Li, C. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981314 · Cited by 1,667 · see citing works
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,258 · see citing works
Nasukawa, T., & Yi, J. (2003) · International Conference on Knowledge Capture · doi:10.1145/945645.945658 · Cited by 1,167 · see citing works
Large-Scale Named Entity Disambiguation Based on Wikipedia Data
Cucerzan, S. (2007) · Cited by 1,064 · see citing works
A shortest path dependency kernel for relation extraction
Bunescu, R., & Mooney, R. J. (2005) · doi:10.3115/1220575.1220666 · Cited by 971 · see citing works
Sarawagi, S. (2008) · Foundations and Trends in Databases · doi:10.1561/1900000003 · Cited by 731 · see citing works
GATE, a General Architecture for Text Engineering
Cunningham, H. (2002) · Computers and the Humanities · doi:10.1023/a:1014348124664 · Cited by 643 · see citing works
Aggarwal, C. C. (2015) · Springer eBooks · doi:10.1007/978-3-319-14142-8_13 · Cited by 626 · see citing works
Open domain event extraction from twitter
Ritter, A., Mausam, M., Etzioni, O., & Clark, S. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339704 · Cited by 596 · see citing works
Active Hidden Markov Models for Information Extraction
Scheffer, T., Decomain, C., & Wrobel, S. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44816-0_31 · Cited by 559 · see citing works
Arabic Natural Language Processing: Challenges and Solutions
Farghaly, A., & Shaalan, K. (2009) · ACM Transactions on Asian Language Information Processing · doi:10.1145/1644879.1644881 · Cited by 557 · see citing works
A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelsPreprint
Yadav, V., & Bethard, S. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11470 · Cited by 542 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
Extracting medication information from clinical text
Uzuner, Ö., Solti, I., & Cadag, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003947 · Cited by 466 · see citing works
Evaluating the State-of-the-Art in Automatic De-identification
Uzuner, Ö., Luo, Y., & Peter Szolovits, B. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2444 · Cited by 459 · see citing works
Test Collection Based Evaluation of Information Retrieval Systems
Sanderson, M. (2010) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000009 · Cited by 447 · see citing works
Information extraction: Techniques and challenges
Grishman, R. (1997) · Lecture notes in computer science · doi:10.1007/3-540-63438-x_2 · Cited by 437 · see citing works
Self-citationEvent Extraction by Answering (Almost) Natural Questions
Du, X., & Cardie, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.49 · Cited by 437 · see citing works
Will, G. K., King, G., Lowe, W., Abbott, T. T. M., Bond, D., Bond, J., Bradford, G., Cobb, C., Pevehouse, J., Quinn, K., et al. (2003) · Cited by 401 · see citing works
Identifying Patient Smoking Status from Medical Discharge Records
Uzuner, Ö., Goldstein, I., Luo, Y., & Kohane, I. S. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2408 · Cited by 359 · see citing works
Analysis of named entity recognition and linking for tweets
Derczynski, L., Maynard, D., Rizzo, G. F., van Erp, M., Gorrell, G., Raphaël Troncy, A., Petrak, J., & Bontcheva, K. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.10.006 · Cited by 340 · see citing works
Recent Named Entity Recognition and Classification techniques: A systematic review
Goyal, A., Gupta, V., & Kumar, M. (2018) · Computer Science Review · doi:10.1016/j.cosrev.2018.06.001 · Cited by 308 · see citing works
A Unified Generative Framework for Various NER Subtasks
Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., & Qiu, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.451 · Cited by 299 · see citing works
“The term “Named Entity” was coined in the Sixth Message Understanding Conference (MUC-6) (Gr-ishman and Sundheim, 1996).”
BackgroundChapman, W. W., Nadkarni, P. M., Hirschman, L., D'Avolio, L. W., Savova, G. K., & Uzuner, O. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000465 · Cited by 296 · see citing works
Information retrieval and text mining technologies for chemistry
Krallinger, M., Rabal, O., Lourenço, A., Oyarzabal, J., & Valencia, A. (2017) · Chemical Reviews · doi:10.1021/acs.chemrev.6b00851 · Cited by 281 · see citing works
Impact of word embedding models on text analytics in deep learning environment: a review
Asudani, D. S., Nagwani, N. K., & Singh, P. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10419-1 · Cited by 274 · see citing works
Named Entity Recognition: Fallacies, challenges and opportunities
Marrero, M., Urbano, J., Sánchez-Cuadrado, S., Morato, J., & Gómez-Berbís, J. M. (2012) · Computer Standards & Interfaces · doi:10.1016/j.csi.2012.09.004 · Cited by 242 · see citing works
Recognizing Obesity and Comorbidities in Sparse Data
Uzuner, Ö. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3115 · Cited by 231 · see citing works
Named Entity Extraction for Knowledge Graphs: A Literature Overview
Al-Moslmi, T., Ocaña, M. G., Opdahl, A. L., & Veres, C. (2020) · IEEE Access · doi:10.1109/access.2020.2973928 · Cited by 238 · see citing works
Text Mining for Big Data Analysis in Financial Sector: A Literature Review
Bach, M. P., Krstić, Ž., Seljan, S., & Turulja, L. (2019) · Sustainability · doi:10.3390/su11051277 · Cited by 235 · see citing works
Mind the GAP: A balanced corpus of gendered ambiguous pronouns
Webster, K., Recasens, M., Axelrod, V., & Baldridge, J. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00240 · Cited by 223 · see citing works
Information extraction: beyond document retrieval
Gaizauskas, R., & Wilks, Y. A. (1998) · Journal of Documentation · doi:10.1108/eum0000000007162 · Cited by 214 · see citing works
Information Extraction: Past, Present and Future
Jakub Piskorski, B., & Yangarber, R. (2012) · Theory and applications of natural language processing · doi:10.1007/978-3-642-28569-1_2 · Cited by 200 · see citing works
Computational methods for Traditional Chinese Medicine: A survey
Lukman, S., He, Y., & Hui, S.-C. (2007) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2007.09.008 · Cited by 195 · see citing works
A Review of Relation Extraction
Yang, Y., Xun, H., & You, H. (2013) · Shuju fenxi yu zhishi faxian · doi:10.11925/infotech.1003-3513.2013.11.05 · Cited by 193 · see citing works
Named Entity Recognition and Classification in Historical Documents: A Survey
Ehrmann, M., Hamdi, A., Pontes, E. L., Romanello, M., & Doucet, A. (2023) · ACM Computing Surveys · doi:10.1145/3604931 · Cited by 216 · see citing works
Automatic paraphrase acquisition from news articles
Shinyama, Y., Sekine, S., & Sudo, K. (2002) · International Conference on Human Language Technology Research · doi:10.3115/1289189.1289218 · Cited by 188 · see citing works
Language, Culture, and Society
Deumert, A. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0030 · Cited by 184 · see citing works
The Oxford Handbook of the History of Linguistics
(2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.001.0001 · Cited by 175 · see citing works
Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and ApproachesPreprint
Storks, S., Gao, Q., & Chai, J. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01172 · Cited by 234 · see citing works
Definition, Dictionaries and Tagger for Extended Named Entity Hierarchy
Sekine, S., & Nobata, C. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3zc57kdyb4p6 · Cited by 169 · see citing works
Information Extraction from Text
Jiang, J. (2012) · Springer eBooks · doi:10.1007/978-1-4614-3223-4_2 · Cited by 166 · see citing works
A Replicable Comparison Study of NER Software: StanfordNLP, NLTK, OpenNLP, SpaCy, Gate
Schmitt, X., Sylvain Kubler, A., Robert, J., Papadakis, M., & Traon, Y. L. (2019) · International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams.2019.8931850 · Cited by 161 · see citing works
MAVEN: A Massive General Domain Event Detection Dataset
Wang, X., Wang, Z., Han, X., Jiang, W., Han, R., Liu, Z., Li, J., Li, P., Lin, Y., & Zhou, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.129 · Cited by 166 · see citing works
Locating complex named entities in web text
Downey, D., Broadhead, M., & Etzioni, O. (2007) · Cited by 157 · see citing works
Itkonen, E. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0035 · Cited by 153 · see citing works
Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network
Wu, Y., Min, J., Jianbo, L., & Xu, H. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-564-7-624 · Cited by 148 · see citing works
SystemT: An Algebraic Approach to Declarative Information Extraction
Chiticariu, L., Krishnamurthy, R., Li, Y., Raghavan, S., Reiss, F. R., & Vaithyanathan, S. (2010) · Cited by 146 · see citing works
Information Extraction, Automatic
Cunningham, H. (2006) · Elsevier eBooks · doi:10.1016/b0-08-044854-2/00960-3 · Cited by 146 · see citing works
SUMMAC: a text summarization evaluation
Mani, I., Klein, G. L., HOUSE, D., Hirschman, L., Firmin, T., & Sundheim, B. M. (2002) · Natural Language Engineering · doi:10.1017/s1351324901002741 · Cited by 143 · see citing works
Self-citationOn-demand information extraction
Sekine, S. (2006) · International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) · doi:10.3115/1273073.1273167 · Cited by 140 · see citing works
Developing a corpus of plagiarised short answers
Clough, P. D., & Stevenson, M. (2010) · Language Resources and Evaluation · doi:10.1007/s10579-009-9112-1 · Cited by 139 · see citing works
CM-Builder: A Natural Language-Based CASE Tool for Object-Oriented Analysis
Harmain, H. M., & Gaizauskas, R. (2003) · Automated Software Engineering · doi:10.1023/a:1022916028950 · Cited by 139 · see citing works
Mixed-initiative development of language processing systems
Day, D. S., Aberdeen, J. S., Hirschman, L., Kozierok, R., Robinson, P., & Vilain, M. (1997) · Conference on Applied Natural Language Processing · doi:10.3115/974557.974608 · Cited by 139 · see citing works
A survey on Named Entity Recognition — datasets, tools, and methodologies
Jehangir, B., Radhakrishnan, S., & Agarwal, R. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100017 · Cited by 138 · see citing works
Liberal Event Extraction and Event Schema Induction
Huang, L., Cassidy, T., Feng, X., Ji, H., Voss, C. R., Han, J., & Sil, A. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1025 · Cited by 137 · see citing works
Fine-Grained Named Entity Recognition in Legal Documents
Leitner, E., Rehm, G., & Schneider, J. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33220-4_20 · Cited by 139 · see citing works
Weakly Supervised Extraction of Computer Security Events from Twitter
Ritter, A., Wright, E., Casey, W., & Mitchell, T. M. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741083 · Cited by 135 · see citing works
Cunningham, H., Humphreys, K., Gaizauskas, R., & Wilks, Y. A. (1997) · Conference on Applied Natural Language Processing · doi:10.3115/974281.974299 · Cited by 132 · see citing works
Mining opinion components from unstructured reviews: A review
Khan, K., Baharudin, B., Khan, A., & Ullah, A. (2014) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2014.03.009 · Cited by 131 · see citing works
Extraction of Adverse Drug Effects from Clinical Records
Aramaki, E., Miura, Y., Tonoike, M., Ohkuma, T., Masuichi, H., Waki, K., & Ohe, K. (2010) · Studies in health technology and informatics · doi:10.3233/978-1-60750-588-4-739 · Cited by 130 · see citing works
Metrics for Evaluation of Ontology-based Information Extraction
Maynard, D., Peters, W., & Li, Y. (2006) · Cited by 128 · see citing works
Hirst, G. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0033 · Cited by 126 · see citing works
Biomedical named entity recognition using deep neural networks with contextual information
Cho, H., & Lee, H. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3321-4 · Cited by 125 · see citing works
Tsai, R. T., Sung, C.-L., Dai, H.-J., Hung, H.-C., Sung, T.-Y., & Hsu, W.-L. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s5-s11 · Cited by 125 · see citing works
Generalisation in named entity recognition: A quantitative analysis
Augenstein, I., Derczynski, L., & Bontcheva, K. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.01.012 · Cited by 123 · see citing works
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 159 · see citing works
CASIE: Extracting Cybersecurity Event Information from Text
Satyapanich, T., Ferraro, F., & Finin, T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6401 · Cited by 119 · see citing works
A Survey on Recent Named Entity Recognition and Relationship Extraction Techniques on Clinical Texts
Bose, P., Srinivasan, S., Sleeman, W. C., Palta, J. R., Kapoor, R., & Ghosh, P. (2021) · Applied Sciences · doi:10.3390/app11188319 · Cited by 117 · see citing works
Landolsi, M. Y., Hlaoua, L., & Romdhane, L. B. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01779-1 · Cited by 125 · see citing works
Knowledge Graphs: A Practical Review of the Research Landscape
Kejriwal, M. (2022) · Information · doi:10.3390/info13040161 · Cited by 113 · see citing works
A statistical information extraction system for Turkish
Tür, G., Hakkani-Tür, D. Z., & Oflazer, K. (2003) · Natural Language Engineering · doi:10.1017/s135132490200284x · Cited by 111 · see citing works
Overview of the BioCreative III Workshop
Arighi, C. N., Lu, Z., Krallinger, M., Cohen, K. B., Wilbur, W. J., Valencia, A., Hirschman, L., & Wu, C. H. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s8-s1 · Cited by 110 · see citing works
Information extraction for enhanced access to disease outbreak reports
Grishman, R., Huttunen, S., & Yangarber, R. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00013-3 · Cited by 108 · see citing works
Self-citationKLUE: Korean Language Understanding EvaluationPreprint
Park, S., Moon, J., Kim, S., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09680 · Cited by 153 · see citing works
Word-Based Morphology from Aristotle to Modern WP (Word and Paradigm Models)
Blevins, J. P. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0017 · Cited by 107 · see citing works
Artificial intelligence: revolutionizing cardiology with large language models
Boonstra, M., Weissenbacher, D., Moore, J. H., Gonzalez‐Hernandez, G., & Asselbergs, F. W. (2023) · European Heart Journal · doi:10.1093/eurheartj/ehad838 · Cited by 107 · see citing works
Ontology-based information extraction and integration from heterogeneous data sources
Paul Buitelaar, C., Cimiano, P., Frank, A., Härtung, M., & Racioppa, S. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2008.07.007 · Cited by 104 · see citing works
Named Entity Recognition and Resolution in Legal Text
Dozier, C., Kondadadi, R., Light, M., Vachher, A., Veeramachaneni, S., & Wudali, R. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12837-0_2 · Cited by 102 · see citing works
Aiding Pronoun Translation with Co-Reference Resolution
Nagard, R. L., & Koehn, P. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 100 · see citing works
A factuality profiler for eventualities in text
Pustejovsky, J. D., & Sauri, R. S. (2008) · Cited by 99 · see citing works
Event graphs for information retrieval and multi-document summarization
Glavas, G., & Šnajder, J. (2014) · Expert systems with applications · doi:10.1016/j.eswa.2014.04.004 · Cited by 101 · see citing works
Finite-state transducer cascades to extract named entities in texts
Friburger, N., & Maurel, D. (2003) · Theoretical Computer Science · doi:10.1016/j.tcs.2003.10.007 · Cited by 93 · see citing works
Fagin, R., Kimelfeld, B., Reiss, F., & Vansummeren, S. (2015) · Journal of the ACM · doi:10.1145/2699442 · Cited by 61 · see citing works
Louvan, S., & Magnini, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.42 · Cited by 94 · see citing works
An information extraction core system for real world German text processing
Neumann, G., Backofen, R., Baur, J., Becker, M., & Braun, C. (1997) · Conference on Applied Natural Language Processing · doi:10.3115/974557.974588 · Cited by 91 · see citing works
An Attentive Neural Architecture for Fine-grained Entity Type Classification
Shimaoka, S., Stenetorp, P., Inui, K., & Riedel, S. (2016) · Automated Knowledge Base Construction · doi:10.18653/v1/w16-1313 · Cited by 89 · see citing works
Nested Named Entity Recognition: A Survey
Wang, Y., Tong, H., Zhu, Z., & Li, Y. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3522593 · Cited by 89 · see citing works
Han, X., Gao, T., Lin, Y., Peng, H. M., Yang, Y., Xiao, C., Liu, Z., Li, P., Zhou, J., & Sun, M. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.75 · Cited by 91 · see citing works
Evaluating and Combining Name Entity Recognition Systems
Jiang, R., Banchs, R. E., & Li, H. (2016) · Named Entities Workshop (NEWS) · doi:10.18653/v1/w16-2703 · Cited by 87 · see citing works
Software Architecture for Language Engineering
Cunningham, H., & Scott, D. (2004) · Natural Language Engineering · doi:10.1017/s1351324904003481 · Cited by 84 · see citing works
Surdeanu, M. (2013) · Cited by 83 · see citing works
An Overview of Named Entity Recognition
Sun, P., Yang, X., Zhao, X., & Wang, Z. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629225 · Cited by 83 · see citing works
What is Event Knowledge Graph: A Survey
Guan, S., Cheng, X., Bai, L., Zhang, F., Li, Z., Zeng, Y., Jin, X., & Guo, J. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3180362 · Cited by 83 · see citing works
Malay Named Entity Recognition Based on Rule-Based Approach
Alfred, R., Leong, L. C., On, C. K., & Anthony, P. (2014) · International Journal of Machine Learning and Computing · doi:10.7763/ijmlc.2014.v4.428 · Cited by 82 · see citing works
CM-Builder: an automated NL-based CASE tool
Harmain, H. M., & Gaizauskas, R. (2000) · doi:10.1109/ase.2000.873649 · Cited by 82 · see citing works
Event coreference for information extraction
Humphreys, K., Gaizauskas, R., & Azzam, S. (1997) · doi:10.3115/1598819.1598830 · Cited by 81 · see citing works
Computational Modeling of Narrative
Mani, I. (2013) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-02147-3 · Cited by 80 · see citing works
Nadeau, D. (2007) · doi:10.20381/ruor-19854 · Cited by 79 · see citing works
- Authors
- Grishman, R., & Sundheim, B.
- Affiliations
- New York University (US)
- Venue
- Proceedings of the 16th conference on Computational linguistics -
- Published
- 1996
- Volume
- 1
- Pages
- 466
- ISSN
- 1525-2477 (ISSN-L)
- OpenAlex
- W2068882115
- Cited by
- 1,408
- References
- 0
- Citations by year
- 2026: 26 · 2025: 37 · 2024: 62 · 2023: 74 · 2022: 64 · 2021: 81 · 2020: 64 · 2019: 73 · 2018: 68 · 2017: 50 · 2016: 61 · 2015: 71 · 2014: 65 · 2013: 117 · 2012: 83
- Type
- conference abstract
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Computational Linguistics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- MUC6, information extraction, benchmark tasks, natural language processing evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
