The DOI matches its Crossref record.
Message Understanding Conference-6: a brief history
Open accessAbstract
We have recently completed the sixth in a series of "Message Understanding Conferences" which are designed to promote and evaluate research in information extraction. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted. We describe some of the motivations for the new format and briefly discuss some of the results of the evaluations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Items 801–900 of 1,000
Generalisation in Named Entity Recognition: A Quantitative Analysis
Augenstein, I., Derczynski, L., & Bontcheva, K. (2017) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1701.02877 · Cited by 4
A Study on Entity Resolution for Email Conversations
Dakle, P. P., Desai, T., & MOLDOVAN, D. I. (2020) · Language Resources and Evaluation · Cited by 5
Towards Few-Shot Event Mention Retrieval: An Evaluation Framework and A Siamese Network Approach
Min, B., Chan, Y. S., & Zhao, L. (2020) · Language Resources and Evaluation · Cited by 4
Knowledge-Based Named Entity Recognition of Archaeological Concepts in Dutch
Vlachidis, A., Tudhope, D. S., & Wansleeben, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-71903-6_6 · Cited by 4
Ali, B. A. B., Mihi, S., Ait‐Mlouk, A., Bazi, I. E., & Laachfoubi, N. (2022) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-211944 · Cited by 5
RetractedGarcía-Ferrero, I., Campos, J. A., Sainz, O., Salaberria, A., & Roth, D. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.186 · Cited by 5
Tracking Behavioral Construct Use through Citations: A Relation Extraction Approach
Li, J., & Larsen, K. R. T. (2013) · Cited by 4
QA@INEX Track 2011: Question Expansion and Reformulation Using the REG Summarization System
Vivaldi, J., & da Cunha, I. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35734-3_24 · Cited by 4
Cultural Knowledge for Named Entity Disambiguation: A Graph-Based Semantic Relatedness Approach
Gentile, A. L., Zhang, Z., Xia, L., & Iria, J. (2010) · Serdica Journal of Computing · doi:10.55630/sjc.2010.4.217-242 · Cited by 4
Analysis and robust extraction of changing named entities
Tsuchiya, M., Endo, S., & Nakagawa, S. (2009) · Named Entities Workshop (NEWS) · doi:10.3115/1699705.1699743 · Cited by 4
Temporal Annotation and Representation
Verhagen, M., & Moszkowicz, J. L. (2009) · Language and Linguistics Compass · doi:10.1111/j.1749-818x.2008.00116.x · Cited by 4
A semantic approach to temporal information processing
Martínez, H. (2012) · RUA, Repositorio Institucional de la Universidad de Alicante (Universidad de Alicante) · Cited by 4
On Modeling Learning Communities
Petrushyna, Z., Klamma, R., & Kravčík, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24258-3_19 · Cited by 4
Pham, Q. H., Nguyen, L.-M., Nguyen, B. T., & Cuong, N. V. (2015) · Named Entities Workshop (NEWS) · doi:10.18653/v1/w15-3907 · Cited by 4
Recognition and Classification of Numerical Entities in Basque
Soraluze, A., Alegria, I., Ansa, O., Arregi, O., & Arregi, X. (2011) · Cited by 4
Blessing, A., Glaser, A., & Kuhn, J. (2015) · Cited by 4
Rule based approach for text segmentation on Indonesian news article using named entity distribution
Saniati, & Purwarianti, A. (2014) · International Conference on Data and Software Engineering · doi:10.1109/icodse.2014.7062668 · Cited by 4
INFORMed PA: A NER for the Italian Public Administration Domain
Passaro, L., Lenci, A., & Gabbolini, A. (2017) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.2440 · Cited by 4
Zhang, H., Zhao, X., & Song, Y. (2021) · Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC) · doi:10.18653/v1/2021.crac-1.1 · Cited by 7
Coupled Hierarchical IR and Stochastic Models for Surface Information Extraction.
Zaragoza, H., & Gallinari, P. (1998) · Electronic workshops in computing · doi:10.14236/ewic/irsg1998.16 · Cited by 4
Named Entity Recognition Algorithm Based on Deep Learning
陈, 娟. (2021) · Computer Science and Application · doi:10.12677/csa.2021.113064 · Cited by 4
EventDNA: a dataset for Dutch news event extraction as a basis for news diversification
Colruyt, C., Clercq, O. D., Desot, T., & Hoste, V. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-022-09623-2 · Cited by 4
Training a Named Entity Recognizer on the Web
Urbansky, D., Thom, J. A., Schuster, D., & Schill, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24434-6_7 · Cited by 4
An Initial Proposal for Cooperative Evaluation on Information Retrieval in Portuguese
Aires, R., Aluísio, S. M., Quaresma, P., Santos, D., & Silva, M. J. (2003) · Lecture notes in computer science · doi:10.1007/3-540-45011-4_36 · Cited by 4
Acquisition of entity-relationship models from natural language specifications using heuristics
Omar, N., Hanna, P. R., & McKevitt, P. (2005) · Ulster University Research Portal (Ulster University) · Cited by 4
Szarvas, G. (2008) · Cited by 4
Resources and Techniques for Multilingual Information Extraction
Busemann, S., & Krieger, H. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2rxmn7tf62ge · Cited by 4
Automated classification of the narrative of medical reports using natural language processing
Goldstein, I. J. (2011) · doi:10.54014/7weq-ywav · Cited by 4
A Brief Survey and Comparative Study of Recent Development of Pronoun Coreference ResolutionPreprint
Zhang, H., Zhao, X., & Song, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12721 · Cited by 4
Information Extraction from Text for Improving Research on Small Molecules and Histone Modifications
Klein, C. (2011) · bonndoc (University of Bonn) · Cited by 4
The Construction of Knowledge Graphs in the Assembly Domain Based on Deep Learning
Qu, J., Li, Y., Du, H., Wang, W., & Fu, W. (2025) · IEEE Access · doi:10.1109/access.2025.3576168 · Cited by 3
Wang, Z., Li, X., Rui, X., Ding, L., & Li, J. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15030124 · Cited by 3
An Empirical Study of the Occurrence and Co-Occurrence of Named Entities in Natural Language Corpora
Saravanan, K., Choudhury, M., Udupa, R., & Kumaran, A. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3ashxybo3fr7 · Cited by 3
Zhong, X., & Cambria, E. (2021) · Socio-affective computing · doi:10.1007/978-3-030-78961-9_2 · Cited by 3
GeoNLPlify: A spatial data augmentation enhancing text classification for crisis monitoring
Decoupes, R., Mathieu Roche, A., & Teisseire, M. (2023) · Intelligent Data Analysis · doi:10.3233/ida-230040 · Cited by 3
Atherton, C., & Blank, D. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0014 · Cited by 3
Named Entity Recognition Using the Web
Rømcke, A., & Johansson, C. (2008) · DSpace repository (University of Tartu) · Cited by 3
L’extraction d’entités nommées : une opportunité pour le secteur culturel ?
Hengchen, S., van Hooland, S., Verborgh, R., & Wilde, M. D. (2015) · I2D - Information données & documents · doi:10.3917/i2d.152.0070 · Cited by 3
A local tree alignment approach to relation extraction of multiple arguments
Kim, S., Jeong, M., & Lee, G. G. (2011) · Information Processing & Management · doi:10.1016/j.ipm.2010.12.002 · Cited by 3
Text analysis and entity extraction in asymmetric threat response and prediction
Chan, E., Ginsburg, J., Eyck, B. T., Rozenblit, J. W., & Dameron, M. (2010) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2010.5484737 · Cited by 3
A New Way for Chinese Place Name Recognition
Zhang, Q., Yuan, Y., Li, N., Wei, X., & Miao, J. (2009) · International Conference on Asian Language Processing · doi:10.1109/ialp.2009.35 · Cited by 3
Everyone's Invited: A New Paradigm for Evaluation on Non-Transferable Datasets
Jurgens, D., Finethy, T., Armstrong, C., & Ruths, D. A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i4.14686 · Cited by 3
Loakes, D. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0007 · Cited by 3
Performance Evaluation of SVM-Based Amazighe Named Entity Recognition
Talha, M., Boulaknadel, S., & Aboutajdine, D. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-74690-6_23 · Cited by 3
Integrating Approximate String Matching with Phonetic String Similarity
Ferri, J., Tissot, H. C., & Fabro, M. D. D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98398-1_12 · Cited by 3
Predictable and Consistent Information Extraction
Kassaie, B., & Tompa, F. W. (2019) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3342558.3345391 · Cited by 3
Toward an NLP Approach for Transforming Paper Contracts into Smart Contracts
Aejas, B., Belhi, A., & Bouras, A. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-19-7663-6_70 · Cited by 3
Named Entity Recognition on Medical text by using Deep Nueral Networks
VeeraSekharReddy, B., Thatha, V. N., Biyyapu, N. S., Krishna, J. S. V. G., Sundaram, D. A., & Sandeep, D. (2023) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat59970.2023.10353439 · Cited by 3
Entity recognition based on heterogeneous graph reasoning of visual region and text candidate
Wang, X., Zhu, N., Li, J., Chang, Y., & Li, Z. (2024) · Machine Learning · doi:10.1007/s10994-023-06456-0 · Cited by 4
Named Entity Recognition in Government Audit Texts Based on ChineseBERT and Character-Word Fusion
Huang, B., Lin, Y., Pang, S., & Fu, L. (2024) · Applied Sciences · doi:10.3390/app14041425 · Cited by 3
Hartmann, B., Tamla, P., Freund, F., & Hemmje, M. (2023) · Irish Conference on Artificial Intelligence and Cognitive Science (AICS) · doi:10.1109/aics60730.2023.10470654 · Cited by 3
Metzger, V. T., Cannon, D. C., Yang, J. J., Mathias, S. L., Bologa, C., Waller, A., Schürer, S. C., Vidović, D., Kelleher, K. J., Sheils, T. K., et al. (2024) · PeerJ · doi:10.7717/peerj.17470 · Cited by 3
Selecting Features for Domain-Independent Named Entity Recognition
Tkachenko, M., & Simanovsky, A. (2012) · Cited by 3
Statistische Verfahren der Textanalyse
Granitzer, M. (2007) · Springer eBooks · doi:10.1007/3-540-29325-6_28 · Cited by 3
A Mixed Approach in Recognising Geographical Entities in Texts
Cristea, D., Gîfu, D., Pistol, I. C., Sfirnaciuc, D., & Niculiță, M. (2016) · Communications in computer and information science · doi:10.1007/978-3-319-32942-0_4 · Cited by 3
Event Detection with Convolutional Neural Networks for Forensic Investigation
Yang, B., Li, N., Lü, Z., & Jiang, J. (2016) · IFIP advances in information and communication technology · doi:10.1007/978-3-319-48390-0_11 · Cited by 3
Event Detection Based on Open Information Extraction and Ontology
Sahnoun, S., Elloumi, S., & Yahia, S. B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28377-3_20 · Cited by 3
Learning soft domain constraints in a factor graph model for template-based information extraction
ter Horst, H., Härtung, M., Cimiano, P., Brazda, N., Müller, H. W., & Klinger, R. (2019) · Data & Knowledge Engineering · doi:10.1016/j.datak.2019.101764 · Cited by 3
Kapetanios, E., Tatar, D., & Sacarea, C. (2013) · Natural Language Processing · doi:10.1201/b15472-19 · Cited by 3
Named entity recognition of power communication planning based on transformer
Weiyue, T. (2022) · IEEE Joint International Information Technology and Artificial Intelligence Conference (ITAIC) · doi:10.1109/itaic54216.2022.9836600 · Cited by 3
TipScreener: A Framework for Mining Tips for Online Review Readers
Luo, H., Song, W., & Zhou, W. (2022) · Journal of theoretical and applied electronic commerce research · doi:10.3390/jtaer17040087 · Cited by 3
Semi-automatic Construction of a Named Entity Dictionary Based on Active Learning
Song, Y., & Kim, H. (2014) · Lecture notes in electrical engineering · doi:10.1007/978-3-662-45402-2_10 · Cited by 3
A Parallel Corpus Labeled Using Open and Restricted Domain Ontologies
Boldrini, E., Ferrández, S., Izquierdo, R., Tomás, D., & González, J. L. V. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-00382-0_28 · Cited by 3
Knowledge-based Named Entity Recognition in Polish
Smywiński-Pohl, A. (2015) · Cited by 3
Methods in biomedical text mining
Rzhetsky, A., & Rodriguez‐Esteban, R. (2007) · Cited by 3
Reasoning and Ontologies in Data Extraction
Flesca, S., Furche, T., & Oro, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33158-9_5 · Cited by 3
DESRM: a disease extraction system for real-time monitoring
Nguyen, M.-T., & Nguyen, T. T. (2015) · International Journal of Computational Vision and Robotics · doi:10.1504/ijcvr.2015.071341 · Cited by 3
Medical informatics methods for the clinical evidence extraction
Song, M. H., Park, D. K., & Lee, Y. H. (2012) · Journal of Korean Medical Association · doi:10.5124/jkma.2012.55.8.741 · Cited by 3
Tree Representations in Probabilistic Models for Extended Named Entities Detection
Dinarelli, M., & Rosset, S. (2012) · Cited by 3
Fine Tuning Features and Post-processing Rules to Improve Named Entity Recognition
Ferrández, Ó., Toral, A., & Muñoz, R. (2006) · Lecture notes in computer science · doi:10.1007/11765448_16 · Cited by 3
Design and evaluation of an Interactive Topic Detection and Tracking interface
Mohd, M. (2010) · doi:10.48730/0z8x-h335 · Cited by 3
Event Extraction using Structured Learning and Rich Domain Knowledge
Minkov, E. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2801131 · Cited by 3
Aliprandi, C., & Marchetti, A. (2011) · Communications in computer and information science · doi:10.1007/978-3-642-22098-2_96 · Cited by 3
A Hybrid Framework for Semantic Relation Extraction over Enterprise Data
Shen, W., Wang, J., Luo, P., & Wang, M. (2015) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.2015070101 · Cited by 3
Buco, Q. L. L., Capcap, J. L. L., Hermocilla, J. C. A., Yumul, C. S., Sagum, R. A., & Pastrana, A. G. (2013) · Journal of Industrial and Intelligent Information · doi:10.12720/jiii.1.1.1-5 · Cited by 3
Temporal Information Processing: A Survey
Berrazega, I. (2012) · International Journal on Natural Language Computing · doi:10.5121/ijnlc.2012.1201 · Cited by 3
Assuming Facts Are Expressed More Than Once.
Betteridge, J., Ritter, A., & Mitchell, T. M. (2014) · The Florida AI Research Society · Cited by 3
NERosetta for the Named Entity Multi-lingual Space
Krstev, C., Zečević, A., Vitas, D., & Kyriacopoulou, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-43808-5_25 · Cited by 3
The Semantic Web for Knowledge and Data Management
(2009) · IGI Global eBooks · doi:10.4018/978-1-60566-028-8 · Cited by 4
MARQUES, C. A. N., Matsuno, I. P., Sinoara, R. A., Rezende, S. O., & Rozenfeld, H. (2015) · Product Management & Development · doi:10.4322/pmd.2015.007 · Cited by 3
Extraction and Disambiguation of Name of Place from Tourism Blogs
Nakatoh, T., Yin, C., & Hirokawa, S. (2011) · ACIS International Symposium on Software and Network Engineering (SSNE) · doi:10.1109/ssne.2011.29 · Cited by 3
Named entities in indexing: A case study of TV subtitles and metadata records
Husevåg, A.-S. R. (2016) · NORA - Norwegian Open Research Archives · Cited by 3
Towards adaptation of named entity classification
Manchanda, P., Fersini, E., Palmonari, M., Nozza, D., & Messina, E. (2017) · ACM Symposium on Applied Computing · doi:10.1145/3019612.3022188 · Cited by 3
General or Universal Grammar from Plato to Chomsky
Maat, J. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0018 · Cited by 3
Poibeau, T., Arcouteil, A., & Grouin, C. (2002) · Cited by 3
Entity Typing Using Distributional Semantics and DBpedia
van Erp, M., & Vossen, P. T. J. M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68723-0_9 · Cited by 3
On Semantic Relation Extraction Over Enterprise Data
Shen, W., Wang, J., Luo, P., & Wang, M. (2017) · Advances in web technologies and engineering book series · doi:10.4018/978-1-5225-5042-6.ch003 · Cited by 3
Capisco: low-cost concept-based access to digital libraries
Hinze, A., Bainbridge, D., Cunningham, S. J., Taube-Schock, C., Matamua, R., Downie, J. S., & RASMUSSEN, E. M. (2018) · International Journal on Digital Libraries · doi:10.1007/s00799-018-0232-3 · Cited by 3
An adaptive model for sequential labeling systems
Elloumi, S. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7558-8 · Cited by 3
Weakly Supervised Attention Networks for Entity Recognition
Patra, B., & Moniz, J. R. A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1652 · Cited by 2
Probabilistic vs deep learning based approaches for narrow domain NER in Spanish
Ramos-Flores, O., Pinto, D., Montes-y-Gómez, M., & Vázquez, A. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-179868 · Cited by 3
TEI-friendly annotation scheme for medieval named entities: a case on a Spanish medieval corpus
Mellado, E. Á., Platas, M. L. D., Fabo, P. R., Sabel, H. B., Ros, S., & González‐Blanco, E. (2021) · Language Resources and Evaluation · doi:10.1007/s10579-020-09516-2 · Cited by 3
Extraction Method of Judicial Language Entities Based On Regular Expression
Jiao, K., & Li, X. (2021) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp51882.2021.9408748 · Cited by 3
Agarwal, O., & Nenkova, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.349 · Cited by 3
Distantly Supervised Named Entity Recognition with Spy-PU Algorithm
Zheng, H., Yu, H., Hao, Y., Wu, Y., & Li, S. (2021) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml52754.2021.9520707 · Cited by 3
RoBERTa and Stacked Bidirectional GRU for Fine-grained Chinese Named Entity Recognition
Li-zhou, L., Zheng, S., & Wang, Q. (2021) · International Conference on Mathematics and Artificial Intelligence (ICMAI) · doi:10.1145/3460569.3460576 · Cited by 3
Adopting ontologies for multisource identity resolution
Yankova, M., Saggion, H., & Cunningham, H. (2008) · doi:10.1145/1452567.1452573 · Cited by 3
EXTENSIVE EVALUATION OF EFFICIENT NLP-DRIVEN TEXT CLASSIFICATION
Basili, R., Moschitti, A., & Pazienza, M. T. (2006) · Applied Artificial Intelligence · doi:10.1080/08839510600753725 · Cited by 3
Language, the Mind, and the Brain
Garnham, A. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0031 · Cited by 3
A Natural Language Interface to Ontology-Based Knowledge Bases
Paredes‐Valverde, M. A., Noguera-Arnaldos, J. Á., Rodríguez-Enríquez, C. A., Valencia‐García, R., & Alor‐Hernández, G. (2015) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-19638-1_1 · Cited by 3
Metadata
- Authors
- Grishman, R., & Sundheim, B.
- Affiliations
- New York University (US)
- Venue
- Proceedings of the 16th conference on Computational linguistics -
- Published
- 1996
- Volume
- 1
- Pages
- 466
- ISSN
- 1525-2477 (ISSN-L)
- OpenAlex
- W2068882115
- Cited by
- 1,408
- References
- 0
- Citations by year
- 2026: 26 · 2025: 37 · 2024: 62 · 2023: 74 · 2022: 64 · 2021: 81 · 2020: 64 · 2019: 73 · 2018: 68 · 2017: 50 · 2016: 61 · 2015: 71 · 2014: 65 · 2013: 117 · 2012: 83
- Type
- conference abstract
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Computational Linguistics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- MUC6, information extraction, benchmark tasks, natural language processing evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
