The DOI matches its Crossref record.
Message Understanding Conference-6: a brief history
Open accessAbstract
We have recently completed the sixth in a series of "Message Understanding Conferences" which are designed to promote and evaluate research in information extraction. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted. We describe some of the motivations for the new format and briefly discuss some of the results of the evaluations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Items 701–800 of 1,000
NEAR-DUPLICATE NEWS DETECTION USING NAMED ENTITIES
Uyar, E. (2009) · Bilkent University Institutional Repository (Bilkent University) · Cited by 6
Semantic Resources and their Applications in Hungarian Natural Language Processing
Miháltz, M. (2010) · Cited by 6
Moving Beyond the Turing Test with the Allen AI Science ChallengePreprint
Schoenick, C., Clark, P. E., Tafjord, O., Turney, P. D., & Etzioni, O. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.04315 · Cited by 6
Extracting knowledge from Twitter and the web
Ritter, A. (2012) · ResearchWorks at the University of Washington (University of Washington) · Cited by 6
Béchet, F. (2011) · doi:10.1002/9781119992691.ch10 · Cited by 6
Moussaoui, T. E., Loqman, C., & Boumhidi, J. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200487 · Cited by 9
Modeling of joint extraction of entity relationships in clinical electronic medical records
Yang, Y., Tong, J., & Zhou, Q. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109161 · Cited by 7
Ner4Opt: named entity recognition for optimization modelling from natural language
Kadıoğlu, S., Dakle, P. P., Uppuluri, K., Politi, R., Raghavan, P., Rallabandi, S. K., & Srinivasamurthy, R. (2024) · Constraints · doi:10.1007/s10601-024-09376-5 · Cited by 10
Synthetic4Health: generating annotated synthetic clinical letters
Ren, L., Belkadi, S., Han, L., Del-Pinto, W., & Nenadić, G. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1497130 · Cited by 12
Fernández, M., Zhang, Z., López, V., Uren, V. S., & Motta, E. (2011) · International Conference on Knowledge Capture · doi:10.1145/1999676.1999680 · Cited by 5
A co‐training based entity recognition approach for cross‐disease clinical documents
Chen, D., Che, N., Le, J., & Pan, Q. (2018) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.4505 · Cited by 5
CollabGraph: A Graph-Based Collaborative Search Summary Visualization
Ilkou, E., Tolmachova, T., Fisichella, M., & Taibi, D. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3242174 · Cited by 5
LegalNERo: A linked corpus for named entity recognition in the Romanian legal domain
Păiș, V., Mitrofan, M., Gasan, C. L., Ianov, A., Ghit, Ă. C., Coneschi, V. S., & Onuț, A. (2023) · Semantic Web · doi:10.3233/sw-233351 · Cited by 5
Analysis of the evolution of COVID-19 disease understanding through temporal knowledge graphs
Negro, A., Montagna, F., Teng, M. N., Neal, T. J., Thomas, S. W., King, S. L., & Khan, R. R. (2023) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2023.1204801 · Cited by 6
Improved Named Entity Tagset for Punjabi Language
Kaur, A., & Josan, G. S. (2014) · International Conference on Recent Advances in Engineering & Computational Sciences · doi:10.1109/raecs.2014.6799638 · Cited by 5
Evaluation of Punjabi Named Entity Recognition using Context Word Feature
Kaur, A., & Josan, G. S. (2014) · International Journal of Computer Applications · doi:10.5120/16913-7011 · Cited by 5
Topic Detection and Tracking interface with named entities approach
Bashaddadh, O. M. A., & Mohd, M. (2011) · International Conference on Semantic Technology and Information Retrieval (STAIR) · doi:10.1109/stair.2011.5995791 · Cited by 5
Transferring Frames: Utilization of Linked Lexical Resources
Borin, L., Forsberg, M., Johansson, R., Muhonen, K., Purtonen, T., & Voionmaa, K. (2012) · Cited by 5
Hong, L., Xu, F., & Uszkoreit, H. (2011) · Cited by 5
Fourie, W., Toit, J. V. D., & Snyman, D. P. (2014) · Cited by 5
The JUMP project: domain ontologies and linguistic knowledge @ work
Basile, P., de Gemmis, M., Iaquinta, L., Gentile, A. L., & Lops, P. (2007) · MADOC (University of Mannheim) · Cited by 5
Noam Chomsky's Contribution to Linguistics
Freidin, R. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0020 · Cited by 6
American Descriptivism (‘Structuralism’)
Blevins, J. P. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0019 · Cited by 5
Srivastava, A., Weber, S., Bourgonje, P., & Rehm, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73706-5_5 · Cited by 5
Anniversary article: Then and now: 25 years of progress in natural language engineering
Tait, J. I., & Wilks, Y. A. (2019) · Natural Language Engineering · doi:10.1017/s1351324919000081 · Cited by 5
Named Entity Recognition Using Part-of-Speech Rules for Telugu
Gorla, S., Velivelli, S., Satpathi, D. K., Murthy, N. L. B., & Malapati, A. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-981-13-9683-0_16 · Cited by 5
Recognition of Named Entities in the Russian Subcorpus Google Books Ngram
Bochkarev, V. V., Khristoforov, S. V., & Shevlyakova, A. V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60887-3_2 · Cited by 5
The first named entity recognizer in Maithili: Resource creation and system development
Priyadarshi, A., & Saha, S. K. (2021) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-210051 · Cited by 5
Seddik, K. M., & Farghaly, A. (2014) · Theory and applications of natural language processing · doi:10.1007/978-3-642-45358-8_8 · Cited by 5
Mariani, I., Karimi, M., Concilio, G., Rizzo, G. F., & Benincasa, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-16075-2_18 · Cited by 5
Expanding UlyssesNER-Br Named Entity Recognition Corpus with Informal User-Generated Text
Costa, R., Albuquerque, H. O., Silvestre, G. P., Silva, N., de Souza, E. P. R., Vitório, D., Nunes, A., Siqueira, F., Tarrega, J. P. M., Beinotti, J. V. P., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16474-3_62 · Cited by 5
Liu, J., Chen, Y., & Xu, J. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3210442 · Cited by 5
Ser, S. E., Shear, K., Snigurska, U. A., Prosperi, M., Wu, Y., Magoč, T., Bjarnadóttir, R. I., & Lucero, R. J. (2023) · JMIR Research Protocols · doi:10.2196/48521 · Cited by 6
Large Scale Text Mining Approaches for Information Retrieval and Extraction
Bellot, P., Bonnefoy, L., Bouvier, V., Duvert, F., & Kim, Y.-M. (2013) · Studies in computational intelligence · doi:10.1007/978-3-319-01866-9_1 · Cited by 5
Dimensions of Interpersonal Relationships: Corpus and Experiments
Rashid, F., & Blanco, E. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1244 · Cited by 5
Cross Script Hindi English NER Corpus from Wikipedia
Ansari, M. Z., Ahmad, T., & Ali, M. A. (2018) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-03146-6_116 · Cited by 5
A Formal Framework for Coupling Document Spanners with Ontologies
Scafoglieri, F., & Lembo, D. (2019) · IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike.2019.00036 · Cited by 5
Ontology-based Document Spanning Systems for Information Extraction
Lembo, D., & Scafoglieri, F. (2020) · International Journal of Semantic Computing · doi:10.1142/s1793351x20400012 · Cited by 5
Ruokolainen, T., & Kettunen, K. (2020) · Cited by 5
RETRACTED: Named Entity Recognition in English Text
Jain, R., Sharma, A. B., Mishra, G. S., Nand, P., & Chakraborty, S. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1712/1/012013 · Cited by 5
RetractedCross-Lingual Cross-Domain Nested Named Entity Evaluation on English Web Texts
Plank, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.158 · Cited by 5
Entity Ranking Based on Category Expansion
Jämsen, J., Näppilä, T., & Arvola, P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85902-4_24 · Cited by 5
Vlachidis, A., & Tudhope, D. S. (2015) · Data Technologies and Applications · doi:10.1108/prog-10-2014-0076 · Cited by 5
Accuracy and Suitability: New Challenges for Evaluation
King, M. (2005) · Computers and the Humanities · doi:10.1007/s10579-005-2695-2 · Cited by 5
Extraction of disease events for a real-time monitoring system
Nguyen, M.-T., & Nguyen, T.-T. (2013) · International Symposium on Information and Communication Technology · doi:10.1145/2542050.2542084 · Cited by 5
Employing named entities for semantic retrieval of news videos in Turkish
Küçük, D., & Yazıcı, A. (2009) · International Symposium on Computer and Information Sciences · doi:10.1109/iscis.2009.5291836 · Cited by 5
Automatic Web Information Extraction Based on Rules
Hu, F., Ruan, T., Shao, Z., & Ding, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24434-6_21 · Cited by 5
Identifying named entities on a University intranet
Althobaiti, M. J., Kruschwitz, U., & Poesio, M. (2012) · Computer Science and Electronic Engineering Conference · doi:10.1109/ceec.2012.6375385 · Cited by 5
Stalker: overcoming linguistic barriers in open source intelligence
Neri, F., Geraci, P., & Pettoni, M. (2010) · International Journal of Networking and Virtual Organisations · doi:10.1504/ijnvo.2011.037160 · Cited by 5
Semantic Integration in Learning from Text.
Bethard, S. J., Nielsen, R. D., Martin, J., Ward, W., & Palmer, M. S. (2007) · Cited by 5
Annotation documentaire et peuplement d'ontologie à partir d'extractions linguistiques
Florence Amardeilh, A., Laublet, P., & Minel, J.-L. (2005) · HAL · Cited by 5
Grouin, C. (2013) · HAL · Cited by 5
An Adaptable IE System to New Domains
Turmo, J., Català, N., & Rodríguez, H. (1999) · Applied Intelligence · doi:10.1023/a:1008332021052 · Cited by 5
Toward summarization of communicative activities in spoken conversation
Niekrasz, J. (2012) · ERA · Cited by 5
Using a Trie-based Structure for Question Analysis
Pizzato, L. A. S. (2004) · Cited by 5
Question answering and query processing for extended knowledge graphs
Yahya, M. (2016) · Max Planck Digital Library · doi:10.22028/d291-25428 · Cited by 5
Afaan Oromo Named Entity Recognition Using Hybrid Approach
Sani, A. (2015) · Cited by 5
MAVEN: A Massive General Domain Event Detection DatasetPreprint
Wang, X., Wang, Z., Han, X., Jiang, W., Han, R., Liu, Z., Li, J.-Z., Li, P., Lin, Y., & Zhou, J. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2004.13590 · Cited by 240
A Unified Generative Framework for Various NER SubtasksPreprint
Yan, H. R., Gui, T., Dai, J., Guo, Q., Zhang, Z., & Qiu, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01223 · Cited by 5
Ji, H. (2009) · Encyclopedia of Database Systems · doi:10.1007/978-0-387-39940-9_204 · Cited by 5
Rijcken, E., Zervanou, K., Mosteiro, P. J., Scheepers, F. E., Spruit, M. R., & Kaymak, U. (2024) · Research Square · doi:10.21203/rs.3.rs-2320804/v2 · Cited by 5
Automatic extraction of outbreak information from news
Liu, B., & Zhang, Y. (2008) · Cited by 5
Document-Level Event Argument Extraction with Sparse Representation Attention
Zhang, M., & Chen, H. (2024) · Mathematics · doi:10.3390/math12172636 · Cited by 4
Chinese medical named entity recognition integrating adversarial training and feature enhancement
Zhang, X., Kao, Y., Che, S., Yan, J., Zhou, S., Guo, S., & Wang, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-98465-3 · Cited by 4
Named Entity Disambiguation Based on Explicit Semantics
Jačala, M., & Tvarožek, J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-27660-6_37 · Cited by 4
The Semantics of Negation Detection in Archaeological Grey Literature
Vlachidis, A., & Tudhope, D. S. (2013) · Communications in computer and information science · doi:10.1007/978-3-319-03437-9_20 · Cited by 4
A Two-Stage Bi-LSTM Model for Chinese Company Name Recognition
Zeng, J., Che, J., Xing, C., & Zhang, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-94361-9_1 · Cited by 4
Improved Named Entity Recognition for Noisy Call Center Transcripts
Davidson, S. R., Hosier, J., Zhou, Y., & Gurbani, V. K. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.40 · Cited by 5
On the link between Education and Industry 4.0: a framework for a data-driven education design
Spada, I., Chiarello, F., Curreli, A., & Fantoni, G. (2022) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon52537.2022.9766534 · Cited by 4
Natural Language Query for Technical Knowledge Graph Navigation
Zhao, Z., Stewart, M. G., Liu, W., French, T., & Hodkiewicz, M. R. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-8746-5_13 · Cited by 4
Name Entity Recognition Tasks: Technologies and Tools
Lagutina, N. S., Vasilyev, A., & Zafievsky, D. D. (2023) · Modeling and Analysis of Information Systems · doi:10.18255/1818-1015-2023-1-64-85 · Cited by 4
Auction-Based Learning for Question Answering over Knowledge Graphs
Agrawal, G., Bertsekas, D. P., & Liu, H. (2023) · Information · doi:10.3390/info14060336 · Cited by 4
CHTopoNER model-based method for recognizing Chinese place names from social media information
Zhang, M., Liu, X., Zhang, Z., Qiu, Y., Jiang, Z., & Zhang, P. (2024) · Journal of Geographical Systems · doi:10.1007/s10109-023-00433-w · Cited by 4
Zhai, Z., Fan, R., Jie, H. J., Xiong, N. N., Zhang, L., Wan, J., & Zhang, L. (2024) · IEEE Access · doi:10.1109/access.2024.3405997 · Cited by 4
Flat and Nested Named Entity Recognition in Arabic Language
Moussaoui, T. E., Loqman, C., & Boumhidi, J. (2024) · International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) · doi:10.1109/iraset60544.2024.10548518 · Cited by 4
Effects of Multiple Annotation Schemes on Arabic Named Entity Recognition
Belhajem, I. (2024) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.8528 · Cited by 4
Named entity recognition for de-identifying Spanish electronic health records
Moreno-Barea, F. J., López-García, G., Mesa, H., Ribelles, N., Alba, E., Jerez, J. M., & Veredas, F. J. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109576 · Cited by 7
Tasks in Named Entity Recognition: Technologies and Tools
Lagutina, N. S., Vasilyev, A., & Zafievsky, D. D. (2024) · Automatic Control and Computer Sciences · doi:10.3103/s0146411624700251 · Cited by 4
OdNER: NER resource creation and system development for low-resource Odia language
Dalai, T., Das, A., Mishra, T. K., & Kumar, P. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100139 · Cited by 5
A Chinese Question Answering System for Specific Domain
Li, T., Hao, Y., Zhu, X., & Zhang, X. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08010-9_64 · Cited by 4
Grounding event references in news
Nothman, J. (2013) · UPT. Syiah Kuala University Library (Syiah Kuala University) · Cited by 4
Topic-Oriented Words as Features for Named Entity Recognition
Zhang, Z., Cohn, T., & Ciravegna, F. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37247-6_25 · Cited by 4
System design for location name recognition in ancient local chronicles
Bao, P., & Zhu, S. (2014) · Library Hi Tech · doi:10.1108/lht-07-2013-0101 · Cited by 4
Ontology-guided Extraction of Complex Nested Relationships
Pandit, S., & Honavar, V. (2010) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2010.98 · Cited by 4
Kulyukin, V. (2012) · The Open Rehabilitation Journal · doi:10.2174/1874943701205010022 · Cited by 4
Empirical Co-occurrence Rate Networks for Sequence Labeling
Zhu, Z., Hiemstra, D., Apers, P. M. G., & Wombacher, A. (2013) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2013.23 · Cited by 4
Les coréférences à l'oral : une expérience d'apprentissage automatique sur le corpus ANCOR
Désoyer, A., Landragin, F., Tellier, I., Lefeuvre-Halftermeyer, A., & Antoine, J.-Y. (2015) · HAL · Cited by 4
Name identification and extraction with formal concept analysis
Taghva, K. (2016) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-016-0514-2 · Cited by 4
Chesworth, D., Harmon, N., Tanner, L., Guerlain, S. A. E., & Balazs, M. (2016) · Systems and Information Engineering Design Symposium · doi:10.1109/sieds.2016.7489305 · Cited by 4
Orthography and the Early History of Phonetics
MacMahon, M. K. C. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0006 · Cited by 5
Deep Neural Query Understanding System at Expedia Group
Chandrasekaran, R., Pathak, H. N., & Yano, T. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378495 · Cited by 4
Improving Named Entity Recognition in Spoken Dialog Systems by Context and Speech Pattern Modeling
Nguyen, M., & Yu, Z. (2021) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2021.sigdial-1.6 · Cited by 4
HTML-LSTM: Information Extraction from HTML Tables in Web Pages Using Tree-Structured LSTM
Kawamura, K., & Yamamoto, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88942-5_3 · Cited by 4
CFinDEE: A Chinese Fine-Grained Financial Dataset for Document-Level Event Extraction
Zhang, T., Liu, M., & Zhou, B. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651921 · Cited by 4
Informed Named Entity Recognition Decoding for Generative Language Models
Deußer, T., Hillebrand, L., Bauckhage, C., & Sifa, R. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825603 · Cited by 13
Context Hidden Markov Model for Named Entity Recognition
Todorović, B. T., Rančić, S. R., & Mulalić, E. H. (2010) · Springer optimization and its applications · doi:10.1007/978-1-4419-6594-3_30 · Cited by 4
Research in information extraction
Grishman, R. (1996) · doi:10.3115/1119089.1119105 · Cited by 4
Self-citationConcept Lattice Mining for Unsupervised Named Entity Annotation
Girault, T. (2008) · Cited by 4
Indexing Names of Persons in a Large Dataset of a Newspaper
Pirovani, J. C. M., Nogueira, M., & de Oliveira, E. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99722-3_15 · Cited by 4
Putra, H. S., Mahendra, R., & Darari, F. (2019) · International Conference on Web Intelligence, Mining and Semantics · doi:10.1145/3326467.3326487 · Cited by 4
Metadata
- Authors
- Grishman, R., & Sundheim, B.
- Affiliations
- New York University (US)
- Venue
- Proceedings of the 16th conference on Computational linguistics -
- Published
- 1996
- Volume
- 1
- Pages
- 466
- ISSN
- 1525-2477 (ISSN-L)
- OpenAlex
- W2068882115
- Cited by
- 1,408
- References
- 0
- Citations by year
- 2026: 26 · 2025: 37 · 2024: 62 · 2023: 74 · 2022: 64 · 2021: 81 · 2020: 64 · 2019: 73 · 2018: 68 · 2017: 50 · 2016: 61 · 2015: 71 · 2014: 65 · 2013: 117 · 2012: 83
- Type
- conference abstract
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Computational Linguistics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- MUC6, information extraction, benchmark tasks, natural language processing evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
