The DOI matches its Crossref record.
Message Understanding Conference-6: a brief history
Open accessAbstract
We have recently completed the sixth in a series of "Message Understanding Conferences" which are designed to promote and evaluate research in information extraction. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted. We describe some of the motivations for the new format and briefly discuss some of the results of the evaluations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Items 601–700 of 1,000
Arretium or Arezzo? A Neural Approach to the Identification of Place Names in Historical Texts
Sprugnoli, R. (2018) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.3627 · Cited by 8
BiLSTM-CRF Model for Named Entity Recognition in Railway Accident and Fault Analysis Report
Li, X., Shi, T., Li, P., Yang, L., & Ma, X. (2018) · doi:10.1145/3321619.3321623 · Cited by 8
Kim, Y.-J., Heider, P. M., Lally, I. R., & Meystre, S. M. (2021) · JMIR Medical Informatics · doi:10.2196/22797 · Cited by 8
Introducing translation dictionary into phrase-based SMT
Okuma, H., Yamamoto, H., & Sumita, E. (2007) · Cited by 8
Kanagavalli, V. R., & Raja, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.0445 · Cited by 8
Cascallar, E. C., Musso, M., Kyndt, E., & Dochy, F. J. R. C. (2015) · Frontline Learning Research · doi:10.14786/flr.v2i5.135 · Cited by 8
Predicting Linguistic Structure with Incomplete and Cross-Lingual Supervision
Täckström, O. (2013) · Diva portal (Dalarna University Library) · Cited by 8
Modelling Entity Instantiations
McKinlay, A., & Markert, K. (2011) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 8
Towards the Necessity for Debiasing Natural Language Inference DatasetsPreprint
Mithun, M. (2020) · doi:10.17605/osf.io/szdkn · Cited by 8
Applying Coreference to Improve Name Recognition
Ji, H., & Grishman, R. (2004) · Cited by 8
Self-citationHempelmann, C. F. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch004 · Cited by 7
Lohani, M., Do, A. S., & Aisa, A. (2024) · Frontiers in Future Transportation · doi:10.3389/ffutr.2024.1458828 · Cited by 7
Rosales-Méndez, H., Hogan, A., & Poblete, B. (2020) · Journal of Web Semantics · doi:10.1016/j.websem.2020.100600 · Cited by 7
BIBC: A Chinese Named Entity Recognition Model for Diabetes Research
Yang, L., Fu, Y., & Dai, Y. (2021) · Applied Sciences · doi:10.3390/app11209653 · Cited by 7
Research on Chinese Semantic Named Entity Recognition in Marine Engine Room Systems Based on BERT
Shen, H., Cao, H., Sun, G., & Chen, D. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11071266 · Cited by 7
Han, D., Wang, Z., Li, Y., ma, X., & Zhang, J. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01551-8 · Cited by 11
Lightweight Named Entity Extraction for Korean Short Message Service Text
Seon, C.-N., Yoo, J., Kim, H., Kim, J., & Seo, J. (2011) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2011.03.006 · Cited by 7
Keyword extraction from Arabic legal texts
Rammal, M., Bahsoun, Z., & Jabbour, M. A. A. (2015) · Interactive Technology and Smart Education · doi:10.1108/itse-11-2013-0030 · Cited by 7
XCS-SL: a rule-based genetic learning system for sequence labeling
Nakata, M., Kovacs, T., & Takadama, K. (2015) · Evolutionary Intelligence · doi:10.1007/s12065-015-0127-9 · Cited by 7
Utilising Wikipedia for Text Mining Applications
Qureshi, M. A. (2016) · ACM SIGIR Forum · doi:10.1145/2888422.2888449 · Cited by 7
Automation of database design through semantic analysis
Omar, N., Hassan, R. B., Arshad, H., & Sahran, S. (2008) · Cited by 7
The Role of Named Entities in Linking News Articles During Preservation
Khan, M., Rahman, A. U., Ullah, M., & Naseem, R. (2019) · Smart innovation, systems and technologies · doi:10.1007/978-3-030-21005-2_5 · Cited by 7
ScienceExamCER: A High-Density Fine-Grained Science-Domain Corpus for Common Entity Recognition
Smith, H., Zhang, Z., Culnan, J., & Jansen, P. A. (2019) · UA Campus Repository (The University of Arizona) · doi:10.48550/arxiv.1911.10436 · Cited by 7
Recent Progress of Named Entity Recognition over the Most Popular Datasets
Albared, M., Ocaña, M. G., Ghareb, A. S., & Al-Moslmi, T. (2019) · International Conference of Intelligent Computing and Engineering (ICOICE) · doi:10.1109/icoice48418.2019.9035170 · Cited by 10
Datasets and Performance Metrics for Greek Named Entity Recognition
Bartziokas, N., Mavropoulos, T., & Kotropoulos, C. L. (2020) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3411408.3411437 · Cited by 8
A method based on multi-standard active learning to recognize entities in electronic medical record
Pan, Q., Huang, C., & Chen, D. (2021) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2021054 · Cited by 7
Intelligent BERT-BiLSTM-CRF Based Legal Case Entity Recognition Method
Sun, M., Zhixin, G., & Deng, X. (2021) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3472634.3474069 · Cited by 7
Lingual markers for automating personality profiling: background and road ahead
Teli, M. A., & Chachoo, M. A. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00184-6 · Cited by 7
Rigou, Y., & Khriss, I. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-16078-3_49 · Cited by 7
An Automatic Speaker Clustering Pipeline for the Air Traffic Communication Domain
Khalil, D., Prasad, A., Motlíček, P., Zuluaga-Gómez, J., Nigmatulina, I., Madikeri, S. R., & Schuepbach, C. (2023) · Aerospace · doi:10.3390/aerospace10100876 · Cited by 7
Research on mixed model-based Chinese relation extraction
Lin, R., Chen, J., Yang, X., & Xu, H. (2010) · doi:10.1109/iccsit.2010.5564530 · Cited by 7
APP Relationship Calculation: An Iterative Process
Liu, M., Wu, C., Zhao, X.-N., Lin, C.-Y., & Wang, X. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2405557 · Cited by 7
Long-Distance Time-Event Relation Extraction
Moschitti, A., Patwardhan, S., & Welty, C. (2013) · Iris (University of Trento) · Cited by 7
Waugh, L. R., Valencia, J. A. Á., Do, T. H., Michelson, K., & Thomas, M. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0028 · Cited by 7
Küçük, D., & Yazıcı, A. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.12.048 · Cited by 7
Cross-Domain Bootstrapping for Named Entity Recognition
Sun, A., & Grishman, R. (2011) · Cited by 7
Self-citationApproaches to Hungarian Named Entity Recognition
Simon, E. (2013) · Cited by 7
FEATURES FOR NAMED ENTITY RECOGNITION IN CZECH LANGUAGE
Král, P. (2011) · International Conference on Knowledge Engineering and Ontology Development · doi:10.5220/0003660104370441 · Cited by 7
Data augmentation process to improve deep learning-based NER task in the automotive industry field
Keraghel, A., Benabdeslem, K., & Canitia, B. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207241 · Cited by 7
Advanced Literature-Mining Tools
Zweigenbaum, P., & Demner‐Fushman, D. (2009) · Springer eBooks · doi:10.1007/978-0-387-92738-1_17 · Cited by 7
Functional and Cognitive Grammars
Siewierska, A. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0022 · Cited by 10
Applying Deep Neural Networks to Named Entity Recognition in Portuguese Texts
Fernandes, I. A. D., Cardoso, H. L., & Oliveira, E. (2018) · International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams.2018.8554782 · Cited by 7
Šandrih, B., Krstev, C., & Stanković, R. M. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_122 · Cited by 7
Intelligent Information Extraction
Jakub Piskorski, B., & Skut, W. (2000) · Springer eBooks · doi:10.1007/978-1-4471-0761-3_13 · Cited by 7
Online Police Station, A State-of-Art Italian Semantic Technology against Cybercrime
Neri, F., Geraci, P., Sanna, G., & Lotti, L. (2009) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2009.20 · Cited by 7
A Hybrid Stepwise Approach for De-identifying Person Names in Clinical Documents
Ferrández, Ó., South, B. R., Shen, S., & Meystre, S. M. (2012) · Cited by 7
Integrating Language Understanding Agents Into the Semantic Web
Java, A., Finin, T., & Nirenburg, S. (2005) · Cited by 7
Molcho, G., Maier, S., Melero, F., & Aliprandi, C. (2014) · doi:10.1109/jisic.2014.63 · Cited by 7
Information extraction from medication leaflets
Aguiar, B. L. (2012) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.34626/1ppy-pw48 · Cited by 7
Active Learning - An Explicit Treatment of Unreliable Parameters
Becker, M. C. (2008) · ERA · Cited by 7
WebRED: Effective Pretraining And Finetuning For Relation Extraction On The WebPreprint
Ormándi, R., Saleh, M., Winter, E., & Rao, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09681 · Cited by 10
Vivaldi, J., & da Cunha, I. (2012) · Cited by 6
Zhang, M., Liu, X., Ma, J., Zhang, Z., Qiu, Y., & Jiang, Z. (2023) · Applied Sciences · doi:10.3390/app13179855 · Cited by 6
Deep mining the textual gold in relation extraction
Sharma, T., & Emmert‐Streib, F. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11042-4 · Cited by 8
Zweigenbaum, P., Lavergne, T., Grabar, N., Thierry Hamon, C., Rosset, S., & Grouin, C. (2013) · Biomedical Informatics Insights · doi:10.4137/bii.s11770 · Cited by 6
MeDetect: A LOD-Based System for Collective Entity Annotation in Biomedicine
Tian, L., Zhang, W., Bikakis, A., Wang, H., Yu, Y. H., Ni, Y., & Cao, F. (2013) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2013.34 · Cited by 6
Identifying and Extracting Named Entities from Wikipedia Database Using Entity Infoboxes
Mohamed, M., & Oussalah, M. (2014) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2014.050725 · Cited by 6
Ibrahim, A. O., Shamsuddin, S. M., & Qasem, S. N. (2015) · Journal of Information and Communication Technology · doi:10.32890/jict2015.14.0.8154 · Cited by 6
Sotomayor, M., & Veloz, F. (2017) · IEEE Ecuador Technical Chapters Meeting (ETCM) · doi:10.1109/etcm.2017.8247537 · Cited by 6
Role Knowledge Prompting for Document-Level Event Argument Extraction
Hu, R., Liu, H., & Zhou, H. (2023) · Applied Sciences · doi:10.3390/app13053041 · Cited by 6
Chinese Nested Named Entity Recognition Based on Boundary Prompt
Li, Z., Song, M. N., Zhu, Y., & Zhang, L. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-6222-8_28 · Cited by 6
A Hybrid Named Entity Recognition System for Aviation Text
Bharathi, A., Ramdin, R., Babu, P., Menon, V., Jayaramakrishnan, C., & Lakshmikumar, S. (2023) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.4185 · Cited by 6
Super-Sense Tagging Using Support Vector Machines and Distributional Features
Basile, P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-35828-9_19 · Cited by 6
Automatically Generated Noun Lexicons for Event Extraction
Arnulphy, B., Tannier, X., & Vilnat, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-28601-8_19 · Cited by 6
CroNER: Recognizing Named Entities in Croatian Using Conditional Random Fields
Karan, M., Glavaš, G., Šarić, F., Šnajder, J., Mijić, J., Šilić, A., & Bašić, B. D. (2013) · Informatica · Cited by 6
Hirschman, L., & Aberdeen, J. S. (2010) · Cited by 6
Named Entities as New Features for Czech Document Classification
Král, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-642-54903-8_35 · Cited by 6
Domain-Adaptive Relation Extraction for the Semantic Web
Xu, F., Uszkoreit, H., Li, H., Adolphs, P., & Cheng, X. (2014) · Cognitive technologies · doi:10.1007/978-3-319-06755-1_22 · Cited by 6
Exploiting Collective Hidden Structures in Webpage Titles for Open Domain Entity Extraction
Song, W., Zhao, S., Zhang, C., Wu, H. R., Wang, H., Liu, L., & Wang, H. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741107 · Cited by 6
A Knowledge Extraction and Management Component to Support Spontaneous Participation
Porwol, L., Hassan, I. A., Ojo, A., & Breslin, J. G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22500-5_6 · Cited by 6
Extracting bilingual terms from the Web
Gaizauskas, R., Paramita, M. L., Barker, E., Pinnis, M., Aker, A., & Solé, M. P. (2015) · Terminology International Journal of Theoretical and Applied Issues in Specialized Communication · doi:10.1075/term.21.2.04gai · Cited by 6
Tran, P.-N., Ta, V.-D., Truong, Q.-T., Duong, Q.-V., Nguyen, T.-T., & Phan, X.-H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-662-49381-6_17 · Cited by 6
van der Hülst, H. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0009 · Cited by 7
Definition of Recipe Terms and Corpus Annotation for their Automatic Recognition
Sasada, T., Mori, S., Yamakata, Y., Maeta, H., & Kawahara, T. (2015) · Journal of Natural Language Processing · doi:10.5715/jnlp.22.107 · Cited by 6
Investigating the Effect of ASR Tuning on Named Entity Recognition
Jannet, M. A. B., Galibert, O., Adda‐Decker, M., & Rosset, S. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1482 · Cited by 6
A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition
Simeonova, L., Simov, K., LMaKP, I.-B., Osenova, P., LMaKP, I.-B., & Nakov, P. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_127 · Cited by 6
Elazar, Y., Basmov, V., Goldberg, Y., & Tsarfaty, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00488 · Cited by 7
Iterative Named Entity Recognition with Conditional Random Fields
Alves-Pinto, A., Demus, C., Spranger, M., Labudde, D., & Hobley, E. (2021) · Applied Sciences · doi:10.3390/app12010330 · Cited by 6
A spatially-aware algorithm for location extraction from structured documents
Sharma, P., Samal, A. K., Soh, L., & Joshi, D. (2022) · GeoInformatica · doi:10.1007/s10707-022-00482-1 · Cited by 6
HFST-SweNER — A New NER Resource for Swedish
Kokkinakis, D., Niemi, J., Hardwick, S., Lindén, K., & Borin, L. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2zd3agtknzgf · Cited by 6
The Construction of a Chinese Named Entity Tagged Corpus: CNEC1.0
Shih, C.-W., Tsai, R. T., Wu, S.-H., Hsieh, C.-C., & Hsu, W. (2004) · Cited by 6
Chinese named entity recognition using modified conditional random field on postal address
Sun, W. (2017) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp-bmei.2017.8302311 · Cited by 6
Extraction of Semantic Relations in Noisy User-Generated Law Enforcement Data
Schraagen, M., & Bex, F. (2019) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icosc.2019.8665497 · Cited by 6
Active Entity Recognition in Low Resource Settings
Gao, N., Karampatziakis, N., Potharaju, R., & Cucerzan, S. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3358109 · Cited by 6
Automatic Acquisition of Usage Information for Language Resources
Kozawa, S., Tohyama, H., Uchimoto, K., & Matsubara, S. (2008) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4vnay7rwzni4 · Cited by 6
Clipping and Analyzing News Using Machine Learning Techniques
Gründel, H., Naphtali, T., Wiech, C., Gluba, J.-M., Rohdenburg, M., & Scheffer, T. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45650-3_11 · Cited by 6
Resource-Bounded Information Extraction: Acquiring Missing Feature Values on Demand
Kanani, P. H., McCallum, A., & Hu, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13657-3_45 · Cited by 6
Wilks, Y. A., & Gaizauskas, R. (1999) · Text, speech and language technology · doi:10.1007/978-94-017-2388-6_8 · Cited by 6
Angelova, G. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-14197-3_6 · Cited by 6
Information Extraction from Free-Text Business Documents
Abramowicz, W., & Jakub Piskorski, B. (2011) · IGI Global eBooks · doi:10.4018/978-1-93177-747-6.ch002 · Cited by 6
A Named Entity Recognition Method Based on Decomposition and Concatenation of Word Chunks
Iwakura, T., Takamura, H., & Okumura, M. (2013) · ACM Transactions on Asian Language Information Processing · doi:10.1145/2499955.2499958 · Cited by 6
Transforming examples into patterns for information extraction
Yangarber, R., & Grishman, R. (1996) · doi:10.3115/1119089.1119109 · Cited by 6
Self-citationLiakata, M., & Pulman, S. (2004) · International Conference on Computational Linguistics · doi:10.3115/1220355.1220382 · Cited by 6
Extracting Geospatial Entities from Wikipedia
Witmer, J. D., & Kalita, J. K. (2009) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2009.62 · Cited by 6
AATOS – A Configurable Tool for Automatic Annotation
Tamper, M., Leskinen, P., Ikkala, E., Oksanen, A., Eetu Mäkelä, A., Heino, E., Tuominen, J., Koho, M., & Eero Hyvönen, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59888-8_24 · Cited by 6
History of the Study of Gesture
Kendon, A. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199585847.013.0004 · Cited by 6
The reusability of induced knowledge for the automatic semantic markup of taxonomic descriptions
Cui, H., & Heidorn, P. B. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20463 · Cited by 6
A Complex Evaluation Architecture for HAREM
Seco, N., Santos, D., Cardoso, N., & Vilela, R. (2006) · Lecture notes in computer science · doi:10.1007/11751984_35 · Cited by 6
Grishman, R. (2006) · Elsevier eBooks · doi:10.1016/b0-08-044854-2/00956-1 · Cited by 6
Self-citationJoachims, T., & Yu, C. N. (2011) · eCommons (Cornell University) · Cited by 6
Metadata
- Authors
- Grishman, R., & Sundheim, B.
- Affiliations
- New York University (US)
- Venue
- Proceedings of the 16th conference on Computational linguistics -
- Published
- 1996
- Volume
- 1
- Pages
- 466
- ISSN
- 1525-2477 (ISSN-L)
- OpenAlex
- W2068882115
- Cited by
- 1,408
- References
- 0
- Citations by year
- 2026: 26 · 2025: 37 · 2024: 62 · 2023: 74 · 2022: 64 · 2021: 81 · 2020: 64 · 2019: 73 · 2018: 68 · 2017: 50 · 2016: 61 · 2015: 71 · 2014: 65 · 2013: 117 · 2012: 83
- Type
- conference abstract
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Computational Linguistics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- MUC6, information extraction, benchmark tasks, natural language processing evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review