ExplainaBoard: An Explainable Leaderboard for NLP
Open accessAbstract
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Graham Neubig. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations. 2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A more granular assessment can be particularly valuable because it allows users to pinpoint where inaccuracies occur and serves as a reward model for developers to refine their generative systems.
“A more granular assessment can be particularly valuable because it (1) not only allows users to pinpoint where inaccuracies occur (Liu et al., 2021) but also (2) serves as a reward model for developers to refine their generative systems (Lightman et al., 2023).”
From FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) · cites this paper as Liu et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602 · see citing works
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218 · see citing works
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
Tjong Kim Sang, E. F., & De Meulder, F. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119195 · Cited by 2,788 · see citing works
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 2,318 · see citing works
Text Summarization with Pretrained Encoders
Liu, Y., & Lapata, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1387 · Cited by 1,779 · see citing works
Self-citationIntroduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity RecognitionPreprint
Sang, E. T. K. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0209010 · Cited by 1,677 · see citing works
(2018) · International Conference on Learning Representations · Cited by 1,621 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
Bootstrap Methods: Another Look at the Jackknife
Efron, B. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_41 · Cited by 1,374 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051 · see citing works
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1425 · Cited by 1,034 · see citing works
Baselines and Bigrams: Simple, Good Sentiment and Topic Classification
Wang, S., & Manning, C. D. (2012) · Cited by 957 · see citing works
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Yamada, I., Asai, A., Shindo, H., Takeda, H., & Matsumoto, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.523 · Cited by 647 · see citing works
Analysis Methods in Neural Language Processing: A Survey
Belinkov, Y., & Glass, J. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00254 · Cited by 485 · see citing works
Findings of the 2019 Conference on Machine Translation (WMT19)
Barrault, L., Bojar, O., Costa‐jussà, M. R., Federmann, C., Fishel, M., Graham, Y., Haddow, B., Huck, M., Koehn, P., Malmasi, S., et al. (2019) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w19-5301 · Cited by 461 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357 · see citing works
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation
Hu, J., Ruder, S., Siddhant, A., Neubig, G., Fırat, O., & Johnson, M. (2020) · International Conference on Machine Learning · Cited by 325 · see citing works
Self-citationPooled Contextualized Embeddings for Named Entity Recognition
Akbik, A., Bergmann, T., & Vollgraf, R. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1078 · Cited by 284 · see citing works
Long Short-Term Memory Neural Networks for Chinese Word Segmentation
Chen, X., Qiu, X., Zhu, C., Liu, P., & Huang, X. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1141 · Cited by 280 · see citing works
Self-citationXGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Liang, Y., Duan, N., Gong, Y., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.484 · Cited by 270 · see citing works
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
Liu, Y., & Liu, P. (2021) · doi:10.18653/v1/2021.acl-short.135 · Cited by 236 · see citing works
Self-citationA high-performance semi-supervised learning method for text chunking
Ando, R. K., & Zhang, T. (2005) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/1219840.1219841 · Cited by 224 · see citing works
Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa
Dai, J., Yan, H. R., Sun, T., Liu, P., & Qiu, X. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.146 · Cited by 190 · see citing works
Self-citationFindings of the 2020 Conference on Machine Translation (WMT20)
Barrault, L., Biesialska, M., Bojar, O., Costa‐jussà, M. R., Federmann, C., Graham, Y., Grundkiewicz, R., Haddow, B., Huck, M., Joanis, E., et al. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.1 · Cited by 178 · see citing works
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Aremu, A., Bosselut, A., Chandu, K. R., Clinciu, M.-A., Das, D., Dhole, K., et al. (2021) · Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) · doi:10.18653/v1/2021.gem-1.10 · Cited by 159 · see citing works
Re-evaluating Evaluation in Text Summarization
Bhandari, M., Gour, P. N., Ashfaq, A., Liu, P., & Neubig, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.751 · Cited by 148 · see citing works
Self-citationUtility is in the Eye of the User: A Critique of NLP Leaderboards
Ethayarajh, K., & Jurafsky, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.393 · Cited by 147 · see citing works
Tenney, I., Wexler, J., Bastings, J., Bolukbasi, T., Coenen, A., Gehrmann, S., Jiang, E., Pushkarna, M., Radebaugh, C., Reif, E., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.15 · Cited by 138 · cited 1× · see citing works
“On the other hand, there are many other glass-box interpretability tools that look at model internals, such as the AllenNLP Interpret (Wallace et al., 2019) and Language Inter-pretability Tool (Tenney et al., 2020).”
BackgroundAllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
Wallace, E., Tuyls, J., Wang, J., Subramanian, S., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations · doi:10.18653/v1/d19-3002 · Cited by 116 · see citing works
SpanNER: Named Entity Re-/Recognition as Span Prediction
Fu, J., Huang, X., & Liu, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.558 · Cited by 112 · see citing works
Self-citationcompare-mt: A Tool for Holistic Comparison of Language Generation Systems
Neubig, G., Dou, Z.-Y., Hu, J., Michel, P., Pruthi, D., & Wang, X. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-4007 · Cited by 109 · see citing works
Self-citationFLERT: Document-Level Features for Named Entity RecognitionPreprint
Schweter, S., & Akbik, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06993 · Cited by 103 · see citing works
Towards Automatic Error Analysis of Machine Translation Output
Popović, M., & Ney, H. (2011) · Computational Linguistics · doi:10.1162/coli_a_00072 · Cited by 102 · see citing works
Identifying Intention Posts in Discussion Forums
Chen, Z., Liu, B., Hsu, M., Castellanos, M., & Ghosh, R. (2013) · Cited by 88 · see citing works
Explaining the Stars: Weighted Multiple-Instance Learning for Aspect-Based Sentiment Analysis
Pappas, N., & Popescu-Belis, A. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1052 · Cited by 74 · see citing works
Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
Fu, J., Liu, P., & Zhang, Q. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6276 · Cited by 74 · see citing works
Self-citationXGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and GenerationPreprint
Liang, Y., Duan, N., Gong, Y., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01401 · Cited by 65 · see citing works
Blast: A Tool for Error Analysis of Machine Translation Output
Stymne, S. (2011) · Cited by 55 · see citing works
Interpretable Multi-dataset Evaluation for Named Entity Recognition
Fu, J., Liu, P., & Neubig, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.489 · Cited by 53 · see citing works
Self-citationGENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation.Preprint
Khashabi, D., Stanovsky, G., Bragg, J., Lourie, N., Kasai, J., Choi, Y., Smith, N. A., & Weld, D. S. (2021) · arXiv (Cornell University) · Cited by 47 · see citing works
A Closer Look at Data Bias in Neural Extractive Summarization Models
Zhong, M., Wang, D., Liu, P., Qiu, X., & Huang, X. (2019) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/d19-5410 · Cited by 45 · see citing works
Self-citationRefSum: Refactoring Neural Summarization
Liu, Y., Dou, Z.-Y., & Liu, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.113 · Cited by 36 · see citing works
Self-citationTowards More Fine-grained and Reliable NLP Performance Prediction
Ye, Z., Liu, P., Fu, J., & Neubig, G. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.324 · Cited by 29 · see citing works
Self-citationRethinkCWS: Is Chinese Word Segmentation a Solved Task?
Fu, J., Liu, P., Zhang, Q., & Huang, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.457 · Cited by 23 · see citing works
Self-citationGeneralized Minimum Bayes Risk System Combination
Duh, K., Sudoh, K., Wu, X., Tsukada, H., & Nagata, M. (2011) · Cited by 18 · see citing works
Stacked Generalizations: When Does It Work?
Ting, K. M., & Witten, I. H. (1997) · International Joint Conference on Artificial Intelligence · Cited by 16 · see citing works
Minimum Bayes-risk System Combination
González-Rubio, J., Juan, A., & Casacuberta, F. (2011) · Cited by 15 · see citing works
Flagged first, then by citations
1 of the 58 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- PMLR.
- Yixin Liu, Dou Ziyi, and Pengfei Liu. 2021. Ref- sum: Refactoring neural summarization. In The 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies.
- Kristina Toutanova, Dan Klein, Christopher D. Man- ning, and Yoram Singer. 2003. Feature-rich part-of- speech tagging with a cyclic dependency network. In Proceedings of the 2003 Human Language Tech- nology Conference of the North American Chapter of the Association for Computational Linguistics, pages 252-259.
Cited by
Sorted by citations
BRIO: Bringing Order to Abstractive Summarization
Liu, Y., Liu, P., Radev, D., & Neubig, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.207 · Cited by 244 · see citing works
Self-citationSimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
Liu, Y., & Liu, P. (2021) · doi:10.18653/v1/2021.acl-short.135 · Cited by 236 · see citing works
Self-citationXTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
Ruder, S., Constant, N., Botha, J. A., Siddhant, A., Fırat, O., Fu, J., Liu, P., Hu, J., Garrette, D., Neubig, G., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.802 · Cited by 133 · see citing works
Self-citationRepairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123 · see citing works
Towards a Unified Multi-Dimensional Evaluator for Text Generation
Zhong, M., Liu, Y., Yin, D., Mao, Y., Jiao, Y., Liu, P., Zhu, C., Ji, H., & Han, J. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.131 · Cited by 117 · see citing works
Self-citationSpanNER: Named Entity Re-/Recognition as Span Prediction
Fu, J., Huang, X., & Liu, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.558 · Cited by 112 · see citing works
Self-citationA Primer on Pretrained Multilingual Language Models
Doddapaneni, S., Ramesh, G., Khapra, M. M., Kunchukuttan, A., & Kumar, P. (2025) · ACM Computing Surveys · doi:10.1145/3727339 · Cited by 72 · see citing works
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
Adelani, D. I., Neubig, G., Ruder, S., Rijhwani, S., Beukman, M., Palen-Michel, C., Lignos, C., Alabi, J. O., Muhammad, S. H., Nabende, P., et al. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.298 · Cited by 65 · see citing works
Self-citationMWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers
Lan, Y., Wang, L., Zhang, Q., Lan, Y., Dai, B. T., Wang, Y., Zhang, D., & Lim, E. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21723 · Cited by 58 · see citing works
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29 · see citing works
“A more granular assessment can be particularly valuable because it (1) not only allows users to pinpoint where inaccuracies occur (Liu et al., 2021) but also (2) serves as a reward”
BackgroundSelf-citationPolyglot Prompt: Multilingual Multitask Prompt Training
Fu, J., Ng, S.-K., & Liu, P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.674 · Cited by 23 · see citing works
Self-citationWhat Makes Chain-of-Thought Prompting Effective? A Counterfactual Study
Madaan, A., Hermann, K. L., & Yazdanbakhsh, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.101 · Cited by 17 · see citing works
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning
Jain, S., Keshava, V., Mysore Sathyendra, S., Fernandes, P., Liu, P., Neubig, G., & Zhou, C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.537 · Cited by 15 · see citing works
Self-citationBenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
Gashteovski, K., Yu, M., Kotnis, B., Lawrence, C., Niepert, M., & Glavaš, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.307 · Cited by 14 · see citing works
Dual Path Modeling for Semantic Matching by Perceiving Subtle Conflicts
Xue, C., Liang, D., Wang, S., Zhang, J., & Wu, W. B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096590 · Cited by 13 · see citing works
SketchXAI: A First Look at Explainability for Human Sketches
Qu, Z., Gryaditskaya, Y., Li, K., Pang, K., Xiang, T., & Song, Y.-Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02234 · Cited by 13 · see citing works
Numerical reasoning in machine reading comprehension tasks: are we there yet?
Al-Negheimish, H., Madhyastha, P., & Russo, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.759 · Cited by 12 · see citing works
DataLab: A Platform for Data Analysis and Intervention
Yang, X., Fu, J., Yuan, W., Viswanathan, V., Liu, Z., Liu, Y., Neubig, G., & Liu, P. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.18 · Cited by 10 · see citing works
Self-citationMoving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking
Santhanam, K., Saad-Falcon, J., Franz, M., Khattab, O., Sil, A., Florian, R., Sultan, M. A., Roukos, S., Zaharia, M., & Potts, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.738 · Cited by 10 · see citing works
TextBox 2.0: A Text Generation Library with Pre-trained Language Models
Tang, T., Li, J., Chen, Z., Hu, Y., Yu, Z., Dai, W., Zhao, W. X., Nie, J., & Wen, J.-R. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.42 · Cited by 8 · see citing works
Li, S., Han, C., Yu, P., Edwards, C., Li, M., Wang, X., Fung, Y. R., Yu, C., Tetreault, J., Hovy, E. H., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.799 · Cited by 8 · see citing works
Predicting Fine-Tuning Performance with Probing
Zhu, Z., Shahtalebi, S., & Rudzicz, F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.793 · Cited by 8 · see citing works
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements
von Werra, L., Tunstall, L., Thakur, A., Luccioni, A. S., Thrush, T., Piktus, A., Marty, F., Rajani, N. F., Mustar, V., & Ngo, H. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.13 · Cited by 7 · see citing works
Hassan, S. B., Noor, M. A., Forhad, S., Tasnim, J., & Siddique, A. H. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-97-8093-8_26 · Cited by 5 · see citing works
IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models
Mosca, E., Dementieva, D., Ajdari, T. E., Kummeth, M., Gringauz, K., Zhou, Y., & Groh, G. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-demo.7 · Cited by 5 · see citing works
Widjaja, H., Gashteovski, K., Rim, W. B., Liu, P., Malon, C. D., Ruffinelli, D., Lawrence, C., & Neubig, G. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.34 · Cited by 4 · see citing works
Self-citationTowards Evaluation of Multi-party Dialogue Systems
Mahajan, K., Santhanam, S., & Shaikh, S. (2022) · International Conference on Natural Language Generation · doi:10.18653/v1/2022.inlg-main.23 · Cited by 4 · see citing works
Linking Surface Facts to Large-Scale Knowledge Graphs
Radevski, G., Gashteovski, K., Hung, C.-C., Lawrence, C., & Glavaš, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.445 · Cited by 3 · see citing works
Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking
Väth, D., Tilli, P., & Vu, N. T. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.14 · Cited by 3 · see citing works
Testing Cross-Database Semantic Parsers With Canonical Utterances
Lent, H., Yavuz, S., Yu, T., Niu, T., Zhou, Y., Radev, D., & Lin, X. (2021) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2021.eval4nlp-1.8 · Cited by 2 · see citing works
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering
Ke, P., Huang, F., Mi, F., Wang, Y., Liu, Q., Zhu, X., & Huang, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.539 · Cited by 2 · see citing works
MuLER: Detailed and Scalable Reference-based Evaluation
Karidi, T., Choshen, L., Patel, G., & Abend, O. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.29
Self-explaining Neural Network for Multi-criteria Sentiment Analysis
Hoang, X.-T., Nguyen, V.-L., Kitagawa, T., & Huynh, V. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-4606-7_8
Local and Global: Multi-View Modeling for Duplicate Question Detection
Wang, H., Lei, J., & Liu, J. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228772
Contextualization and Generalization in Entity and Relation Extraction
Taillé, B. (2022) · theses.fr (ABES) · doi:10.70675/72fea173z5549z4bd5z8332zf81f26d3c2b9
Diagnosing LLM Benchmark: A Psychometric Analysis of Difficulty and Discrimination
Qin, J., Zhang, X., Feng, D., Ding, B., & Zhai, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31673-8_10
- Authors
- Liu, P., Fu, J., Xiao, Y., Yuan, W., Chang, S., Dai, J., Liu, Y., Ye, Z., & Neubig, G.
- Affiliations
- Carnegie Mellon University (US)
- Fudan University (CN)
- The Ohio State University (US)
- Corresponding author
- Neubig, Graham
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations
- Published
- 2021
- Pages
- 280-289
- OpenAlex
- W3173322864
- Cited by
- 35
- References
- 58
- Citations by year
- 2026: 1 · 2025: 4 · 2023: 11 · 2022: 13 · 2021: 6
- FWCI
- 3.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) (publishedVersion, cc-by)
- Funders
- Air Force Research Laboratory (FA8750-19-2-0200)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- explainable AI, natural language processing, model evaluation, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
