The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets
Open accessAbstract
Mor Geva, Yoav Goldberg, Jonathan Berant. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These large performance disparities across users are practical concerns in consumer-facing applications, and they can also indicate that models are exploiting biases or spurious correlations in the data (Badgeley et al., 2019; Geva et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Geva et al., 2019
In the citing paper
“Performance disparities across individuals have been observed in a wide range of tasks and applications, including in natural language processing (Geva et al., 2019), automatic speech recognition (Koenecke et al., 2020; Tatman, 2017), federated learning (Li et al., 2019b; Caldas et al., 2018), and medical imaging (Badgeley et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Geva et al., 2019
In the citing paper
“More directly, across a few natural language processing tasks where examples are annotated by crowdworkers, models have been observed to perform well on annotators that are commonly seen at training time, but fail to generalize to unseen annotators, suggesting that models are merely learning annotator-specific patterns and not the task (Geva et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Geva et al., 2019
In the citing paper
“These large performance disparities across users are practical concerns in consumer-facing applications, and they can also indicate that models are exploiting biases or spurious correlations in the data (Badgeley et al., 2019; Geva et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Geva et al., 2019
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P. E., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1260 · Cited by 1,602
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, A., Herzig, J., Lourie, N., & Berant, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1421 · Cited by 919
Self-citation
Cited by
Sorted by citations
Items 201–282 of 282
Guideline Bias in Wizard-of-Oz Dialogues
Hansen, V. P. B., & Søgaard, A. (2021) · doi:10.18653/v1/2021.bppf-1.2 · Cited by 3
Contextualizing Variation in Text Style Transfer Datasets
Schoch, S., Du, W., & Ji, Y. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.22 · Cited by 3
Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering
Yang, S., Huang, X., Lau, J. H., & Erfani, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.88 · Cited by 4
Leveraging Data Recasting to Enhance Tabular Reasoning
Jena, A., Gupta, V., Shrivastava, M., & Eisenschlos, J. M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.328 · Cited by 3
Resolving the Human-subjects Status of Machine Learning's Crowdworkers
Kaushik, D., Lipton, Z. C., & London, A. J. (2023) · Queue · doi:10.1145/3639452 · Cited by 5
Target Consistency for Domain Adaptation: when Robustness meets TransferabilityPreprint
Ouali, Y., Bouvier, V., Tami, M., & Hudelot, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14263 · Cited by 4
Shinoda, K., Sugawara, S., & Aizawa, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03238 · Cited by 3
Tempska, P., Urbaniak, R., Dowgiałło, M., Ptaszynski, M. E., Zajączkowski, A., Milewska, M., Leliwa, G., Marcińczuk, M., Brochocki, M., & Wroczyński, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104316 · Cited by 3
Generative Context Pair Selection for Multi-hop Question Answering
Dua, D., dos Santos, C. N., Ng, P., Athiwaratkun, B., Xiang, B., Gardner, M., & Singh, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.561 · Cited by 2
Locke’s Holiday: Belief Bias in Machine Reading
Søgaard, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.649 · Cited by 2
A resampling-based method to evaluate NLI models
Salvatore, F. D. S., Finger, M., Hirata, R., & Patriota, A. G. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000268 · Cited by 2
Complex Reasoning in Natural Languag
Zhao, W., Geva, M., Lin, B. Y., Yasunaga, M., Madaan, A., & Yu, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-tutorials.2 · Cited by 5
Self-citationAnswer Quality Aware Aggregation for Extractive QA Crowdsourcing
Zhu, P., Wang, Z., Hauff, C., Yang, J., & Anand, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.457 · Cited by 2
Saxon, M., Wang, X., Xu, W., & Wang, W. Y. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.223 · Cited by 2
Honda, U., Oka, T., Zhang, P., & Mita, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00701 · Cited by 3
Learning to Ask for Conversational Machine Learning
Srivastava, S., Labutov, I., & Mitchell, T. M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1426 · Cited by 2
k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations
Wong, K., & Paritosh, P. K. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.42 · Cited by 2
Realistic Data Augmentation Framework for Enhancing Tabular Reasoning
Kumar, D., Gupta, V., Sharma, S., & Zhang, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.324 · Cited by 2
Neglected Free Lunch – Learning Image Classifiers Using Annotation Byproducts
Han, D., Choe, J., Chun, S.-Y., Chung, J. J. Y., Chang, M., Yun, S., Song, J. Y., & Oh, S. J. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.01848 · Cited by 5
A guide to the dataset explosion in QA, NLI, and commonsense reasoning
Rogers, A., & Rumshisky, A. (2020) · doi:10.18653/v1/2020.coling-tutorials.5 · Cited by 2
Crowdsourcing Ecologically-Valid Dialogue Data for German
Frommherz, Y., & Zarcone, A. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.686050 · Cited by 2
Stochastic Adversarial Gradient Embedding for Active Domain AdaptationPreprint
Bouvier, V., Very, P., Chastagnol, C., Tami, M., & Hudelot, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01843 · Cited by 2
Challenges and Considerations with Code-Mixed NLP for Multilingual SocietiesPreprint
Srivastava, V. P., & Singh, M. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07823 · Cited by 13
Competency Problems: On Finding and Removing Artifacts in Language DataPreprint
Gardner, M., Merrill, W., Dodge, J., Peters, M. E., Ross, A., Singh, S., & Smith, N. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08646 · Cited by 25
Generalization in NLI: Ways (Not) To Go Beyond Simple HeuristicsPreprint
Bhargava, P., Валентинович, Д. А., & Rogers, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01518 · Cited by 16
Machines Do Not Decide Hate Speech: Machine Learning, Power, and the Intersectional ApproachPreprint
Kim, J. Y. (2021) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/chvgp · Cited by 2
Søgaard, A. (2024) · Theoria · doi:10.1111/theo.12581 · Cited by 1
Towards Stable Natural Language Understanding via Information Entropy Guided Debiasing
Du, L., Ding, X. H., Sun, Z., Liu, T., Qin, B., & Liu, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.161 · Cited by 1
Measuring Adversarial Datasets
Bai, Y., Huang, R., Viswanathan, V., Kuo, T.-S., & Wu, T. (2023) · doi:10.18653/v1/2023.artofsafety-1.4 · Cited by 2
Improving Robustness in Language Models for Legal Textual Entailment Through Artifact-Aware Training
Wehnert, S., Murugadas, V., Naik, P., & Luca, E. W. D. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-3076-6_16 · Cited by 1
Dai, R., Tan, Y., Mo, L., Liang, S., Huo, G., Luo, J., & Cheng, Y. (2024) · Proceedings of the 2024 International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658040 · Cited by 1
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
Ohmer, X. I., Bruni, E., & Hupkes, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00529 · Cited by 18
Schneider, J., Eisenhardt, D., Utama, C., & Meske, C. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-80122-8_5 · Cited by 1
Intent Classification in Chatbots with Human-AI Collaborated Large-Scale Dataset
Hoque, F., Karim, R., Kuri, R., Hasan, K. N., & Rana, A. (2025) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecce64574.2025.11013223 · Cited by 1
Cheap Learning: Maximizing Performance of Language Models for Social Data Science Using Minimal Data
Castro-González, L., Chung, Y.-L., Kirk, H. R., Francis, J., Williams, A. R., Johansson, P., & Bright, J. (2025) · Sociological Methods & Research · doi:10.1177/00491241251340608 · Cited by 2
LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity
Arunkumar, A., Sharma, S., Agrawal, R., Chandrasekaran, S., & Bryan, C. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14840 · Cited by 1
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Sugawara, S., & Tsugita, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.861 · Cited by 1
AbLit: A Resource for Analyzing and Generating Abridged Versions of English Literature
Roemmele, M., Shaffer, K., Olsen, K., Wang, Y., & DeNeefe, S. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.269 · Cited by 1
Building Low-Resource NER Models Using Non-Speaker AnnotationPreprint
Tsygankova, T., Marini, F., Mayhew, S., & Roth, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09627 · Cited by 2
Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection ModelsPreprint
Chandu, K. R., Sharma, P. K., Changpinyo, S., Thapliyal, A., & Soricut, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05175 · Cited by 3
Targeted Data Acquisition for Evolving Negotiation AgentsPreprint
Kwon, M., Karamcheti, S., Cuéllar, M.-F., & Sadigh, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07728 · Cited by 1
Semi-supervised Active RegressionPreprint
Devvrit, F., Rajaraman, N., & Awasthi, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06676 · Cited by 1
Semantic Answer Similarity for Evaluating Question Answering ModelsPreprint
Risch, J., Möller, T., Gutsch, J., & Pietsch, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06130 · Cited by 29
Osborne, M. R., Cratsley, M. J., Massey, M., & Rice, V. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/tc5vq · Cited by 1
Honday, U., Oka, T., Zhang, P., & Mita, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4867631 · Cited by 1
A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation
Chen, J., Chen, W., Lin, Z., & Vong, C. M. (2026) · arXiv · doi:10.48550/arxiv.2608.24419
“…shortcut learning in deep networks [17], syntactic heuristics passing inference benchmarks [34], annotator identity recoverable from labels [18], counterfactually augmented data proposed against such correlations [28], the benchmarking critique that a leaderboard number can be…”
Overestimation of Syntactic Representationin Neural Language ModelsPreprint
Kodner, J., & Gupta, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05067
Personal Semantic Variations in Word Meanings: Induction, Application, and Analysis
Oba, D., Sato, S., Akasaki, S., Yoshinaga, N., & Toyoda, M. (2020) · Journal of Natural Language Processing · doi:10.5715/jnlp.27.467
Hidden Biases in Unreliable News Detection DatasetsPreprint
Zhou, X., Elfardy, H., Christodoulopoulos, C., Butler, T., & Bansal, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.10130 · Cited by 1
Evaluating Dataset Creation Heuristics for Concept Detection in Web Pages Using BERT
París, M., & Jäschke, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-82147-0_14
On learning an interpreted language with recurrent modelsPreprint
Paperno, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04128
An Empirical Investigation of Learning from Biased Toxicity LabelsPreprint
Nanda, N., Uesato, J., & Gowal, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01577
Does Putting a Linguist in the Loop Improve NLU Data Collection?Preprint
Parrish, A., Huang, W. C. W., Agha, O., Lee, S.-H., Nangia, N., Warstadt, A., Aggarwal, K., Allaway, E., Linzen, T., & Bowman, S. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07179 · Cited by 8
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic EvaluationPreprint
Rebuffel, C., Scialom, T., Soulier, L., Piwowarski, B., Lamprier, S., Staiano, J., Scoutheeten, G., & Gallinari, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07555 · Cited by 5
Generative Context Pair Selection for Multi-hop Question AnsweringPreprint
Dua, D., dos Santos, C. N., Ng, P., Athiwaratkun, B., Xiang, B., Gardner, M., & Singh, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08744 · Cited by 1
Detecting Community Sensitive Norm Violations in Online ConversationsPreprint
Park, C. Y., Mendelsohn, J., Radhakrishnan, K., Jain, K., Kanakagiri, T., Jurgens, D., & Tsvetkov, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04419 · Cited by 5
Tsuchiya, M., & Watarai, T. (2022) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.37-4_a-lc3
Convolutional network fabric pruning with label noise
Benjelloun, I., Lamiroy, B., & Koudou, A. E. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10507-2
Argument structures of political debates
Haddadan, S. (2022) · De Gruyter eBooks · doi:10.1515/9783110723991-004
On the Effectiveness of Images in Multi-modal Text Classification: An Annotation Study
Ma, C., Shen, A., Yoshikawa, H., Iwakura, T., Beck, D., & Baldwin, T. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3565572
Blackfoot Words: a database of Blackfoot lexical forms
Weber, N. B., Brown, T., Celli, J., Denham, M., Dykstra, H., Hernandez-Merlin, R., Hochstein, E., Hwang, P., Kidd, N., Kulmizev, D., et al. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-022-09631-2
Beinborn, L., & Hollenstein, N. (2023) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-43260-6_4
TalkUp: Paving the Way for Understanding Empowering Language
Njoo, L., Park, C., Stappart, O., Thielk, M., Yi, C., & Tsvetkov, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.625 · Cited by 4
Ecologically Valid Explanations for Label Variation in NLI
Jiang, N., Tan, C., & de Marneffe, M.-C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.712 · Cited by 2
Mitigating Annotation Artifacts in Physical Commonsense Reasoning Benchmark
Mok, W.-L., & Kim, S. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386712
Exploring Explanations Improves the Robustness of In-Context LearningPreprint
Honday, U., & Oka, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5277637
Feitosa, A., Carneiro, É., & Guedes, G. P. (2025) · doi:10.5753/kdmile.2025.247589
A Study of Fairness and Bias of Deep Learning Models
Lohani, K., Meena, T., Vanjara, D., Barde, G., Raj, A., Rawat, S., Chaubey, G., Kumar, B., & Singh, K. P. (2025) · IEEE International Conference on Computer Vision and Machine Intelligence (CVMI) · doi:10.1109/cvmi66673.2025.11337368
Cui, X., Huang, Z., & Abeynayake, N. R. (2026) · Datenbank-Spektrum · doi:10.1007/s13222-026-00529-9 · Cited by 1
AI Production Process Security Attacks and Mitigation Techniques
Yu, H., & Halak, B. (2025) · Springer eBooks · doi:10.1007/978-3-032-08514-6_7
Neural Approaches to Historical Words Reconstruction
Fourrier, C. (2022) · theses.fr (ABES) · doi:10.70675/647b8a58zeea0z48cdz8229zf1d7ee8b0742 · Cited by 2
Event extraction from facsimiles of ancient documents for history studies
Baptiste, B. (2022) · theses.fr (ABES) · doi:10.70675/76b9d155z354dz46d2z94eazc1a6832a5895
Vers la mitigation des biais en traitement neuronal des langues
Berre, G. L. (2023) · theses.fr (ABES) · doi:10.70675/f37c3802z821az4a6dzb877zf631994cacd5
What Stays and What Goes: Auditing the Impact of LLM Summarization on News Partisanship
Savgira, P., Kreiss, E., & Hosseinmardi, H. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799057
RGPO: Ranking-Guided Preference Optimization for Reliable Clinical Reasoning
Hsu, C. T., Ding, J.-E., Hsu, H.-L., Hsu, C.-H., Yang, S., Yao, L.-H., Liao, C.-C., Liu, F., & Hung, F. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3688221
The Human Factor in Data Cleaning: Exploring Preferences and Biases
AbdElazim, H., Islam, S., & Milani, M. (2026) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew71238.2026.00014
Semantic Consensus-Based Multi-LLM Architecture for TÜBİTAK Proposal Generation
Aktaş, Y., Yi̇ği̇t, T., Ersoy, M., & Üncü, İ. S. (2026) · Advances in Artificial Intelligence Research · doi:10.54569/aair.1894025
SAGE: Scalable AI Governance & Evaluation
Le, B. H., Lu, X., Stern, N., Liu, W., Lapchuk, I., Li, X., Zheng, B., Rosenberg, K., Huang, J., Zhang, Z., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818476 · Cited by 5
Feitosa, A., Fasano, A., & Guedes, G. (2026) · Anais do Brazilian Workshop on Social Network Analysis and Mining (BraSNAM) · doi:10.5753/brasnam.2026.23296
SciTables : A Dataset and Evaluation Framework for Complex Table-to-Text Generation
Alizade, M., Kong, T., & Maity, S. K. (2026) · Proceedings of the VLDB Endowment · doi:10.14778/3836663.3836697
Metadata
- Authors
- Geva, M., Goldberg, Y., & Berant, J.
- Affiliations
- Tel Aviv University (IL)
- Bar-Ilan University (IL)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 1161-1166
- OpenAlex
- W2969282158
- Cited by
- 284
- References
- 28
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 19 · 2023: 40 · 2022: 56 · 2021: 83 · 2020: 56 · 2019: 6
- FWCI
- 19.22
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Israel Science Foundation
- Tel Aviv University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- annotator bias, dataset construction, task modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
