The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Can Large Language Models Be an Alternative to Human Evaluations?
Open accessAbstract
Human evaluation is indispensable and inevitable for assessing the quality of texts generated by machine learning models or written by humans.However, human evaluation is very difficult to reproduce and its quality is notoriously unstable, hindering fair comparisons among different natural language processing (NLP) models and algorithms.Recently, large language models (LLMs) have demonstrated exceptional performance on unseen tasks when only the task instructions are provided.In this paper, we explore if such an ability of the LLMs can be used as an alternative to human evaluation.We present the LLMs with the exact same instructions, samples to be evaluated, and questions used to conduct human evaluation, and then ask the LLMs to generate responses to those questions; we dub this LLM evaluation.We use human evaluation and LLM evaluation to evaluate the texts in two NLP tasks: openended story generation and adversarial attacks.We show that the result of LLM evaluation is consistent with the results obtained by expert human evaluation: the texts rated higher by human experts are also rated higher by the LLMs.We also find that the results of LLM evaluation are stable over different formatting of the task instructions and the sampling algorithm used to generate the answer.We are the first to show the potential of using LLMs to assess the quality of texts and discuss the limitations and ethical considerations of LLM evaluation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GPT-3.5 has been proved to have positive correlation with human annotations (Chiang and Lee, 2023a;Wang et al., 2023a)”
From Bayesian Calibration of Win Rate Estimation with LLM Evaluators (2024) · cites this paper as Chiang and Lee, 2023a
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent efforts using LLMs as evaluators (LLM-as-a-judge (Zheng et al., 2023)) show parity with human judgment (Chiang and Lee, 2023b)”
From YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering (2025) · cites this paper as Chiang and Lee, 2023b
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“a growing literature finds that LLM ratings of text quality track human ratings (Chiang and Lee, 2023; Liu et al., 2023; Zheng et al., 2023)”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Chiang and Lee, 2023
The cited source says what the claim attributes to it.
In the citing paper
“Recently, the remarkable instruction-following ability of LLMs has underscored their potential for automatic evaluation [51, 190, 314].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242
Research works printed but not linked: 3
- R Botsch. 2011. Chapter 12: Significance and mea- sures of association. Scopes and Methods of Political Science.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj, Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Cited by
Sorted by citations
Items 401–477 of 477
Large Language Model Evaluation
坂井, 優. (2024) · Institutional Repositories DataBase (IRDB) · doi:10.34413/dr.02100
Evaluasi Multi-Expert terhadap Sistem Rekomendasi Diet Non-Klinis Berbasis Generative AI
Mukafasyadiah, D., & Setiawan, M. A. (2026) · Jurnal Sistem Komputer dan Informatika (JSON) · doi:10.30865/json.v7i3.9587
EvaluAId: Human-AI Collaborative Evaluation of Open-Ended Student Essays
Zhang, C., Ju, K. P., Lu, X., Yen, Y.-C. G., & Rzeszotarski, J. M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790814 · Cited by 2
Large Language Models as MT Evaluators: Statistical Evidence from a Low-Resource Setting
Munková, D., Munk, M., Chotar, M., & Hajek, P. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11468166
Hu, G., Wei, T., Hou, H., Chen, Y., Zhao, Z., & Pu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7078-2_13
Which Private Attributes do VLMs Agree on and Predict Well?
Hrynenko, O., Baranouskaya, D., Baia, A. E., & Cavallaro, A. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461923
LLM-Driven MCTS for Conditional Story Generation via Logic-Guided Evidence Tree Optimization
Wu, H., Tian, Z., Huang, Z., Lin, N., Song, Y., Wen, Z., Lu, M., Liu, F., & Li, D. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3687020
Bojić, I., Ravaut, M., Ma, S. H. X., Tan, D., Ho, A. H. Y., & Khong, A. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54des2aenesc
Can LLMs Replace Human Raters? Evaluating EFL Learners’ English Summaries
Kato, M. (2026) · International Education Studies · doi:10.5539/ies.v19n3p31
Cutillas, A. L., Bacalso, F., Tomol, C. J., Albarracin, M., Campita, R. A., Benolirao, E., Yamagishi, K. D., & Ocampo, L. A. (2026) · Algorithms · doi:10.3390/a19050406
A First-Order Logic-Based Alternative to Reward Models in RLHF
Jiang, C., & Zhu, X. (2026) · ICON · doi:10.1109/icnlp69856.2026.11527781
Next-Generation AI: A Review of Large Language Models, Pipelines, and Security Challenges
Gupta, A., & Kumari, D. R. (2025) · International Conference on Information Management & Machine Intelligence (ICIMMI) · doi:10.1145/3793449.3793505
Proxy Analysis in Bias Testing
Djennane, L., Kardkovács, Z. T., Benatallah, B., Gaci, Y., & Farah, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28117-3_15
Can LLM Agents Really Simulate Clients in Counseling?
Yu, S., Kim, D., Roh, Y., Kwon, K., & Oh, H. (2026) · ACM Symposium on Applied Computing · doi:10.1145/3748522.3779828
A Controlled Study of Decoding-Time Truthfulness Methods on Instruction-Tuned LLMs
Sun, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_15
Automated radiology report generation: A comprehensive review
Huang, L., Islam, T., Miron, A., HONE, K. S., & Li, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133320
Limaye, D., Shah, R., & Jonelgadda, C. (2026) · Research Square · doi:10.21203/rs.3.rs-10047888/v1
Closing the Feedback Loop: Automated Quality Assurance for LLM-Generated Feedback
Bulut, O., & Walsh, C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29788-4_19
Bhat, S., & Varma, V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29763-1_18
A Review of Automated Short Answer Grading Systems
Karan, V. K. V., & Dewan, M. A. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30015-7_12
Revisiting Multi-Level Network Modeling and Simulation With Domain-Adapted LLMs
Wu, D., Liu, J., Lin, X., Xu, D., Gao, P., & Liu, Y. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3708012
Automated bias blind spots: Examining stereotype associations in VLM-as-a-judge paradigms
Hong, R. S., Ovalle, A., Ung, M., Spiliopoulou, E., Sagun, L., Williams, A., & Ross, C. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806459 · Cited by 1
A Case Study in Building a Cloud-Based AI-Powered Interview Platform
Patel, R. K., Sharma, Y., Rama, J., & Patel, N. R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9725-2.ch011
Propaganda Signals in LLMs: Perspectival Divergence and Narrative Framing in the Russia-Ukraine War
Shabat, O., Guy, I., & Radinsky, K. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.557
Ghost Context: Measuring Cross-Context Interference in Long-Context Language Models
Namboothiri, R. (2026) · Workshop on Trustworthy NLP (TrustNLP) · doi:10.18653/v1/2026.trustnlp-main.19
RSDA: Restoring Stale Data Affinity via Dynamic Renovation Strategy for Mitigating Data Scarcity
Liang, Y., Zhu, J., Shi, W., Guo, H., Cui, Y., Shen, J., Ma, G., Liu, J., Niu, Q., Wang, Y., et al. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.375
Uchida, M., & Nakanishi, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30836-8_21
Tang, W., Gui, T., Ma, H., & Chen, M. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581585
Tang, R., & Wu, F. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582239
Martín, C. A., Torres, J., Aguilar, R. M., Alayón, S., & Bacallado, M. A. (2026) · Applied Sciences · doi:10.3390/app16136793
Liu, C., Qu, R., Yu, J., Jin, F., He, J., Wang, J., & Pan, J. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2026.2698388
Can LLMs Accurately Score Medical Diagnoses and Clinical Reasoning?
Rouillard, A., Mundia, S., Camara, L., Dangor, Z., Gramanie, M. C., Kalla, I., Madhi, S. A., Morar, K., Ncube, M. T., Saloojee, H., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30710-1_44
Twelve Voices, One Decision: Evaluating a Persona-Memory FOMC Simulator Across Seventeen MeetingsPreprint
Suh, J. Y., & Kim, S.-H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6932158
Sucal, V., Jullien, M., Njifenjou, A., & Lefèvre, F. (2026) · Journées d'Études sur la Parole (JEP) · doi:10.21437/jep.2026-43
Enhancing bias detection in text classification models through high-quality diverse test cases
Djennane, L., Kardkovács, Z. T., Benatallah, B., Gaci, Y., Gaaloul, W., & Farah, Z. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108253
Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments
Christakopoulou, E., Patel, V., Velaga, H., & Gaikwad, S. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808460
Toward explaining value concerns in software artifacts: A human-in-the-loop method
Mougouei, D., Pham, S. V., Azarnik, A., Fahmideh, M., Nurwidyantoro, A., Mougouei, E., Haque, A., Dam, H. K., Rafi, S., & Khan, J. A. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108254
da Silva, D. C. A., Ferreira, C. L., & Silva, S. D. S. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25035-3_21
AutoChecklist: Automated Checklist Refinement for LLM Judges
Uniyal, M., Singh, M. P., Verbruggen, G., Le, V., & Gulwani, S. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805263
Song, W., Wang, X., Liang, S., & Wang, J. (2026) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2026.3716609
Capital Markets LLM Reliability Score (CM-LRS): From Plausible to BankablePreprint
Ahuja, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6865059
Robustly Grounding LLM-Derived Meta-Causality for Causal Discovery
Wang, X., Ban, T., Chen, L., Gao, Y., Lin, X., Zhou, X., Lyu, D., & Chen, H. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3718207
From Issues to Insights: RAG-based Explanation Generation from Software Engineering Artifacts
Pöttgen, D., Sadeghi, M., Unterbusch, M., & Vogelsang, A. (2026) · IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE) · doi:10.1145/3786164.3788448
Cross-Linguistic Epistemic Variance in Large Language Model Cognition
Nizami, M. S., Khan, M. R. S., & Bilal¹, M. (2026) · International Journal of Innovations in Science and Technology · doi:10.33411/ijist/1932 · Cited by 1
Evaluating Reliability and Bias of Large Language Models in Automated Essay Scoring
Lee, Y. (2026) · Computers in the Schools · doi:10.1080/07380569.2026.2712300
Diagnosing Evidence Utilization in Multimodal Document Question Answering
Basak, D., Bose, D., Goswami, K., & Desarkar, M. S. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817956
CALM-TS: Risk-Controlled LLM Labeling for Time-Series via Calibrated Selective Gating
XIAO, Y., Li, S., & Huang, Y. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817859
Pandilova, E., Petrov, M., Chorbev, I., Gjorgjevikj, D., & Trajanov, D. (2026) · Information · doi:10.3390/info17080790
Tchuindjang, M., Duran, N., Legg, P., & Medjek, F. (2026) · Cybersecurity · doi:10.1186/s42400-026-00633-z
Bahri, M., Zhang, R., Alshakhsi, S. A. A., Dou, D., Al‐Thani, D., Yankouskaya, A., & Ali, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7354076
LLM-Based Annotation and Benchmarking for ABSA in Bengali-English Code-Mixed E-Commerce Reviews
Papri, I. J., Hossain, M. F., Masud, M. A., & Golder, A. (2026) · International Conference on Power, Electronics, Communications, Computing, and Intelligent Infrastructure (PECCII) · doi:10.1109/peccii70991.2026.11662020
Falcón, S. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-1042-8.ch006
Indirect Prompt Injection in Municipal Document-Processing Copilots: Attacks, Defences and Harm
Cisneros-González, J., Ondiviela, J. A., & Sánchez-Soriano, J. (2026) · Street Art & Urban Creativity · doi:10.62161/sauc.v12.6399
Towards Task-Agnostic Evaluation Framework for LLM-Embedded Applications
Widanapathiranage, D. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3774748.3787633
EmbedCopilot-Bench: Evaluating Vision-Language Models for Hardware-Aware Embedded System Development
Yuan, D., Deng, Y., & Chen, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37132-4_19
Assessing Mental Health Counseling Interventions with LLMs
Naher, N., Langfus, J. A., Pfiffner, L., & Sukthankar, G. (2025) · 2025 IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat67162.2025.00068
Lee, G., Kim, Y., Youn, H., Roh, H., & Borrmann, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7450622
Zhong, Y., El-Diraby, T. E., & Goodfellow, S. D. (2026) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-7192
Evaluating Spoken Language Features in Conversational Models: The Case of Disfluencies and Feedbacks
Silem, O., Fleig, M., Blache, P., Oufaida, H., & Becerra-Bonache, L. (2025) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2025.sigdial-1.23
Tchuindjang, M., Duran, N., Legg, P., & Medjek, F. (2026) · International Symposium on Networks, Computers and Communications (ISNCC) · doi:10.1109/isncc70543.2026.11693865
Designing Reliable LLM-as-a-judge Measurement Systems for Multi-turn Business AgentsPreprint
Luo, K., & Gao, M. (2026) · Research Square · doi:10.21203/rs.3.rs-11184291/v1 · Cited by 1
Automated Essay Marking with Data-Efficient Augmentation and Score-Token Weighted Loss
Shaka’a, Y., Aburumman, N., Anagnostou, A., Taylor, S., & Li, Y. (2026) · International Conference on Computer Vision and Data Mining (ICCVDM) · doi:10.1109/iccvdm71812.2026.11709134
Zhu, X., Wu, Z., Wang, T., Yan, Y., Zhou, W., Wu, J., Jiang, Z., & Zheng, Z. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832292
Hong, S., Kim, J., Kim, J., Park, S.-Y., Kim, Y., & Kang, P. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21924-0
On-Device GitHub Contributions Audit utilizing Recursive Language Models
Effendy, C. L., Djayadi, D., Wati, P. M., & Saputri, T. R. D. (2026) · 2026 3rd International Conference on Technology Innovation and Its Applications (ICTIIA) · doi:10.1109/ictiia70925.2026.11717373
Al-Ghamdi, S. A., Al-Khalifa, H., Al-Khalifa, S., Panda, S., Agarwal, A., & Patel, H. L. (2026) · Universal Access in the Information Society · doi:10.1007/s10209-026-01387-8
Advancing depression detection with role-playing LLMs and synthetic psychological interviews
Liu, Y., Hu, J., Chen, Y., & Zhang, Y. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03337-5
Lee, J., & Oh, S. H. (2026) · Applied Sciences · doi:10.3390/app162010253
Li, K., Sun, Y., Shan, M., & Yang, Z. J. (2026) · MRS Energy & Sustainability · doi:10.1557/s43581-026-00170-9
Tsaneva, S., Herwanto, G. B., Llugiqi, M., & Sabou, M. (2026) · Semantic Web · doi:10.1177/22104968261499449
QatarGuard: A Culturally-Aware Safety Benchmark for Arabic and Multilingual LLMs
Zaghouani, W., Aldous, K. K., Biswas, M. R., & Abdelkreem, M. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1710
An LLM-Assisted Fuzzing Framework for Vehicle ECU Vulnerability Discovery
Huo, Q., Ning, Y., Mu, S., & Sun, D. (2026) · doi:10.1109/eecct71142.2026.11746521
Yan, Z., Jiang, Z., Ma, C., Dong, F., Tong, Z., & Lin, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-42929-2_18
An Empirical Analysis of Formatting Bias in Automated LLM-Based Code Assessment
Gunawan, V. V., Welen, C. J., Gunawan, A. A. S., & Pradana, R. C. (2026) · International Conference on Information Technology Research and Innovation (ICITRI) · doi:10.1109/icitri71999.2026.11751339
Qonita, Q., Faraby, S. A., Gozali, A. A., Suwastika, N. A., Ramdania, D. R., & Adiwijaya, A. (2026) · International Conference on Data Science and Its Applications (ICoDSA) · doi:10.1109/icodsa71861.2026.11754593
Metadata
- Authors
- Chiang, C.-H., & Lee, H.-Y.
- Affiliations
- National Taiwan University (TW)
- Corresponding author
- Lee, Hung-yi
- ORCID
- Lee, Hung-yi
- Venue
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2023
- Pages
- 15607-15631
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4385571232
- Cited by
- 331
- References
- 40
- Citations by year
- 2026: 90 · 2025: 130 · 2024: 80 · 2023: 31
- FWCI
- 58.65
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Delta Electronics
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- large language models, LLM evaluation, human evaluation, text quality assessment, adversarial attacks, assessment consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
