When "a helpful assistant" is not really helpful: Personas in system prompts do not improve performances of large language models
Open accessAbstract
Prompting serves as the major way humans interact with Large Language Models (LLM).Commercial AI systems commonly define the role of the LLM in system prompts.For example, ChatGPT uses "You are a helpful assistant" as part of its default system prompt.Despite current practices of adding personas to system prompts, it remains unclear how different personas affect a model's performance on objective tasks.In this study, we present a systematic evaluation of personas in system prompts.We curate a list of 162 roles covering 6 types of interpersonal relationships and 8 domains of expertise.Through extensive analysis of 4 popular families of LLMs and 2,410 factual questions, we demonstrate that adding personas in system prompts does not improve model performance across a range of questions compared to the control setting where no persona is added.Nevertheless, further analysis suggests that the gender, type, and domain of the persona can all influence the resulting prediction accuracies.We further experimented with a list of persona search strategies and found that, while aggregating results from the best persona for each question significantly improves prediction accuracy, automatically identifying the best persona is challenging, with predictions often performing no better than random selection.Overall, our findings suggest that while adding a persona may lead to performance gains in certain settings, the effect of each persona can be largely random.Code and data are available at https://github.com/Jiaxin-Pei/ Prompting-with-Social-Roles.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior evidence suggests limited impact of persona ablation on task performance in similar setups.
“A dedicated persona ablation was omitted due to significant computational cost and prior evidence suggesting limited impact on task performance in similar setups (Zheng et al., 2024).”
From Voting or Consensus? Decision-Making in Multi-Agent Debate (2025) · cites this paper as Zheng et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 5
- Orianna Demasi, Yu Li, and Zhou Yu. 2020. A multi- persona chatbot for hotline counselor training. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 3623-3636.
- Erving Goffman. 2016. The presentation of self in everyday life. In Social Theory Re-Wired, pages 482- 493. Routledge.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Con- ference on Learning Representations (ICLR).
- Risto Miikkulainen. 2007. Creating intelligent agents in games. In Frontiers of engineering: Reports on leading-edge engineering from the 2006 symposium, page 15. National Academies Press.
- Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances in Neural In- formation Processing Systems, 33:5776-5788.
Cited by
Sorted by citations
Song, I., Park, S., Pendse, S. R., Schleider, J. L., Choudhury, M. D., & Kim, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713883 · Cited by 42 · see citing works
Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
Neumann, A., Kirsten, E., Zafar, M. B., & Singh, J. P. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732038 · Cited by 28 · see citing works
The Impact of Prompt Programming on Function-Level Code Generation
Khojah, R., de Oliveira, F. D., Mohamad, M., & Leitner, P. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3587794 · Cited by 48 · see citing works
Ahlgren, T. L., Sunde, H. F., Kemell, K., & Nguyen‐Duc, A. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107832 · Cited by 15 · see citing works
Voting or Consensus? Decision-Making in Multi-Agent Debate
Kaesberg, L. B., Becker, J., Wahle, J. P., Ruas, T., & Gipp, B. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.606 · Cited by 14 · see citing works
Ray, U., Arteaga, C., Ahn, Y., & Park, J. (2024) · Engineering Construction & Architectural Management · doi:10.1108/ecam-09-2024-1259 · Cited by 16 · see citing works
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
de Araujo, P. H. L., & Roth, B. (2025) · PLoS ONE · doi:10.1371/journal.pone.0325664 · Cited by 11 · see citing works
Zhu, Y., Lu, Y., Xie, H. X., Ye, J., & Chen, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104365 · Cited by 11 · see citing works
Schorcht, S., Müller, F. A., & Buchholtz, N. F. (2026) · ZDM · doi:10.1007/s11858-026-01784-6 · Cited by 13 · see citing works
Phelps, S., & Russell, Y. I. (2025) · Journal of Physics Complexity · doi:10.1088/2632-072x/ada711 · Cited by 35 · see citing works
Grazhdanski, G., Vasilev, V., Vassileva, S., Taskov, D., Antova, I., Koychev, I. K., & Boytcheva, S. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104906 · Cited by 10 · see citing works
Synthetic imaginaries of “sensitive” AI: On ambient amplification and jail(break)ing as method
Pilipets, E., & Geboers, M. (2025) · Platforms & Society · doi:10.1177/29768624251378110 · Cited by 9 · see citing works
Simulating social perception with large language models: perceptions of China’s common prosperity
Jiang, Z., Huang, B., Ge, J., Lin, C., Xu, Y., & Yu, J. (2025) · Journal of Chinese Governance · doi:10.1080/23812346.2025.2603822 · Cited by 9 · see citing works
Nugumanova, A., Alzhanov, A., Mansurova, A., & Мансурова, М. (2025) · Electronics · doi:10.3390/electronics14112098 · Cited by 5 · see citing works
Long context window-based zero-shot legal interpretation of building codes and regulations
Lee, J., & Lee, G. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106450 · Cited by 6 · see citing works
Chen, Z., Liu, C., Ren, Y., Song, G., & Zhang, X. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100827 · Cited by 4 · see citing works
Enhancing design concept diversity: multi-persona prompting strategies for large language models
Feng, W. B., Hélie, S., & Panchal, J. H. (2025) · Design Science · doi:10.1017/dsj.2025.10037 · Cited by 4 · see citing works
Generative AI Chatbot Prompting for Excellent Customer Service in Tourism
Kekäläinen, T., Heinonen-Kemppi, J., Pesonen, J. A., Sthapit, E., & Garrod, B. (2026) · Services Marketing Quarterly · doi:10.1080/15332969.2025.2610137 · Cited by 4 · see citing works
Large Language Models for Ontology Engineering: A Systematic Literature Review
Li, J., Garijo, D., & Poveda‐Villalón, M. (2026) · Semantic Web · doi:10.1177/22104968261465514 · Cited by 9 · see citing works
Barbera, D. L., Lunardi, R., Zhuang, M., & Roitero, K. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744581 · Cited by 4 · see citing works
Performance of vision language models for optic disc swelling identification on fundus photographs
Li, K. Z., Nguyen, T. T., & Moss, H. E. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1660887 · Cited by 4 · see citing works
Agentic Recommender Systems: A Systematic Literature Review
Portugal, I. D. S., Alencar, P., & Cowan, D. D. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3657900 · Cited by 3 · see citing works
Tensen, D., Carey, M. D., & Grainger, P. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2026.2619902 · Cited by 4 · see citing works
Molaei, S., Robert, L., & Banović, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757437 · Cited by 2 · see citing works
Using AI for User Representation: An Analysis of 83 Persona Prompts
Salminen, J. O., Amin, D., & Jansen, B. J. J. (2025) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa66935.2025.11315350 · Cited by 2 · see citing works
Large Language Model–Based Agents for Physical Activity and Cognitive Training: Scoping Review
Silacci, A., Giachetti, B., Angelini, L., Lopomo, N. F., Andreoni, G., Mugellini, E., Cherubini, M., & Caon, M. (2026) · JMIR AI · doi:10.2196/80123 · Cited by 2 · see citing works
Mousavi, R., Kitchens, B., Oliver, A. G., & Abbasi, A. (2026) · Information Systems Research · doi:10.1287/isre.2024.1143 · Cited by 2 · see citing works
Zero-Shot Prediction of Conversational Derailment With Large Language Models
Nonaka, K., & Yoshida, M. (2025) · IEEE Access · doi:10.1109/access.2025.3554548 · Cited by 3 · see citing works
Steinmann, I., Stovner, R. B., Hatlevik, O. E., Øgreid, A. K., & Herseth, J. (2025) · European Journal of Teacher Education · doi:10.1080/02619768.2025.2528812 · Cited by 2 · see citing works
The Effects of Demographic Instructions on LLM Personas
de Paula, A. F. M., Culpepper, J. S., Moffat, A., Cherumanal, S. P., Scholer, F., & Trippas, J. R. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730255 · Cited by 3 · see citing works
An Empirical Study of Vulnerabilities in Python Packages and Their Detection
Quan, H., Wang, J., Li, X., Zhuo, T. Y., Chen, X., & Du, X. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793357 · Cited by 3 · see citing works
LLM Shots: Best Fired at System or User Prompts?
Halil, U., Huang, J., Graux, D., & Pan, J. Z. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717814 · Cited by 1 · see citing works
Bounding Reason: Inferentialism, Naturalism, and the Discursive Agency of LLMs
Malík, J., & Hubálek, M. (2025) · Global Philosophy · doi:10.1007/s10516-025-09761-6 · Cited by 1 · see citing works
A Role-Aware Multi-Agent Framework for Financial Education QA
Zhu, A., & Du, Y. (2025) · International Conference on AI in Finance · doi:10.1145/3768292.3770345 · Cited by 2 · see citing works
Active Personas for Synthetic User Feedback: A Design Science Study
Simaremare, M. E. S., & Edison, H. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-14518-5_20 · Cited by 1 · see citing works
Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J. P., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 2 · see citing works
Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Python
Adnan, M., & Kuhn, C. C. N. (2026) · International Conference on Advances in Artificial Intelligence and Machine Learning (AAIML) · doi:10.1109/aaiml67890.2026.11498216 · Cited by 1 · see citing works
The ballad of LLM agents: philosophical reasoning for chemistry
Harb, H., Sun, Y. P., Unal, M., Chia, N., Yang, Z., Ingram, B. J., & Assary, R. S. (2026) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae792d · Cited by 1 · see citing works
Ma, C.-X., Peng, F.-L., Li, Z.-J., & Qiao, Y. (2026) · ENGINEERING Structure and Civil Engineering · doi:10.1007/s11709-026-1329-1 · Cited by 1 · see citing works
Nakagawa, S., & Yamamoto, A. (2026) · Nutrients · doi:10.3390/nu18172892 · Cited by 1 · see citing works
Buskirk, T. D., Eck, A., & Timbrook, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5377878 · Cited by 1 · see citing works
Personas Shift Clinical Action Thresholds in Large Language ModelsPreprint
Klang, E., Gorenshtein, A., Omar, M., & Nadkarni, G. N. (2026) · medRxiv · doi:10.64898/2026.01.01.26343302 · Cited by 1 · see citing works
Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks
Haase, J., Gonnermann-Müller, J., Hanel, P. H. P., Leins, N., Kosch, T., Mendling, J., & Pokutta, S. (2026) · arXiv · doi:10.48550/arxiv.2601.21339
Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration
Xu, J., Wu, X., Zhang, Z., Xu, Y., Yang, Y., Zhu, J., Xiao, L., Wu, W., & He, L. (2026) · arXiv · doi:10.48550/arxiv.2606.18259
Mikhailova, S. S., Timofeev, A. N., Timoshenko, A. V., & Zakharov, A. S. (2025) · 2018Systems of Signal Synchronization, Generating and Processing in Telecommunications (SYNCHROINFO) · doi:10.1109/synchroinfo65403.2025.11079329
Liu, Q., Zuo, Y., Ye, R., Chang, Y., Zhang, X., & Wu, L. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-1525-7_2
GenAI in Patent DraftingPreprint
Xie, W. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5392009
Wagner, M., Frideres, F., Schürmann, M., Klar, M., & Aurich, J. C. (2025) · Lecture notes in mechanical engineering · doi:10.1007/978-3-032-07675-5_6
Design and Implementation of a Companion Robot with LLM-Based Hierarchical Emotion Motion Generation
Lim, Y., Cho, J., Lee, D.-Y., Choi, D., & Lee, D.-W. (2025) · Applied Sciences · doi:10.3390/app152312759
Popescu-Cevei, P. L., & Santa, I. (2025) · IEEE International Symposium for Design and Technology in Electronic Packaging (SIITME) · doi:10.1109/siitme67657.2025.11293702
An, M., Kwon, C., Lee, Y., Hur, J.-H., Lee, D., Huai, V., Zheng, B., Yu, M., Liu, J., Pugh, J., et al. (2026) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3770762.3772536
Keyword-Centered Rescheduling for LLM Agents
Ji, Y., Lee, I., & Sung, Y. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10561-2
TRUST: Transparent, Robust and Ultra-sparse Trees
Dorador, A. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7081-2_5
Does It Still Make Sense? Organizing and Summarizing Resources in Cross-Session Aggregated Search
Momeni, M., & Hoeber, O. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787873 · Cited by 1 · see citing works
Mapping Research on Social Cognitive Bias in Large Language Models
Zhang, S., Li, J., Gao, Y., Li, Y., Zhang, X., & Li, X. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00159
Take a Deep Breath: Enhancing LLM Performance on Math Word Problems Through Prompting
Jiang, N. (2025) · Artificial Intelligence x Humanities, Education, and Art (AIxHEART) · doi:10.1109/aixheart65685.2025.00019
Tang, Y., Liu, Z., Chen, Z., Yang, J., Xiang, L., Xu, Z., & He, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-9875246/v1
The Context Dependency of Demographic Bias in LLM Annotation
Brown, M. A., Atreja, S., Hemphill, L., & Wu, P. Y. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812212
Wang, C.-C., Liu, F.-C., Xie, J., Lai, C.-M., & Lin, C.-S. (2026) · Education and Information Technologies · doi:10.1007/s10639-026-14086-x
LLMPT: A Template for Prompt Documentation in Scientific Publications
Steuck, P.-F., Bierschwale, D., Schmidt, R., Shidujaman, M., & Knackstedt, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29596-5_12
An Agentic System for Context-Aware Synthetic Data Generation
Rubaiat, S. Y., Sakib, S. N., & Jamil, H. M. (2026) · Proceedings of the Workshop on Synthetic Data Generation and Management for Building AI Systems · doi:10.1145/3814574.3816750
Beyond Benchmarks: A User-Centric Framework for Evaluating Large Language Models
Abdrabou, Y., Abdelrahman, Y., Bozkir, E., Mazen, Y., Alt, F., & Kasneci, E. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816227
Persona-as-Configuration: Generative Stakeholder Reporting for Agricultural Floods
Larsen, O. A., Santilli, T., Daghero, F., & Moghaddam, M. T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39143-8_12
On the Limited Effect of Role Prompting in LLM-based Code Vulnerability Detection
Guan, J., Wang, H., Yin, Y., & Zhao, Y. (2026) · International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE) · doi:10.1109/icbase70763.2026.11619486
Challenges in Automatic Analysis of Privacy Policy Documents with LLMs
Laakso, R., Harshani, S., Salmi, S., Rajapaksha, S., & Rauti, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32023-0_38
Much Ado about Prompting: LLM classification of text messages from experiments
Çelebi, C., & Penczynski, S. P. (2026) · PLoS ONE · doi:10.1371/journal.pone.0354757
Ciotti, F. (2026) · DigitCult - Scientific Journal on Digital Cultures · doi:10.54103/2531-5994/31801
Stifini, D., Penna, A. D., Mihaljevic, A. L., Bitzer, M., Eickhoff, C., & Capobianco, I. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03069-4
Abejez-Arrizabalaga, Á., Pellejero-Sagastizábal, G., Aznar-Gimeno, R., Franco, B., Letona-Giménez, S., Morte-Romea, E., Origüen, J., Pérez-Rodríguez, M. T., Retamar, P., Ruizendaal, E., et al. (2026) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2026.08.025
Lin, Y., & Xiao, N. (2026) · Spatial Cognition and Computation · doi:10.1080/13875868.2026.2730164
Is that Really You? Understanding Persona Adoption in LLM-Based Requirements Engineering
Morente, P., Motger, Q., & Oriol, M. (2026) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re68928.2026.00052
Modeling Like Peeling an Onion: Layerwise Analysis-Driven Automatic Behavioral Model Generation
Huang, Y., Hu, M., Chen, X., Jin, Z. G., & Xiao, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787806
Generative AI Technologies, Techniques and Tensions: A Primer
Behrens, J. T. (2026) · Springer eBooks · doi:10.1007/978-3-032-24819-0_2 · Cited by 1 · see citing works
Role Prompting: Style or Substance? Evaluating Expert Personas in Large Language Models
Leiba, M., Haviv, I., Rasin, L., & Leibu, I. (2026) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew72749.2026.00067
Evaluating the Robustness of Safety-Aligned LLM Behavior to Short Contextual PrefixesPreprint
Cholewa, M., & Głomb, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10590385/v1
Prompt-Enhanced Multi-Layer Fusion Embeddings for Text Classification
Lv, J., Qin, D., & Xu, C. (2026) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml71097.2026.11687057
Do AI stories align with the laws of the story economy?
Raipola, J., Mäkelä, M., Björninen, S., & Piippo, L. H. (2026) · Narrative Inquiry · doi:10.1075/ni.25163.rai
Multimodal LLM Pipeline for Automated Analysis of Technical Graphical Documents
Studniarz, S., & Maczka, M. (2026) · AI · doi:10.3390/ai7100403
Integrating agentic RAG and multi-agent data augmentation for dog-whistle hate speech detection
Lin, Y., Wang, Z., Yin, D., & Zhao, X. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03316-w
Nakagawa, S., & Yamamoto, A. (2026) · medRxiv · doi:10.64898/2026.10.09.26365134
Pistofidis, P., Petrou, A., & Botsaris, P. N. (2026) · Technologies · doi:10.3390/technologies14100666
Prompt Engineering for Small Language Models: Evaluating ICL for Portuguese Sentiment Analysis
Schuck, A. D. F., Garcia, G. L., Manesco, J. R. R., Paiola, P. H., Passos, L. A., & Papa, J. P. (2026) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2026.25253
ExpertTwins: Auditable Long Horizon Scientific Panels through Grounded ExpertisePreprint
pour, A. F., & Shirinifard, A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.06.757114
He, M., Ji, W., Ran, J., Cui, J., Yan, W., & Liang, J. (2026) · Applied Cognitive Psychology · doi:10.1002/acp.70297
Cheimariotis, I., Costa, J. N., Piazza, F., Lodi, C., Rataj, M., Zawieska, J., Stępniak, M., & Grosso, M. (2026) · European Transport Research Review · doi:10.1186/s12544-026-00854-w
- Authors
- Zheng, M., Pei, J., Logeswaran, L., Lee, M., & Jurgens, D.
- Affiliations
- Stanford University (US)
- Carnegie Mellon University (US)
- University of Illinois Chicago (US)
- University of Michigan (US)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Published
- 2024
- Pages
- 15126-15154
- OpenAlex
- W4404792888
- Cited by
- 82
- References
- 0
- Citations by year
- 2026: 48 · 2025: 33 · 2024: 1
- FWCI
- 24.24
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2143529)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- system prompts, personas, large language models, prompting, role-based prompting, factual question answering, interpersonal relationships, ChatGPT
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
