The DOI matches its Crossref record; 30 of the 32 references it cites were found in a registry or an index.
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Open accessAbstract
A key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly. In this work, we introduce StrategyQA, a question answering (QA) benchmark where the required reasoning steps are implicit in the question, and should be inferred using a strategy. A fundamental challenge in this setup is how to elicit such creative questions from crowdsourcing workers, while covering a broad range of potential strategies. We propose a data collection procedure that combines term-based priming to inspire annotators, careful control over the annotator population, and adversarial filtering for eliminating reasoning shortcuts. Moreover, we annotate each question with (1) a decomposition into reasoning steps for answering it, and (2) Wikipedia paragraphs that contain the answers to each step. Overall, StrategyQA includes 2,780 examples, each consisting of a strategy question, its decomposition, and evidence paragraphs. Analysis shows that questions in StrategyQA are short, topic-diverse, and cover a wide range of strategies. Empirically, we show that humans perform well (87%) on this task, while our best baseline reaches an accuracy of ∼ 66%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“3 . 7% of questions are classified as requiring reasoning and 7 . 3% are multi-step.”
From SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge (2025) · cites this paper as Geva et al., 2021
In the citing paper
“Compared to fine-tuned models, LLMs keep the superiority on most datasets, such as StrategyQA [40] and ARC-C [28].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [40]
Citations
Cites
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P. E., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1260 · Cited by 1,602
Self-citationAnnotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1300 · Cited by 655
A Corpus for Reasoning about Natural Language Grounded in Photographs
Suhr, A., Zhou, S., Zhang, A. Q., Zhang, I., Bai, H., & Artzi, Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1644 · Cited by 634
Flagged first, then by citations
2 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Robertson, 1995, Okapi at TREC-3, 109
Cited by
Sorted by citations
Items 201–300 of 348
M o N a C o : More Natural and Complex Questions for Reasoning Across Dozens of Documents
Wolfson, T., Trivedi, H. K., Geva, M., Goldberg, Y., Roth, D., Khot, T., Sabharwal, A., & Tsarfaty, R. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.64 · Cited by 12
Self-citationSelect prompting with chain-of-thought paired with large language models
Che, X., Wu, W., Chen, Y., Jiang, L., & Li, Q. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131511 · Cited by 3
Wang, J., Zheng, C., Tang, Z., & Xie, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131704 · Cited by 2
Data Preparation for Large Language Models
Liang, H., Wong, Z. H., Liu, R., Wang, Y.-H., Qiang, M.-Y., Zhao, Z.-Y., Shen, C.-Y., He, C.-H., Zhang, W.-T., & Cui, B. (2026) · Journal of Computer Science and Technology · doi:10.1007/s11390-026-5948-8 · Cited by 6
Sun, X., Wei, S., Bosch, J. A., Echizen, I., Sugawara, S., & Ali, A. E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798613 · Cited by 2
UA-RAG: Uncertainty-aware dynamic retrieval-augmented generation
Niu, M., Su, J., & Tian, C. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134357 · Cited by 2
RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoning
Liu, Y., Wang, R., Chu, Y., Hao, B., Lin, Y., Liu, S., & Shao, M. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818056 · Cited by 2
Ali, Z., Aziz, A., Ali, A., Ullah, A., Aurangzeb, M., & Nazir, M. A. (2023) · International Conference on Machine Vision, Image Processing and Imaging Technology (MVIPIT) · doi:10.1109/mvipit60427.2023.00028 · Cited by 2
Complex Reasoning in Natural Languag
Zhao, W., Geva, M., Lin, B. Y., Yasunaga, M., Madaan, A., & Yu, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-tutorials.2 · Cited by 5
Self-citationGradually Excavating External Knowledge for Implicit Complex Question Answering
Liu, C., Li, X., Shang, L., Jiang, X., Liu, Q., Lam, E. Y. M., & Wong, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.961 · Cited by 2
A Brief Study of Prompting Techniques for Reasoning Tasks
Gaur, S., Dagar, A., Punia, A., & Kumar, P. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3601-0_12 · Cited by 2
Improving Factuality by Contrastive Decoding with Factual and Hallucination Prompts
Lv, B., Feng, A., & Xie, C. (2024) · Sensors · doi:10.3390/s24217097 · Cited by 3
Evaluating the Rationale Understanding of Critical Reasoning in Logical Reading Comprehension
Kawabata, A., & Sugawara, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.9 · Cited by 3
DINGO: Towards Diverse and Fine-Grained Instruction-Following Evaluation
Gu, Z., Sun, X., Lian, F., Kang, Z., Xu, C., & Fan, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29768 · Cited by 3
Understanding Logical Reasoning Ability of Large Language ModelsPreprint
Chan, E. (2024) · SSRN Electronic Journal · doi:10.20944/preprints202408.1712.v1 · Cited by 2
A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory
Wei, Q., Yang, T., Wang, Y., Li, X., Li, L., Yin, Z., Zhan, Y., Holz, T., Lin, Z., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2510.02373 · Cited by 1
“…the performance against a direct poisoning attack, we follow the configuration of (Chen et al., 2024) which uses a knowledge-intensive QA agent operating on the ReAct-StrategyQA (Geva et al., 2021), and a healthcare agent managing electronic health records in the EHRAgent (Shi et al., 2024).”
InfluentialSimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge
Haas, L., Yona, G., D'Antonio, G., Goldshtein, S., & Das, D. (2025) · arXiv · doi:10.48550/arxiv.2509.07968 · Cited by 1
“Metadata enrichment: using a prompted classifier, we detect which questions might require reasoning capabilities (i.e. calculating the difference between two dates) and which questions are multi-step (requiring multiple sequential information steps) in nature (Geva et al., 2021; Yang et al., 2018).”
Multi-LLM Collaborative Search for Complex Problem Solving
Yang, S., Li, Y., Lam, W., & Cheng, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.18873 · Cited by 1
HCoTT: Hierarchical Chain-of-Thought Distillation
Wang, Z., Zhuang, X., Zhu, Z., & Zou, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890771 · Cited by 1
Program Verification to Defend Chain-of-Thought Attacks for LLM Services
He, S., Jiang, J., Liu, H., & Tang, T. (2025) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws67624.2025.00031 · Cited by 1
Peng, Z., Liu, X., Yang, G., Liu, J., Peng, X., & Long, Y. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.11.040 · Cited by 1
Murugaraj, K., Lamsiyah, S., & Theobald, M. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402490 · Cited by 1
Liu, J., Du, W., Li, Q., Wang, X., Zhou, Z., & Liu, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448276 · Cited by 1
Prompt Engineering 101 Prompt Engineering Guidelines from a Linguistic Perspective
Han, W., Wei, X., Cui, X., Cheng, N., Jiang, G., Qian, W., & Zhang, C. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8367-0_34 · Cited by 3
Ezemba, J., McComb, C., & Tucker, C. S. (2025) · IEEE Workshop on Design Automation for CPS and IoT (DESTION) · doi:10.1145/3722573.3727826 · Cited by 1
SGEU: enhancing LLM reasoning via backward exemplar generation and verification
Wang, Z., Zhou, X., Yang, Y., Ma, B., Wang, L., & Dong, R. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06529-8 · Cited by 1
Ha, R., Li, C., Pu, R., Zhang, L., Zhang, X., & Su, S. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.532 · Cited by 1
Tree-of-Quote Prompting Improves Factuality and Attribution in Multi-Hop and Medical Reasoning
Xu, J., Li, Y., Zhang, Z., Luk, A., Jobanputra, M., Oza, S., Murray, A., Kasula, M. R., Parker, A., & Eyre, D. W. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.285 · Cited by 1
Problem decomposition guided by reasoning utility for complex reasoning in LLMs
Guo, Y., Tan, H., Li, R., Li, X., Sun, X., Qiang, P., & Hu, Z. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104509 · Cited by 1
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1
Li, W., Wang, J., Yu, L.-C., & Zhang, X. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3677587 · Cited by 1
EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents
Li, L., & Lu, Z. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792995 · Cited by 1
Robustness of Prompting: Enhancing Robustness of Large Language Models Against Prompt Attacks
Mu, L., Chu, G., Li, N., Sang, L., & Zhang, Y. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3697684 · Cited by 2
Feynman Explanation Prompting: A Pedagogically Grounded Framework for Large Language Models
Bildirici, F., Karakuş, M., & Kızıltepe, R. S. (2026) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/ichora69329.2026.11537090 · Cited by 1
iBRAHIMOĞLU, N., Demirezen, M. U., & Yıldız, F. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01490-x · Cited by 1
Fact-CoT: Explicit Fact-Enhanced Chain-of-Thought Prompting for Commonsense Reasoning
Dai, L., & Qin, K. (2023) · International Conference on Computer Science and Automation Technology (CSAT) · doi:10.1109/csat61646.2023.00092 · Cited by 1
ToFC: Tree-of-Fact with Continued Best-First Search for Commonsense Reasoning
Dai, L., & Qin, K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649921 · Cited by 1
Integrating Classical Planners with GPT-Based Planning Policies
Tummolo, M., Rossetti, N., Gerevini, A., Olivato, M., Putelli, L., & Serina, I. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-80607-0_24 · Cited by 1
Advances in reasoning by prompting large language models: A survey
Hong, R., Pang, X., & Zhang, C. (2026) · Cybernetics and Intelligence · doi:10.26599/cai.2024.9390004 · Cited by 3
TRIPLEx: Triple Extraction for Explanation
Setzu, M., Monreale, A., & Minervini, P. (2021) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi52975.2021.00015 · Cited by 1
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Sugawara, S., & Tsugita, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.861 · Cited by 1
Identifying Physical Object Use in Sentences
Jiang, T., & Riloff, E. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.781 · Cited by 1
When Do Decompositions Help for Machine Reading?
Wei, K., Lawrie, D., Durme, B. V., Chen, Y., & Weller, O. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.219 · Cited by 2
Hint of Pseudo Code (HoPC): Zero-Shot Step by Step Pseudo Code Reasoning Prompting
Lei, I. T., Zhu, Z., Yu, H., Yao, Y., & Deng, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4088-4_36 · Cited by 1
Evaluating and Improving the Reasoning Abilities of Language Models
Helwe, C. (2024) · theses.fr (ABES) · doi:10.70675/7fc70e59zdd27z4b0fza38dz03a2c67cda50 · Cited by 1
GIER: Gap-Driven Self-Refinement for Large Language Models
Dewri, R. (2025) · arXiv · doi:10.48550/arxiv.2509.00325
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs
Cui, H., Li, S., Zhang, Y., & Shi, Q. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.629
Knowledge Corpus Error in Question Answering
Lee, Y., Oh, P., & Thorne, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.616
Freehal: A Framework for Editing Large Language Model HallucinationsPreprint
Lim, J., Shin, J., Lee, Y., & Song, H.-J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4893610
Miandoab, K. E., & Sarathy, V. (2024) · Qeios · doi:10.32388/8un60r
Allein, L., Truşcǎ, M. M., & Moens, M. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104234 · Cited by 1
The Benefits in Shallow: Merge Decoding Across Large Language Model Layers
Zhou, Y., Zhou, C., Xie, W., Wang, X., Chen, J., Ni, Z., & Li, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_30
Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language ModelsPreprint
Gao, K., Xia, S., Xu, K., Torr, P. H. S., & Gu, J. (2024) · Qeios · doi:10.32388/758n37
Multi-Document Question Answering with Lightweight Embeddings-based Document Reranker
Aitymbetov, N., Lee, M.-H., & Tu, N. A. (2024) · International Conference on Advanced Technologies for Communications · doi:10.1109/atc63255.2024.10908118
Zhang, Y., Zhang, B., Li, Z., Li, M., Cheng, N., Chen, M., Wei, T., Ma, J., Wang, S., & Xiao, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887840 · Cited by 1
Liu, J., & Liu, C. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4298-4_16
Handling Causal Tasks by Large Language Models: From Discovery to ReasoningPreprint
Xu, Y., Chen, H., Wang, Y., Wu, H., Wang, S., & Liu, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5222342
Evaluating Logical Reasoning Ability of Large Language Models
Chan, E., & Chan, A. C.-F. (2026) · doi:10.1109/icnlp69856.2026.11527900 · Cited by 1
InVis: Interactive Neural Visualization System for Human-Centered Data InterpretationPreprint
Xie, M., & Chen, S. (2025) · Research Square · doi:10.21203/rs.3.rs-6565387/v1
InVis: Interactive Neural Visualization System for Human-Centered Data InterpretationPreprint
Xie, M., & Chen, S. (2025) · doi:10.36227/techrxiv.174667701.10499039/v1
Mitigating Hallucinations in LLMs through Context-Aware Decoding and Layer Comparison
Yu, S., Kim, G., & Kang, S. (2025) · Journal of Digital Contents Society · doi:10.9728/dcs.2025.26.4.995
Fan, Z., Zhang, B., He, M. X., Teng, Z., & Jianping Fan, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5269808
Leveraging Correctness-Based Learning for Efficient Reasoning via Automated Process Supervision
Yao, Y., Liu, M., Deng, G., Liu, Z., Wu, D. O., Guo, Z., Wu, H., & Song, L. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3581101
A Meta-Analysis of Cloud-Based Llms in Implicit Multi-Step Reasoning
You, J., Gao, Y., Xu, Z., & Zhao, L. (2025) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit65432.2025.11035021
InVis: Interactive Neural Visualization System for Human-Centered Data Interpretation
Xie, M., & Chen, S. (2025) · International Conference on Communications, Information System and Computer Engineering (CISCE) · doi:10.1109/cisce65916.2025.11065878
DO-FacT: Discrete-Optimized Fact-Tree Prompting Framework for Commonsense Reasoning
Wen, T., Zhao, D., Dai, L., & Shao, S. (2025) · International Conference on Software Engineering and Information Management (ICSIM) · doi:10.1145/3725899.3727894
Zhu, R., Wang, B., Zhao, D., Liu, J., He, R., & Hou, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730170
Singh, A., Srivastav, J., Banerjee, J., Jindal, S., & Mohapatra, A. K. (2025) · International Conference on Networks and Cryptology (NETCRYPT) · doi:10.1109/netcrypt65877.2025.11102417
Inspire Me with Your Questions: Repurposing Historical Questions for New Documents
Liu, Y., Cao, Y., & Luo, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02088-8_6
Step-Wise Selection from Multiple Reasoning Chains to Enhance LLMs for Multi-hop QA
Xu, T., Murayama, Y., & Izumi, K. (2025) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.40-5_a-p21
Keyan, J. I. N., Zhong, J., & Xu, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5624245
Query, Decompose, Compress: Structured Query Expansion for Efficient Multi-Hop Retrieval
Yun, J., & Kim, Y. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3760968 · Cited by 1
Yan, J., Liu, L., Pan, Y., Chen, S., Xiang, Y., & Tang, B. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.413 · Cited by 5
Confidence-guided Refinement Reasoning for Zero-shot Question Answering
Jang, Y., Choi, W. S., Jung, M., Lee, M., & Zhang, B. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.354 · Cited by 1
Zhao, L., Zhou, J., Cao, J., & Zhu, W. (2025) · PLoS ONE · doi:10.1371/journal.pone.0333508
Query Generation Pipeline with Enhanced Answerability Assessment for Financial Information Retrieval
Kim, H., Yoo, Y., & Kwak, Y. (2025) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3768292.3770354
Wang, P., Zeng, X., Zhu, C., Xu, B., Mao, Z., & Zhang, Y. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228172
Enhancing Prompting with Deep Understanding and Extended Reasoning for Solving Mathematical Problems
Yang, Z., Huo, R., Yang, J., Wang, L., & Zhou, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4367-0_23
Exploring and Evaluating the Reasoning Steps in Multi-hop Machine Reading Comprehension
XANH, H. T., テ, ホ. サ., & XANH, H. T. (2024) · SOKENDAI (Graduate University for Advanced Studies)
FoVer: First-Order Logic Verification for Natural Language Reasoning
Pei, Y., Du, Y., & Jin, X. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.41
Liu, Z., Geng, Q., Liu, Y., Yang, C., & Pan, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5761-5_22
Zhao, W. X., Zhou, K., Li, J., Tang, T., & Wen, J.-R. (2025) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-96-6259-3_12
Extract-Related-Generate: A Question Decomposed Retrieval-Augmented Generation
Xie, Y., Yang, M., & He, N. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-6946-5_42
YAN, S., Wang, W., Chen, H., & Wang, J. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08333-8
Hendriks, D., Spitzer, P., Kühl, N., & Satzger, G. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3671872 · Cited by 3
Lu, G., Zhan, W., Peng, L., & Li, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00615-8
Towards Self-Evolving Agents: A Dual-Process Framework for Continual Context Refinement
Teng, L., Ni, W., Song, L., Shi, J. F., & Li, Y. (2026) · Electronics · doi:10.3390/electronics15061232
Accelerating Personalization Signal Learning via Synthetic Data
Parveen, D., Kang, D., Paruchuri, A., Kayal, D., & Mallapragada, P. K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_17
ART: Adaptive Reasoning Trees for Explainable Claim Verification
Wadhwa, S., Kumar, H., Yang, G., Nishar, A. A. M., Mohanty, P., Shinde, S., & Wu, Y. (2026) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2026.findings-eacl.28
Lightweight Dual-Process Reasoning for Edge and Data-Driven AI Systems
Chen, N., Mao, J., Li, T., & Zhang, X. (2025) · International Conference on Data Science and Edge Computing (ICDSEC) · doi:10.1109/icdsec67721.2025.11439183
PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs
Margapuri, V., Kazanjian, G., & Kosaraju, N. K. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00016
Employing Transformers and Humans for Textual-Claim Verification
Saeed, M. (2022) · theses.fr (ABES) · doi:10.70675/3e02a3a6z8281z4d84z8f38ze617eaaf969b
A Scalable Two-Phase Method for Low-Overhead Retrieval-Augmented Language Models
Hayawi, K., & Shahriar, S. (2025) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet63943.2025.11472215
ReLoX: Enhancing Reasoning in Small Language Models with Task-Specific LoRA Experts
Zhu, Z. (2026) · International Symposium on Computer Applications and Information Systems (ISCAIS) · doi:10.1109/iscait69154.2026.11477224
The Example Saturation Effect: The Hidden Role of Input Difficulty in In-Context Learning
Fan, X., Peng, B., Sun, H., Guo, J., Yang, C., & Zhang, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463529
Hierarchical Voting Decoder for Resolving Knowledge Conflicts
An, L., Chen, S., Li, Z., Wu, K., Yang, S., Pei, X., Wang, J., & Liu, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461241
LLM-Adaptive Reasoning Clustering for Automated Chain-of-Thought Prompting
He, Q., Yan, K., Qian, S., & Wang, C. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462308
Joint Modeling of Typicality and Uncertainty for Sot-Based Few-Shot LLM Reasoning
Wang, B., Wei, P., Ren, Z., Li, S., Zhang, J., & Chen, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464177
Consensus-Awarded Multi-Agent Debate Via Adversarial Interaction
Zheng, H., Zhang, Y., Shah, R. R., Guo, B., & Yu, Z. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462230
Metadata
- Authors
- Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J.
- Affiliations
- Tel Aviv University (IL)
- Allen Institute for Artificial Intelligence (US)
- University of Pennsylvania (US)
- Corresponding author
- Geva, Mor
- Khashabi, Daniel
- Segal, Elad
- Khot, Tushar
- Roth, Dan
- Berant, Jonathan
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2021
- Volume
- 9
- Pages
- 346-361
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3159959439
- Cited by
- 242
- References
- 32
- Citations by year
- 2026: 77 · 2025: 71 · 2024: 38 · 2023: 42 · 2022: 6 · 2021: 8
- FWCI
- 8.27
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- question answering benchmark, multi-hop reasoning, StrategyQA, data collection, human performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
