CoChat
Sign Up
Cited by 1Topic ModelingSemantic Web and OntologiesNatural Language Processing Techniques

The DOI matches its Crossref record.

SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

Open access
Ousidhoum, Nedjma, Myung, Junho, Perez Almendros, Carla, Jin, Jiho, Keleg, Amr, Beloucif, Meriem, Zhou, Yi, Agerri, Rodrigo, Araujo, Vladimir, Baes, Naomi, Barry, James, Boisson, Joanne, Chen, Nancy, De Kock, Christine·2026·Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
DOI 10.18653/v1/2026.semeval-1.455
Open this paper in CoChat — free accountOpen PDF

Abstract

Nedjma Ousidhoum, Junho Myung, Carla Perez Almendros, Jiho Jin, Amr Keleg, Meriem Beloucif, Yi Zhou, Rodrigo Agerri, Vladimir Araujo, Naomi Baes, James Barry, Joanne Boisson, Nancy Chen, Christine De Kock. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). 2026.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2026.semeval-1.455

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • BLEnD Track 2 comprises N=47,014 English MCQs across 30 country-language pairs with approximately uniform gold labels (A=24%, B=26%, C=26%, D=24%).

    “BLEnD Track 2 comprises N = 47 , 014 English MCQs (four options each) across 30 countrylanguage pairs with approximately uniform gold labels (A = 24%, B = 26%, C = 26%, D = 24%).”

    From CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering (2026) · cites this paper as Ousidhoum et al., 2026

    The Track B: Multiple-Choice Questions section (page unavailable) confirms that questions are in English and have four answer options. Our copy does not substantiate the claimed total of 47,014 MCQs, 30 country-language pairs, or the A/B/C/D gold-label percentages; searches for the total and label distribution found no matching passage. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 36

  1. Faisal Muhammad Adam, Lukman Jibril Aliyu, Sani Aji, Abdulhamid Abubakar, and Aliyu Rabiu Shuaibu. 2026. HausaNLP at SemEval-2026 Task 7: Prompt-based Hausa Cultural Question Answering. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  2. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, Al- ham Fikri Aji, Jacki O'Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling "culture" in LLMs: A survey. In Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15763-15784, Miami, Florida, USA. Association for Computational Linguistics. · DOI
  3. Isaac Nyadu Adjei and Saurav K. Aryal. 2026. Howard University-AI4PC at SemEval-2026 Task 7: Cultur- ally Aware Multilingual Model Routing Through a Mixture-of-Specialists Framework. In Proceedings of the 20th International Workshop on Semantic Eval- uation (SemEval-2026).
  4. Yusser Al Ghussin, Daniil Gurgurov, Yasser Hamidul- lah, Josef van Genabith, Cristina España-Bonet, and Simon Ostermann. 2026. DFKI-MLT at SemEval- 2026 Task 7: Steering Multilingual Models To- wards Cultural Knowledge. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  5. Mai Alkhamissi, Yunze Xiao, Badr AlKhamissi, and Mona T. Diab. 2026. Hire your anthropologist! re- thinking culture benchmarks through an anthropolog- ical lens. In Findings of the Association for Compu- tational Linguistics: EACL 2026, pages 1218-1235. Association for Computational Linguistics.
  6. Danileth Almanza, Jairo Serrano, Edwin Puertas, and Juan Carlos Martinez Santos. 2026. VerbaNexAI at SemEval-2026 Task 7: Integrating Web Snippets and RAG for the Evaluation of Multilingual Cul- tural Knowledge in LLMs. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  7. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Sheng- guang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingx- uan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609. · arXiv
  8. Liliia Bogdanova, Shiran Sun, Lifeng Han, Natalia Amat-Lefort, and Flor Miriam Plaza-del Arco. 2026. FLANS at SemEval-2026 Task 7: RAG with Open- Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures. In Proceed- ings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  9. Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388. · arXiv
  10. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. Massively multi-cultural knowledge acquisition & lm benchmarking. arXiv preprint arXiv:2402.09369. · arXiv
  11. Jianning Gao, Xianling Mao, Shumin Shi, Duanzhi Zhaxi, Yingbo Sun, Xiandeng Li, and Binyang Li. 2026. uir-cis-7 at SemEval-2026 Task 7: Zero- Shot Chain-of-Thought Reasoning for Cross-Cultural Daily Knowledge. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  12. Reihaneh Iranmanesh, Ophir Frieder, and Nazli Gohar- ian. 2026. GUIR at SemEval-2026 Task 7: Prob- ing Cultural Knowledge in LLMs via Multi-Agent Debate. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  13. Meizhi Jin, Zhichao Meng, Junqi Yin, Lianxin Jiang, and Jianyu Li. 2026. king001 at SemEval-2026 Task 7: Cross-Language Cultural Everyday Knowledge Q&A System Based on RAG. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  14. Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki, and Timothy Baldwin. 2024. Eval- uating gender bias in large language models via chain-of-thought prompting. arXiv preprint arXiv:2401.15585. · arXiv
  15. Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024. Click: A bench- mark dataset of cultural and linguistic intelligence in korean. In Proceedings of the 2024 Joint In- ternational Conference on Computational Linguis- tics, Language Resources and Evaluation (LREC- COLING 2024), pages 3335-3346.
  16. Fajri Koto, Rahmad Mahendra, Nurul Aisyah, and Timothy Baldwin. 2024. Indoculture: Exploring geographically-influenced cultural commonsense rea- soning across eleven indonesian provinces. arXiv preprint arXiv:2404.01854. · arXiv
  17. Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2025. Culturally aware and adapted nlp: A taxon- omy and a survey of the state of the art. Preprint, arXiv:2406.03930. · arXiv
  18. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez García, Hwaran Lee, Shamsuddeen Has- san Muhammad, Kiwoong Park, Anar Sabuhi Rza- yev, Nina White, Seid Muhie Yimam, Moham- mad Taher Pilehvar, Nedjma Ousidhoum, Jose Camacho-Collados, and Alice Oh. 2024. Blend: A benchmark for llms on everyday knowledge in di- verse cultures and languages. In Advances in Neural Information Processing Systems, volume 37, pages 78104-78146.
  19. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2024. Having beer after prayer? measuring cultural bias in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 16366-16393, Bangkok, Thailand. Association for Computational Linguistics. · DOI · Paper page
  20. Roberto Navigli, Simone Conia, and Björn Ross. 2023. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Informa- tion Quality, 15(2):1-21.
  21. Jingke Ning. 2026. LocuPrompt at SemEval-2026 Task 7: A Multilingual Prompting Framework for Cross- Cultural Everyday Knowledge in LLMs. In Proceed- ings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  22. Nedjma Ousidhoum, Meriem Beloucif, and Saif M. Mo- hammad. 2025. Building better: Avoiding pitfalls in developing language resources when data is scarce. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8881-8894, Vienna, Austria. Association for Computational Linguistics. · DOI
  23. Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. 2024. Survey of cultural awareness in language models: Text and beyond. Preprint, arXiv:2411.00860. · arXiv
  24. Mohammad Marufur Rahman, Rakshitha Rao Ail- neni, and Sanda Harabagiu. 2026. UTD-HLTRI at SemEval-2026 Task 7: Bridging Cultural Knowl- edge Gaps in LLMs via Web-Augmented Context. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  25. Aditya Singh and Rickarya Das. 2026. CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Re- trieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  26. Guijin Son, Hanwool Lee, Suwan Kim, Huiseo Kim, Jae cheol Lee, Je Won Yeom, Jihyu Jung, Jung woo Kim, and Songseong Kim. 2024. Hae-rae bench: Evaluation of korean knowledge in language mod- els. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7993-8007.
  27. Jiwoo Song, Sihyeong Yeom, and Harksoo Kim. 2026. K-NLPers at SemEval-2026 Task 7: Multiple LLM Agent Debate System for Everyday Knowledge Across Diverse Languages and Cultures. In Proceed- ings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  28. Swetha Krishna Sriram and Nirupama Sekar. 2026. Models Without Borders at SemEval-2026 Task 7: Bridging Cultural Contexts with Search-Grounded QA. In Proceedings of the 20th International Work- shop on Semantic Evaluation (SemEval-2026).
  29. Cheng Tang, Zhichao Meng, and Meizhi Jin. 2026. chengtang at SemEval-2026 Task 7: A Retrieval- Augmented Generation Framework for Cultural Per- spective Alignment in Everyday MCQs. In Proceed- ings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  30. Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, and Jafar Razmara. 2026. Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low- Resource Cultural Reasoning in Multilingual Ques- tion Answering. In Proceedings of the 20th Interna- tional Workshop on Semantic Evaluation (SemEval- 2026).
  31. Kongqiang Wang, Peng Zhang, and Qingli Tan. 2026. Wangkongqiang at SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  32. Yen Yee Yam and Hong Meng Yam. 2026. Pinetree at SemEval-2026 Task 7: A Large-Scale Failure Anal- ysis of Cultural Grounding in Language Models. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  33. Xiao Yao and Liang Yang. 2026. Agentic at SemEval- 2026 Task 7: A Qwen-based System for Accurate Retrieval of Everyday Knowledge Across Diverse Languages and Cultures. In Proceedings of the 20th International Workshop on Semantic Evalua- tion (SemEval-2026).
  34. Naitian Zhou, David Bamman, and Isaac L Bleaman. 2025. Culture is not trivia: Sociocultural theory for cultural nlp. In Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25869-25886.
  35. Yi Zhou, Danushka Bollegala, and Jose Camacho- Collados. 2024. Evaluating short-term temporal fluc- tuations of social biases in social media data and masked language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19693-19708, Miami, Florida, USA. Association for Computational Lin- guistics. · DOI
  36. Yi Zhou, Jose Camacho-Collados, and Danushka Bol- legala. 2023. A predictive factor analysis of social biases and task-performance in pretrained masked language models. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 11082-11100, Singapore. Associ- ation for Computational Linguistics. · DOI

Cited by

Sorted by citations

Metadata

Authors
Ousidhoum, N., Myung, J., Perez Almendros, C., Jin, J., Keleg, A., Beloucif, M., Zhou, Y., Agerri, R., Araujo, V., Baes, N., Barry, J., Boisson, J., Chen, N., & De Kock, C.
Affiliations
  • Cardiff University (GB)
  • Korea Advanced Institute of Science and Technology (KR)
  • Mohamed bin Zayed University of Artificial Intelligence (AE)
  • Uppsala University (SE)
  • University of the Basque Country (ES)
  • The University of Melbourne (AU)
  • IBM (United States) (US)
  • Agency for Science, Technology and Research (SG)
ORCID
Venue
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Publisher
Association for Computational Linguistics
Published
2026
Pages
3823-3837
DOI
10.18653/v1/2026.semeval-1.455
OpenAlex
W7166839344
Cited by
0
References
0
FWCI
0.00
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
International Workshop on Semantic Evaluation (SemEval) (publishedVersion, cc-by)
Funders
  • UK Research and Innovation
  • Ministry of Science and ICT, South Korea
  • European Regional Development Fund
  • Institute for Information and Communications Technology Promotion
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
everyday knowledge, intercultural knowledge, semantic evaluation, knowledge representation, multilingual knowledge base

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review