The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Al-Jawaab at Qur’an QA 2023 Shared Task: Exploring Embeddings and GPT Models for Passage Retrieval and Reading Comprehension
Open accessAbstract
This paper introduces a comprehensive system designed to address two natural language processing tasks: Passage Retrieval (Task A) and Reading Comprehension (Task B), applied to datasets related to the Holy Qur'an.Task A was treated as a measurement of a textual similarity problem where the system leverages OpenAI's "text-embedding-ada-002" embedding model to transform textual content into numerical representations, with cosine similarity serving as the proximity metric.Task B focuses on the extraction of answers from Qur'anic passages, employing the Generative Pre-trained Transformer-4 (GPT-4) language model.In Task A, the system is evaluated using the Mean Average Precision (MAP) metric, achieving MAP scores of 0.109438 and 0.06426543057 on the development and test datasets with an optimal similarity threshold set at 0.85.Task B evaluation employs partial Average Precision (pAP), where our system surpasses a baseline whole-passage retriever with pAP scores of 0.470 and 0.5393130538 on the development and test datasets, respectively.Holy Qur'an, passage retrieval, reading comprehensive, embeddings
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Research works printed but not linked: 12
- Giambattista Amati. 2009. BM25, pages 257-260. · DOI · Paper page
- Kenneth Ward Church. 2017. Word2vec. Natural Lan- guage Engineering, 23(1):155-162.
- Alexis Conneau and Douwe Kiela. 2018. Senteval: An evaluation toolkit for universal sentence representa- tions. arXiv preprint arXiv:1803.05449. · arXiv
- Rana Malhas and Tamer Elsayed. 2020. AyaTEC: Build- ing a Reusable Verse-based Test Collection for Ara- bic Question Answering on the Holy Qur'an. ACM Transactions on Asian and Low-Resource Language Information Processing, 19(6):1-21.
- Rana Malhas and Tamer Elsayed. 2022. Arabic ma- chine reading comprehension on the holy qur'an us- ing cl-arabert. Information Processing Management, 59(6):103068.
- Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2022. Qur'an QA 2022: Overview of the First Shared Task on Question Answering over the Holy Qur'an. In Proceedings of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5) at the 13th Language Resources and Evaluation Con- ference (LREC 2022), pages 79-87.
- Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2023. Qur'an QA 2023 Shared Task: Overview of Passage Retrieval and Reading Comprehension Tasks over the Holy Qur'an. In Proceedings of the First Arabic Natural Language Processing Confer- ence (ArabicNLP 2023), Singapore.
- Rana R Malhas. 2023. ARABIC QUESTION ANSWER- ING ON THE HOLY QUR'AN. Ph.D. thesis. OpenAI. 2023a. Embeddings -openai api. September 12, 2023.
- Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word rep- resentation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
- Faisal Rahutomo, Teruaki Kitasuka, and Masayoshi Ar- itsugi. 2012. Semantic cosine similarity. In The 7th international student conference on advanced science and technology ICAST, volume 4, page 1.
- Mihai Surdeanu, Massimiliano Ciaramita, and Hugo Zaragoza. 2011. Learning to rank answers to non- factoid questions from web collections. Computa- tional linguistics, 37(2):351-383.
- Ellen M Voorhees. 2001. Evaluation by highly relevant documents. In Proceedings of the 24th annual in- ternational ACM SIGIR conference on Research and development in information retrieval, pages 74-82.
Cited by
Sorted by citations
Burhan at IslamicEval: Fact-Augmented and LLM-Driven Retrieval for Islamic QA
Basheer, M., Mansour, W., Touma, A., & Qadeib Alban, A. (2025) · Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks · doi:10.18653/v1/2025.arabicnlp-sharedtasks.73
“…Passage Retrieval (i.e., the task of identifying and ranking candidate passages that potentially contain answers to a given question), have become a matter of paramount importance (Malhas et al., 2022) and presents unique and significant challenges (Malhas et al., 2022; Zekiye and Amroush, 2023).”
Metadata
- Authors
- Zekiye, A., & Amroush, F.
- ORCID
- Zekiye, Abdulrezzak
- Venue
- Proceedings of ArabicNLP 2023
- Published
- 2023
- Pages
- 743-747
- OpenAlex
- W4389518324
- Cited by
- 4
- References
- 2
- Citations by year
- 2026: 2 · 2024: 2
- FWCI
- 0.71
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- dx.doi.org
- Licence
- cc-by
- Locations
- Arabic Natural Language Processing Conference (ArabicNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- Quran, passage retrieval, reading comprehension, answer extraction, embeddings, cosine similarity, GPT-4, text similarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
