Item analysis of multiple choice questions: A quality assurance test for an assessment tool
Open accessAbstract
Background The item analysis of multiple choice questions (MCQs) is an essential tool that can provide input on validity and reliability of items. It helps to identify items which can be revised or discarded, thus building a quality MCQ bank.Methods The study focussed on item analysis of 90 MCQs of three tests conducted for 150 first year Bachelor of Medicine and Bachelor of Surgery (MBBS) physiology students. The item analysis explored the difficulty index (DIF I) and discrimination index (DI) with distractor effectiveness (DE). Statistical analysis was performed by using MS Excel 2010 and SPSS, version 20.0.Results Of total 90 MCQs, the majority, that is, 74 (82%) MCQs had a good/acceptable level of difficulty with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult and nine (10%) were too easy. A total of 72 (80%) items had an excellent to acceptable DI and 18 (20%) had a poor DI with an overall mean DI of 0.31 ± 0.12. There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001). The mean DE was 32.35 ± 31.3 with 73% functional distractors in all. The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good. The standard error of measurement was 1.22.Conclusion Our study helped teachers identify good and ideal MCQs which can be part of the question bank for future and those MCQs which needed revision. We recommend that item analysis must be performed for all MCQ-based assessments to determine validity and reliability of the assessment.
AI snapshot
Written from the paper’s full text.
TL;DR
This study performed item analysis on 90 multiple choice questions from three physiology assessments for 150 first-year MBBS students to evaluate difficulty, discrimination, distractor effectiveness, and reliability. Most items had acceptable difficulty and discrimination, but only 20% met all criteria for an ideal MCQ.
Key findings
- Of 90 MCQs, 74 (82%) had good/acceptable difficulty (DIF I 30–70%; mean 55.32 ± 7.4), 7 (8%) were too difficult, and 9 (10%) were too easy; 34 (38%) were excellent/ideal (DIF I 50–60%).
- For discrimination, 72 (80%) items had acceptable-to-excellent DI (>0.20) and 18 (20%) had poor DI (≤0.20); 61 (68%) had good-to-excellent DI (≥0.25), with overall mean DI 0.31 ± 0.12. DIF I and DI showed a weak correlation (r = 0.140, p < .0001).
- Of 270 distractors, 198 (73%) were functional and 72 (27%) were non-functional; DE was poor in 2 (2%), moderate in 8 (9%), good in 50 (56%), and excellent in 30 (33%) items. Mean DE was 32.35 ± 31.3.
- Reliability was Cronbach alpha 0.85, KR-20 0.71, and SEM 1.22; 18 (20%) MCQs fulfilled all three criteria for an ideal MCQ (DIF I 30–70%, DI ≥0.25, 100% DE).
Methods
- Item analysis of 90 single-best-response MCQs from three internal assessments for 150 first-year MBBS physiology students; each MCQ had a stem and four options with one correct response and three distractors, scored 1 for correct and 0 for wrong, with no negative marking.
- The upper and lower 27% (41 students each) were classified as high and low achievers; DIF I was calculated as (H + L) × 100/N, DI as 2 × (H − L)/N, and DE was based on non-functional distractors selected by <5% of students.
- Internal consistency was assessed with Cronbach alpha, KR-20, and SEM; skewness and kurtosis were also measured; analysis used MS Excel 2010 and SPSS version 20.0.
Limitations
- Not stated in the paper; the text does not include an explicit limitations section.
- Evident from the text: the study was conducted in a single institution and single discipline (physiology) with first-year MBBS students across three internal assessments, so generalizability is limited.
- Evident from the text: the analysis is descriptive and does not report follow-up after revising or discarding flawed items, nor does it evaluate whether item revisions improved future assessments.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 4 cited sources checked · 0 supported · 0 flagged
- Findingown data§ Results
11 (12%) items had an acceptable DI (0.21-0.24).
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Quote in the paperNot a substantial claim - Findingown data§ Results
29 (32%) items showed an excellent DI (≥0.35).
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Quote in the paperNot a substantial claim - Findingown data§ Results
Seven (8%) MCQs were too difficult (DIF I <30%).
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Quote in the paperNot a substantial claim - Findingown data§ Results
Of the 90 MCQs, 74 (82%) had a good/acceptable level of difficulty (DIF I 30-70%).
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Quote in the paperSubstantial claim - Findingown data§ Results
Among all MCQs, 34 (38%) had excellent/ideal levels of difficulty (DIF I 50-60%).
“Among all MCQs, 34 (38%) MCQs had excellent/ideal levels of difficulty (DIF I 50–60%) (Table 2).”
Quote in the paperNot a substantial claim
- Findingown data§ Results
32 (36%) items showed a good DI (0.25-0.34).
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Quote in the paperNot a substantial claim - Findingown data§ Results
The overall mean DIF I was 54.95 ± 13.4.
“Item analysis parameters Difficulty index Discrimination index Distractor effectiveness Mean ± SD 54.95 ± 13.4 0.31 ± 0.12 32.35 ± 31.3”
Quote in the paperNot a substantial claim - Findingown data§ Results
Two (2%) items had poor DE (0%).
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
Quote in the paperNot a substantial claim - Findingown data§ Results
There was a significant weak correlation between DIF I and DI (r = 0.140, p < .0001).
“There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001).”
Quote in the paperNot a substantial claim - Findingown data§ Discussion
The mean DIF I for the 74 good/acceptable MCQs was 55.32 ± 7.4.
“In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30–70%) with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Quote in the paperSubstantial claim - Findingown data§ Results
50 (56%) items had good DE (66.6%).
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
Quote in the paperNot a substantial claim - Findingown data§ Results
The mean DE was 32.35 ± 31.3.
“The mean DE was 32.35 ± 31.3 with 73% functional distractors in all.”
Quote in the paperNot a substantial claim - Findingown data§ Results
Nine (10%) MCQs were too easy (DIF I >70%).
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Quote in the paperNot a substantial claim - Findingown data§ Results
Of the 90 MCQs, 18 (20%) had a poor discrimination index (DI ≤ 0.20).
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Quote in the paperNot a substantial claim - Findingown data§ Results
The overall mean DI was 0.31 ± 0.12.
“A total of 72 (80%) items had an excellent to acceptable DI and 18 (20%) had a poor DI with an overall mean DI of 0.31 ± 0.12.”
Quote in the paperNot a substantial claim - Findingown data§ Results
The reliability measure of test items by Kuder-Richardson Formula 20 was 0.71.
“The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good.”
Quote in the paperSubstantial claim - Findingown data§ Results
30 (33%) items had excellent DE (100%).
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
Quote in the paperNot a substantial claim - Findingown data§ Results
18 (20%) MCQs fulfilled all three criteria for an ideal MCQ (good/acceptable DIF I 30-70%, high DI ≥0.25, and 100% DE).
“Our study showed that 18 (20%) MCQs fulfilled all the three criterias (a good/acceptable level of the DIF I [30–70%] with a high DI [≥0.25] and 100% DE.) which should be present for an ideal MCQ.”
Quote in the paperSubstantial claim - Findingown data§ Results
The skewness and kurtosis values were 0.59 and 0.69, respectively.
“The skewness and kurtosis values were 0.59 and 0.69, respectively.”
Quote in the paperNot a substantial claim - Findingown data§ Results
The reliability measure of test items by Cronbach alpha was 0.85.
“The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good.”
Quote in the paperSubstantial claim - Findingown data§ Results
The standard error of measurement was 1.22.
“The standard error of measurement was 1.22.”
Quote in the paperNot a substantial claim - Findingown data§ Results
Of the total 270 distractors for 90 items, 198 (73%) were functional distractors and 72 (27%) were non-functional distractors.
“Of the total 270 distractors for 90 items, 198 (73%) were functional distractors (FDs) and 72 (27%) were NFDs.”
Quote in the paperNot a substantial claim - Findingown data§ Results
Eight (9%) items had moderate DE (33.3%).
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
Quote in the paperNot a substantial claim - Premisecitation§ Introduction
The item analysis of an assessment tool provides input about validity and reliability of the item.
“The item analysis of an assessment tool provides input about validity and reliability of the item.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] Kiat, 2018, The influence of distractor strength and response order on MCQ responding, Educ Psychol source
Not checked in the free run.
- Interpretationown data§ Discussion
The weak correlation between DIF I and DI indicates that the relationship between DIF I and DI of 90 MCQ items is dome shaped rather than linear.
“The weak correlation between the DIF I and DI also indicates that relationship between the DIF I and DI of 90 MCQ items is dome shaped rather than linear.”
Quote in the paperSubstantial claim - Interpretationown data§ Discussion
The majority of MCQs were in the category of good MCQs.
“Our study showed mean and standard deviation for DIF I (good), DI (excellent) and DE (moderate) which suggested that the majority of MCQs were in the category of good MCQs.”
Quote in the paperSubstantial claim - Interpretationown data§ Discussion
The values of the DIF I and DI indicated that not too easy and too difficult questions were part of the examination.
“The values of the DIF I and DI indicated that not too easy and too difficult questions were part of the examination.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 2
- May combine two claims. 5
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown data§ DiscussionThe distribution of data is approximately normal.
“In this study, Skewness and Kurtosis values (0.59 and 0.69) represent the moderately skewed data with tail on the right side of the distribution and data are lighttailed (lack of outliers) i.e distribution of data is approximately normal.17”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] Mishra, 2019, Descriptive statistics and normality tests for statistical data, Ann Card Anaesth source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ IntroductionThe SEM is directly related to the reliability of the test.
“The SEM is directly related to the reliability of the test.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [11] McManus, 2012, The misinterpretation of the standard error of measurement in medical education: a primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement, Med Teach source
Not checked in the free run.
May combine two claims.
Interpretationown data§ DiscussionThe overall validity of the test was good with a Cronbach alpha of 0.85, KR 20 = 0.71 and SEM 1.22.
“In this study, the results showed that the overall validity of the test was good with a Cronbach alpha of 0.85, KR 20 = 0.71 and SEM 1.22.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown data§ DiscussionThe items were acceptably difficult and were good at differentiating students with higher and lower abilities.
“Therefore, items were acceptably difficult and were good at differentiating students with higher and lower abilities.”
Quote in the paperSubstantial claimMay combine two claims.
Premisecitation§ DiscussionMore non-functional distractors make an MCQ easy and more functioning distractors make it difficult.
“More NFD makes MCQ easy and vice versa with more functioning distractors making it difficult.23”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23] Tarrant, 2009, An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis, BMC Med Educ source
Not checked in the free run.
May combine two claims.
Interpretationown data§ DiscussionThe items analysed in this study were neither too easy nor too difficult.
“Items analysed in this study were neither too easy nor too difficult which is acceptable.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown data§ DiscussionThe weak correlation between DIF I and DI indicates that with increasing DIF I, the ability to discriminate between high and low scorers decreased and fewer easy questions were used in MCQ tests.
“The present study shows weak correlation between the DIF I and DI (r = 0.140, p < .0001), indicating that with increasing the DIF I, the ability to discriminate between the high and low scorers decreased and fewer easy questions were used in MCQ tests.”
Quote in the paperSubstantial claim
The authors state this as an assumption.
Any test score contains an error.
“Based on the assumption that any test score contains an error, SEM is used to estimate a band or interval within which a person's true score would fall, that is, the score if there were no error of measurement.”
The paper relies on this without support.
A non-functional distractor can reduce the discrimination power of an MCQ.
“A non-functional distractor can reduce the discrimination power of a MCQ.”
The paper relies on this without support.
The efficiency of MCQ assessment is solely based upon the quality of test MCQs.
“The efficiency of MCQ assessment is solely based upon the quality of test MCQs.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The DIS ranged from 1.0 to − 1.0.
“The DIS ranged from 1.0 to − 1.0.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
Item analysis provides feedback about item construction and its validity and reliability after it appears in an examination.
“Item analysis provides feedback about item construction and its validity and reliability after it appears in an examination [5, 35, 36].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
A positive DIS indicates that high achievers answered the item more correctly than lower achievers.
“A positive DIS indicates that high achievers answered the item more correctly than lower achievers and vice versa [35, 44].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
Such findings were reported in anatomy exam [48], community Medicine [49], and multidisciplinary summative exams [50, 51] that were constructed traditionally.
“Such findings were reported in anatomy exam [48], community Medicine [49], and multidisciplinary summative exams [50, 51] that were constructed traditionally.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Positive correlation is described as a decrease
MajorA positive correlation indicates that the two indices tend to increase together, whereas the discussion says discrimination decreased as the difficulty index increased.
“We found a weak correlation between the DIF I and DI (r = 0.140, p < .0001).”
“with increasing the DIF I, the ability to discriminate between the high and low scorers decreased”
Mean test score is presented as both a score and a percentage
MinorThe results give 47.35 as a mean score out of 90, while Table 1 labels the same value a percentage. A score of 47.35 out of 90 is not 47.35%.
“The test score of class ranged from 27 to 82 (total score 90) with a mean test score of 47.35 ± 11.6 (mean ± SD).”
“Percentage of mean test score ± SD 47.35 ± 11.6”
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Descriptive statistics and normality tests for statistical data
Gupta, A., Mishra, P., Pandey, C., Singh, U., Sahu, C., & Keshri, A. (2019) · Annals of Cardiac Anaesthesia · doi:10.4103/aca.aca_157_18 · Cited by 2,628 · cited 1× · see citing works
“e distribution of data is approximately normal.(17) In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30e70%) with a mean DIF I of 55.”
BackgroundThe theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953 · see citing works
Assessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872 · see citing works
Cain, M. K., Zhang, Z. J., & Yuan, K. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0814-1 · Cited by 1,172 · see citing works
The selection of upper and lower groups for the validation of test items.
Kelley, T. L. (1939) · Journal of Educational Psychology · doi:10.1037/h0057123 · Cited by 777 · see citing works
Improving the fairness of multiple-choice questions: a literature review
McCoubrie, P. (2004) · Medical Teacher · doi:10.1080/01421590400013495 · Cited by 318 · see citing works
The Approximate Sampling Distribution of Kuder-Richardson Reliability Coefficient Twenty
Feldt, L. S. (1965) · Psychometrika · doi:10.1007/bf02289499 · Cited by 296 · see citing works
Tarrant, M., Ware, J. S., & Mohammed, A. M. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-40 · Cited by 238 · see citing works
Item analysis of multiple choice questions: Assessing an assessment tool in medical students
Rao, C., Kishan Prasad, H., Sajitha, K., Permi, H., & Shetty, J. (2016) · International Journal of Educational and Psychological Researches · doi:10.4103/2395-2296.189670 · Cited by 96 · see citing works
Correlation between difficulty & discrimination indices of MCQs in formative exam in Physiology
Pande, S., Pande, S. R., Parate, V., Nikam, A. P., & Sh, A. (2013) · South-East Asian Journal of Medical Education · doi:10.4038/seajme.v7i1.149 · Cited by 61 · see citing works
Karelia, B. (2013) · International e-Journal of Science Medicine & Education · doi:10.56026/imu.7.2.41 · Cited by 47 · see citing works
EditorialCompetency-based assessment: making it a reality
Davis, M. H., & Harden, R. M. (2003) · Medical Teacher · doi:10.1080/0142159032000153842 · Cited by 45 · see citing works
McManus, I. C. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.670318 · Cited by 32 · see citing works
Item analysis as tool to validate multiple choice question bank in pharmacology
Date, A. P., Borkar, A. S., Badwaik, R. T., Siddiqui, R. A., Shende, T. R., & Dashputra, A. V. (2019) · International Journal of Basic & Clinical Pharmacology · doi:10.18203/2319-2003.ijbcp20194106 · Cited by 27 · see citing works
Itemized Analysis of Questions of Multiple Choice Question (MCQ) Exam
25, S. S., Patel, K., & Mahajan, N. (2012) · International Journal of Scientific Research · doi:10.15373/22778179/feb2013/94 · Cited by 21 · see citing works
Christian, D. S., Prajapati, A. C., Rana, B. M., & Dave, V. (2017) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20172004 · Cited by 20 · see citing works
Study on item and test analysis of multiple choice questions amongst undergraduate medical students
Ingale, A. S., Giri, P., & Doibale, M. K. (2017) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20171764 · Cited by 18 · see citing works
The influence of distractor strength and response order on MCQ responding
Kiat, J. E., Ong, A. R., & Ganesan, A. (2018) · Educational Psychology · doi:10.1080/01443410.2017.1349877 · Cited by 15 · see citing works
Student assessment: Moving over to programmatic assessment
Singh, T. (2016) · International Journal of Applied and Basic Medical Research · doi:10.4103/2229-516x.186955 · Cited by 14 · see citing works
Item Analysis of Multiple Choice Questions in Medical Licensing Examination
Gomboo, A., Gombo, B., Munkhgerel, T., Sumberzul, N., & Badamdorj, O. (2019) · Central Asian Journal of Medical Sciences · doi:10.24079/cajms.2019.06.009 · Cited by 9 · see citing works
Difficulty Index, Discrimination Index and Distractor Efficiency in Multiple Choice Questions
Mahjabeen, W., Alam, S., Hassan, U., Zafar, T., Butt, R., Konain, S., & Rizvi, M. (2018) · Annals of PIMS-Shaheed Zulfiqar Ali Bhutto Medical University · Cited by 7 · see citing works
Flagged first, then by citations
3 of the 25 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Patil, 2015, Item analysis of medicine multiple choice questions (MCQs) for undergraduate (3rd year MBBS) students, Res J Pharmaceut Biol Chem Sci, 6, 1242
Cited by
Sorted by citations
Cheung, B. H. H., Lau, G. K. K., Wong, G. T. C., Lee, E. Y. P., Kulkarni, D., Seow, C. S., Wong, R., & Co, M. T.-H. (2023) · PLOS ONE · doi:10.1371/journal.pone.0290691 · Cited by 195 · see citing works
Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05239-y · Cited by 128 · see citing works
Meo, S. A., Al-Masri, A. A., Alotaibi, M., Meo, M. Z. S., & Meo, M. O. S. (2023) · Healthcare · doi:10.3390/healthcare11142046 · Cited by 74 · see citing works
Agarwal, M., Sharma, P., & Goswami, A. (2023) · Cureus · doi:10.7759/cureus.40977 · Cited by 72 · see citing works
Rezigalla, A. A., Eleragi, A. M. S., Elhussein, A. B., Alfaifi, J. A., Alghamdi, M. A. A., Ameer, A., Yahia, A. I. O., Mohammed, O. A., & Adam, M. I. E. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05433-y · Cited by 52 · see citing works
Using artificial intelligence to create biology multiple choice questions for higher education
Nasution, N. E. A. (2023) · Agricultural and Environmental Education · doi:10.29333/agrenvedu/13071 · Cited by 38 · see citing works
AI in medical education: uses of AI in construction type A MCQs
Rezigalla, A. A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05250-3 · Cited by 36 · see citing works
Quality assurance and validity of AI-generated single best answer questions
Ahmed, A., Kerr, E., & O’Malley, A. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06881-w · Cited by 39 · see citing works
Item Analysis of Multiple-choice Questions (MCQs): Assessment Tool For Quality Assurance Measures
Elgadal, A., & Mariod, A. A. (2021) · Sudan Journal of Medical Sciences · doi:10.18502/sjms.v16i3.9695 · Cited by 23 · see citing works
Chauhan, A., Khaliq, F., & Nayak, K. R. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00471-z · Cited by 23 · see citing works
Agarwal, M., Sharma, P., & Wani, P. (2025) · Cureus · doi:10.7759/cureus.81871 · Cited by 18 · see citing works
Item Analysis: Concept and Application
Rezigalla, A. A. (2022) · IntechOpen eBooks · doi:10.5772/intechopen.100138 · Cited by 18 · see citing works
Lasala, N. J. (2023) · Recoletos Multidisciplinary Research Journal · doi:10.32871/rmrj2311.01.07 · Cited by 28 · see citing works
Rashwan, N. I., Aref, S. R., Nayel, O. A., & Rizk, M. H. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05153-3 · Cited by 22 · see citing works
Medical Education for the 21st Century
Firstenberg, M. S., & Stawicki, S. P. (2021) · IntechOpen eBooks · doi:10.5772/intechopen.95701 · Cited by 12 · see citing works
Woods, C. D., Naroo, S., Zeri, F., Bakkar, M. M., Barodawala, F., Evans, V., Fadel, D., Kalikivayi, L., Lira, M., Maseedupally, V. K., et al. (2023) · Contact lens & anterior eye : the journal of the British Contact Lens Association · doi:10.1016/j.clae.2023.101821 · Cited by 14 · see citing works
Developing a Scientific Literacy Assessment Instrument for Portuguese 3rd Cycle Students
Coppi, M., Fialho, I., & Cid, M. (2023) · Education Sciences · doi:10.3390/educsci13090941 · Cited by 12 · see citing works
Assessment of multiple-choice questions by item analysis for medical students’ examinations
Nojomi, M., & Mahmoudi, M. (2022) · Research and Development in Medical Education · doi:10.34172/rdme.2022.024 · Cited by 11 · see citing works
Dallı, Ö. E., & Girgin, N. K. (2023) · Journal of Wound Care · doi:10.12968/jowc.2023.32.sup6a.lxxv · Cited by 10 · see citing works
Kaewboonlert, N., Poontananggul, J., Pongsuwan, N., & Bhakdisongkhram, G. (2024) · JMIR Medical Education · doi:10.2196/58898 · Cited by 9 · see citing works
Diep, N. H., Thuy, H. T. P., Lai, D. T. B., Việt, V. V., & Chung, N. T. K. (2023) · International Journal of Education and Practice · doi:10.18488/61.v11i2.3336 · Cited by 9 · see citing works
Educational Data Mining: Multiple Choice Question Classification in Vocational School
Sucipto, S., Prasetya, D. D., & Widiyaningtyas, T. (2024) · Matrik Jurnal Manajemen Teknik Informatika dan Rekayasa Komputer · doi:10.30812/matrik.v23i2.3499 · Cited by 7 · see citing works
Skalka, J., & Valko, M. (2024) · IEEE Access · doi:10.1109/access.2024.3485505 · Cited by 7 · see citing works
Yao, D., Zhang, X., & Liu, Y. (2022) · Sustainability · doi:10.3390/su142215226 · Cited by 7 · see citing works
Kulshreshtha, S., Gupta, G., Goyal, G., Gupta, K., & Davda, K. (2024) · The Journal of Obstetrics and Gynecology of India · doi:10.1007/s13224-023-01904-2 · Cited by 6 · see citing works
Dhanvijay, A. K. D., Dhokane, N. B., Balgote, S., Kumari, A., Juhi, A., Mondal, H., & Gupta, P. (2023) · Cureus · doi:10.7759/cureus.44049 · Cited by 6 · see citing works
Dhanvijay, A. K. D., Kumari, A., Pinjar, M. J., Kumari, A., Ganguly, A., Priya, A., Juhi, A., Gupta, P., & Mondal, H. (2025) · AJP Advances in Physiology Education · doi:10.1152/advan.00197.2025 · Cited by 5 · see citing works
Gedamu, S., & Schauber, S. K. (2024) · Teaching and Learning in Medicine · doi:10.1080/10401334.2024.2428191 · Cited by 5 · see citing works
Olipas, C. N. P., & Luciano, R. G. (2024) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2024.03.01 · Cited by 5 · see citing works
Fauzie, M., Pada, A. U. T., & Supriatno, S. (2021) · Jurnal Penelitian dan Evaluasi Pendidikan · doi:10.21831/pep.v25i2.42603 · Cited by 5 · see citing works
Mufti, A., Ali, K., Sharma, G., & Saleem, M. (2025) · Avicenna Journal of Medicine · doi:10.1055/s-0045-1813040 · Cited by 4 · see citing works
Item Analysis in Multiple Choice Questions: A Study on Question Difficulty and Authors’ Evaluation
Konakcı, S. (2024) · Journal of Basic and Clinical Health Sciences · doi:10.30621/jbachs.1427464 · Cited by 4 · see citing works
Chen, Y., Yau, K. W., Chai, C. S., Chiu, T. K. F., Meng, H. M. L., King, I., Wong, S. W. H., & Yam, Y. (2024) · International Symposium on Educational Technology · doi:10.1109/iset61814.2024.00010 · Cited by 3 · see citing works
Ganji, K. K., Ananthakrishnan, N., Manivasakan, S., Alruwaili, M. K., Alonazi, M. A., & Algarni, H. A. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07871-8 · Cited by 3 · see citing works
Assessment of distractor efficiency of MCQS in item analysis.
Ansari, M., Sadaf, R., Akbar, A., Rehman, S., Chaudhry, Z. R., & Shakir, S. (2022) · The Professional Medical Journal · doi:10.29309/tpmj/2022.29.05.6955 · Cited by 3 · see citing works
Shenoy, V., Ravi, P., & Chandy, D. (2023) · National Journal of Clinical Anatomy · doi:10.4103/njca.njca_9_23 · Cited by 3 · see citing works
Quality Assurance of Multiple-Choice Questions Test Through Item Analysis
Farooq, M. U.-Z., & Mashood, S. (2023) · Life and Science · doi:10.37185/lns.1.1.315 · Cited by 3 · see citing works
CHENG, Y., Lai, H., Chen, I., Chiu, Y., Lee, P.-I., Peters, K., & Lee, P. (2024) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000616 · Cited by 3 · see citing works
Lee, H.-Y., Yune, S. J., Lee, S. Y., Im, S. J., & Kam, B. S. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05577-x · Cited by 3 · see citing works
Morshed, A., & Al-Jabaly, S. M. (2024) · Journal of Infrastructure Policy and Development · doi:10.24294/jipd.v8i6.3858 · Cited by 3 · see citing works
Huang, J., Liang, T., Li, J., Liu, Q., Pang, J., Xu, Y., & Xie, H. W. (2025) · PeerJ · doi:10.7717/peerj.19505 · Cited by 2 · see citing works
Mondal, H., Kumari, A., Priya, A., Dhanvijay, A. K. D., Gayen, R. K., Mondal, S., & Sarangi, P. K. (2025) · Cureus · doi:10.7759/cureus.99299 · Cited by 2 · see citing works
Zisberg, A., Birati, Y., Fraiman, P.-S., Sheps, J., Peterfreund, I., Gilmartin, M. J., Kolpina, L. V., & Squires, A. P. (2026) · BMC Nursing · doi:10.1186/s12912-026-04332-5 · Cited by 2 · see citing works
Shammas, M., Nagda, S. J., Shah, C. J., Baxi, G. D., Gadde, P., Sachdeva, S., Gupta, D., Wali, O., Dhall, R. S., & Gajdhar, S. (2024) · Journal of Dental Education · doi:10.1002/jdd.13462 · Cited by 2 · see citing works
Alawgali, S. M. (2024) · Mağallaẗ al-ʿulūm wa-al-dirāsāt al-insāniyyaẗ. · doi:10.37376/jsh.vi80.6626 · Cited by 2 · see citing works
An analysis of calculus application questions in grade 12 mathematics exams in South Africa
Machisi, E. (2024) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/15431 · Cited by 2 · see citing works
AI versus human-generated MCQs: The need for psychometric analysis
Mondal, H., & Dhanvijay, A. K. D. (2025) · Medical Teacher · doi:10.1080/0142159x.2025.2532776 · Cited by 2 · see citing works
Using item analysis to evaluate hand hygiene self-assessments at Alberta health services
Kumar, V. (2022) · American Journal of Infection Control · doi:10.1016/j.ajic.2022.08.030 · Cited by 2 · see citing works
The equation for medical multiple-choice question testing time estimation
Kreepala, C., Keeratibharat, N., Aekgawong, S., Wattanavaekin, K., Danjittrong, T., Juntararuangtong, T., & Chombandit, T. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002010 · Cited by 1 · see citing works
Understanding the factors affecting duration in answering MCQ examination: The students’ perspective
Kreepala, C., Thongsombat, S., Wattanavaekin, K., Danjittrong, T., Keeratibharat, N., & Juntararuangtong, T. (2025) · The Asia Pacific Scholar · doi:10.29060/taps.2025-10-2/oa3332 · Cited by 1 · see citing works
Lisac, D. V., Radanović, I., & Šašić, S. Š. (2025) · Natural sciences education · doi:10.1002/nse2.70023 · Cited by 1 · see citing works
García-Beltrán, E., & Sandoval, M. A. C. (2025) · Edutec Revista Electrónica de Tecnología Educativa · doi:10.21556/edutec.2025.94.4079 · Cited by 2 · see citing works
Adaptive E-Assessment Framework
Pratiwi, O. N., Fa’rifah, R. Y., Pamungkas, I. R., Megasari, R., & Callista, A. S. (2023) · International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS) · doi:10.1109/icadeis58666.2023.10270915 · Cited by 1 · see citing works
Starke, S. J., Rivera, M. B. M., Krishnan, S., & Shah, M. (2023) · Journal of General Internal Medicine · doi:10.1007/s11606-023-08551-3 · Cited by 1 · see citing works
Test Item Analysis: A Catalyst for Quality Assurance of Assessment in Outcome-Based Education
Pujari, A. A., Sawashe, K. S., Watharkar, S. R., & Nishandar, S. V. (2024) · Journal of Engineering Education/Journal of engineering education transformations/Journal of engineering education transformation · doi:10.16920/jeet/2024/v37is2/24125 · Cited by 1 · see citing works
Multiple-Choice Questions in Basic Biomedical Science Module
Permasutha, M. B., Rahayu, G. R., Giri, M. K. W., & Pradiptha, D. A. G. F. (2024) · Jurnal Pendidikan dan Pengajaran · doi:10.23887/jpp.v57i1.63314 · Cited by 1 · see citing works
Othman, A. S., Idris, S., Latiff, A. A., Manan, N. A., & Omar, N. S. (2024) · Education in Medicine Journal · doi:10.21315/eimj2024.16.s1.3 · Cited by 1 · see citing works
Rynando, N. G., Kusumawinakhyu, T., & Annisa, Y. (2025) · International Journal of Islamic and Complementary Medicine · doi:10.55116/ijicm.v6i2.94 · Cited by 1 · see citing works
Implementación de prácticas innovadoras en la gestión educativa del Instituto de Idiomas de la UPS
Universidad Politécnica Salesiana, I. D. I., & Albán, A. J. G. (2025) · doi:10.17163/abyaups.127.20 · Cited by 1 · see citing works
1C-57, N. F., Gulia, R., & Saha, T. K. (2022) · Batna Journal of Medical Sciences (BJMS) · doi:10.48087/bjmsoa.2022.9105
Shahid, R., Asad, M. B., & Khalid, R. (2022) · Pakistan Journal of Health Sciences · doi:10.54393/pjhs.v3i05.200
Lasheras, F. S., Curbelo, J., Romero, J. B., Guerrero, A. G., Cristóbal, C. P. S., Villacampa, T., & Fonseca, P. J. (2022) · Mathematics · doi:10.3390/math10234543
Kreepala, C., Thongsombat, S., Wattanavaekin, K., Danjittrong, T., Keeratibharat, N., & Juntararuangtong, T. (2023) · Research Square · doi:10.21203/rs.3.rs-3019852/v1
Nguyen, N. T. T., Hao, N. T., Thu, H., & Nguyen, L. T. H. (2023) · Journal of Educational and Social Research · doi:10.36941/jesr-2023-0120
Scoping Review: Item Analysis Pada Multiple Choice Questions (MCQs) dalam Pembelajaran
Yunita, S., Kristianti, Y. D., Martin, N., Harefa, N., Yana, E., Aritonang, P. Z., Safitri, D. N., Susanti, A. I., & Adnani, Q. E. S. (2023) · Jurnal Inovasi Pendidikan MH Thamrin · doi:10.37012/jipmht.v7i1.1671
Pinem, Y. A., Fajar, N. S., & Widagdo, D. (2024) · Jurnal Manajemen Dirgantara · doi:10.56521/manajemen-dirgantara.v16i02.1044
Analisis Kualitas Tes dan Butir Soal Integral pada Evaluasi Formatif Matematika Teknik
Trianingsih, L. (2023) · Indonesian Journal Of Civil Engineering Education · doi:10.20961/ijcee.v9i2.83605
Analisis Kualitas Tes dan Butir Soal Integral pada Evaluasi Formatif Matematika Teknik
Trianingsih, L. (2023) · Indonesian Journal Of Civil Engineering Education · doi:10.20961/ijcee.v9i2.84711
Yakout, S. M., Hadi, K. E., Alolayan, M., Alasmari, M. M., Alhaji, K. M., Alobaid, L. M., Alonazi, M. A., & Alharbi, M. K. (2024) · Journal Of Advanced Zoology · doi:10.53555/jaz.v45i3.4138
The psychological capital questionnaire for newly licensed nurses: A validity and reliability studyPreprint
Jin, M., Wang, J., Qian, R., Yang, H., Huang, S., Liao, M., Liu, X., Liao, D., & Qi, S. (2024) · Research Square · doi:10.21203/rs.3.rs-4120162/v1
Language and math skills among post-secondary deaf students
Allala, S. K. (2024) · doi:10.53555/kuey.v30i4.1685
Quality of multiple choice question items: item analysis
Alemu, A. T., Tesfa, H., Mulugeta, A., Fenta, E. T., & Belay, M. A. (2024) · International Journal of Scientific Reports · doi:10.18203/issn.2454-2156.intjscirep20241316
Raoof, R. M. (2024) · Annals of the College of Medicine Mosul/Annals of the College of Medecine · doi:10.33899/mmed.2024.146653.1252
Comparison of Methods for Identifying Rapid Guessing Behavior in Microlearning Courses
Skalka, J., & Valko, M. (2025) · Springer eBooks · doi:10.1007/978-3-031-82243-8_4
Enhancing Biochemistry Assessments: Our Experience with MCQ Validation in Punjab
Gupta, S., Kaur, H., Kaur, V., Kaur, S., & Lakha, N. (2025) · Indian Journal of Medical Biochemistry · doi:10.5005/jp-journals-10054-0254
Improving High-Stakes Obstetrics & Gynaecology Exams: A 3-year Analysis of SBAs
Phyo, Z., Soe, T. T., Aung, Y. P., Naing, T. T., Hein, T. I. A., & Aung, D. (2025) · South-East Asian Journal of Medical Education · doi:10.4038/seajme.v19i1.651
Quansah, N., Quansah, F., & Dzakadzie, Y. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1658971
Additional file 1 of Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · Figshare · doi:10.6084/m9.figshare.26699639
Rashwan, N. I., Aref, S. R., Nayel, O. A., & Rizk, M. H. (2024) · Figshare · doi:10.6084/m9.figshare.26684097
Additional file 3 of Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · Figshare · doi:10.6084/m9.figshare.26699645
Additional file 2 of Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · Figshare · doi:10.6084/m9.figshare.26699642
Nagasrilatha, B., Mohiddin, S. K., & Shahid, M. (2025) · The Journal of Medical Research · doi:10.4103/jmr.jmr_80_25
Loganathan, S. (2026) · Journal of English Language Teaching · doi:10.66121/m3rgst76
Difficulty level of EFL test designed by pre‑service teachers
Rudy, M., Yusuf, F. N., Emilia, E., & Gunawan, W. (2026) · Australian Review of Applied Linguistics · doi:10.1075/aral.25009.rud
Kour, H., Kaur, M., Kanga, R. R., Ramesha, H., & Patil, P. P. (2026) · Indian Journal of Physiology and Pharmacology · doi:10.25259/ijpp_294_2025
Ismail, A., Al-Modallal, H. M., Aldardeir, N. E., Gomma, N. H., & Eltohamy, N. A. E. (2026) · Journal of Nursing Management · doi:10.1155/jonm/3985380
How to Use Large Language Model Chatbots in Multiple-choice Question Generation
Mondal, H., Dhanvijay, A. K. D., & Mondal, S. (2026) · Indian Journal of Vascular and Endovascular Surgery · doi:10.4103/ijves.ijves_126_25
Agha, S., Rumayyan, A. R. A., Qazi, N., & Shah, S. A. A. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09376-4
Dong, B., Zhou, Y., Qu, Y., Zhai, J., Zhao, S., Yu, J., Qin, Y., & Xu, T. (2026) · Virtual Reality · doi:10.1007/s10055-026-01380-9
Mondal, H., Sarangi, P. K., Mondal, S., Dhanvijay, A. K. D., Gayen, R. K., Mukhopadhyay, A., & Mishra, R. (2026) · Cureus · doi:10.7759/cureus.109190
AI Literacy Test for University Students: Design Principles, Content Validation, and Pilot Testing
Zhang, L., & Zhang, S. (2026) · International Conference on Educational and Information Technology · doi:10.1109/iceit68991.2026.11521668
Effectiveness of a Numerical Problem-Solving Module in Enhancing Renal Physiology Comprehension
Agarwal, M., Goyal, M., & Mishra, P. (2026) · Cureus · doi:10.7759/cureus.110591
Sangam, M. R., Praveen, K., Vinay, G., Kaur, A., & Deka, R. (2026) · National Journal of the Society of Medical Anatomists · doi:10.4103/njsoma.njsoma_3_26
Using AI to Enhance Knowledge Among STEM Students
Heredia, A. S., Díaz, M., Regueiro, J. M. P., Garcia-Ortiz, C. E., & Vázquez‐Villegas, P. (2026) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon67543.2026.11574114
Benchmarking large language models against practicing clinicians on psychopathological assessment
Lenz, E., Naamanka, J., Trabert, W., Bottlender, R., Malchow, B., Meyer‐Lindenberg, A., Gradinger, T., & Schwarz, E. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02852-7
Wang, Z., Hui, T., Minhus, S., & Majid, I. (2026) · International Journal of Fashion Design Technology and Education · doi:10.1080/17543266.2026.2697200
Kulkarni, S. L., Daragad, M., & Kour, H. (2026) · Medical Journal of Dr D Y Patil Vidyapeeth · doi:10.4103/mjdrdypu.mjdrdypu_567_25
- Authors
- Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V.
- Affiliations
- Armed Forces Medical College (IN)
- Corresponding author
- Jaipurkar, Raksha
- Venue
- Medical Journal Armed Forces India
- Publisher
- Elsevier BV
- Published
- 2021-02
- Volume
- 77
- Issue
- Suppl 1
- Pages
- S85-S89
- ISSN
- 0377-1237 (ISSN-L)
- PMC
- PMC7873707
- OpenAlex
- W3127024416
- Cited by
- 109
- References
- 25
- Citations by year
- 2026: 24 · 2025: 19 · 2024: 33 · 2023: 21 · 2022: 9 · 2021: 3
- FWCI
- 14.27
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Journal Armed Forces India (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- item analysis, multiple-choice questions, item difficulty, item discrimination, distractor effectiveness, question bank, test validity, test reliability, medical education assessment, physiology education, MBBS students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
