The DOI matches its Crossref record; 22 of the 25 references it cites were found in a registry or an index; none of the 4 sources its claims cite is checked for support yet.
Item analysis of multiple choice questions: A quality assurance test for an assessment tool
Open accessAbstract
Background The item analysis of multiple choice questions (MCQs) is an essential tool that can provide input on validity and reliability of items. It helps to identify items which can be revised or discarded, thus building a quality MCQ bank.Methods The study focussed on item analysis of 90 MCQs of three tests conducted for 150 first year Bachelor of Medicine and Bachelor of Surgery (MBBS) physiology students. The item analysis explored the difficulty index (DIF I) and discrimination index (DI) with distractor effectiveness (DE). Statistical analysis was performed by using MS Excel 2010 and SPSS, version 20.0.Results Of total 90 MCQs, the majority, that is, 74 (82%) MCQs had a good/acceptable level of difficulty with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult and nine (10%) were too easy. A total of 72 (80%) items had an excellent to acceptable DI and 18 (20%) had a poor DI with an overall mean DI of 0.31 ± 0.12. There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001). The mean DE was 32.35 ± 31.3 with 73% functional distractors in all. The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good. The standard error of measurement was 1.22.Conclusion Our study helped teachers identify good and ideal MCQs which can be part of the question bank for future and those MCQs which needed revision. We recommend that item analysis must be performed for all MCQ-based assessments to determine validity and reliability of the assessment.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This study performed item analysis on 90 multiple choice questions from three physiology assessments for 150 first-year MBBS students to evaluate difficulty, discrimination, distractor effectiveness, and reliability. Most items had acceptable difficulty and discrimination, but only 20% met all criteria for an ideal MCQ.
Key findings
- Of 90 MCQs, 74 (82%) had good/acceptable difficulty (DIF I 30–70%; mean 55.32 ± 7.4), 7 (8%) were too difficult, and 9 (10%) were too easy; 34 (38%) were excellent/ideal (DIF I 50–60%).
- For discrimination, 72 (80%) items had acceptable-to-excellent DI (>0.20) and 18 (20%) had poor DI (≤0.20); 61 (68%) had good-to-excellent DI (≥0.25), with overall mean DI 0.31 ± 0.12. DIF I and DI showed a weak correlation (r = 0.140, p < .0001).
- Of 270 distractors, 198 (73%) were functional and 72 (27%) were non-functional; DE was poor in 2 (2%), moderate in 8 (9%), good in 50 (56%), and excellent in 30 (33%) items. Mean DE was 32.35 ± 31.3.
- Reliability was Cronbach alpha 0.85, KR-20 0.71, and SEM 1.22; 18 (20%) MCQs fulfilled all three criteria for an ideal MCQ (DIF I 30–70%, DI ≥0.25, 100% DE).
Methods
- Item analysis of 90 single-best-response MCQs from three internal assessments for 150 first-year MBBS physiology students; each MCQ had a stem and four options with one correct response and three distractors, scored 1 for correct and 0 for wrong, with no negative marking.
- The upper and lower 27% (41 students each) were classified as high and low achievers; DIF I was calculated as (H + L) × 100/N, DI as 2 × (H − L)/N, and DE was based on non-functional distractors selected by <5% of students.
- Internal consistency was assessed with Cronbach alpha, KR-20, and SEM; skewness and kurtosis were also measured; analysis used MS Excel 2010 and SPSS version 20.0.
Limitations
- Not stated in the paper; the text does not include an explicit limitations section.
- Evident from the text: the study was conducted in a single institution and single discipline (physiology) with first-year MBBS students across three internal assessments, so generalizability is limited.
- Evident from the text: the analysis is descriptive and does not report follow-up after revising or discarding flawed items, nor does it evaluate whether item revisions improved future assessments.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
23 Own result · 7 Own interpretation · 4 Not checked yet
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among all MCQs, 34 (38%) MCQs had excellent/ideal levels of difficulty (DIF I 50–60%) (Table 2).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Item analysis parameters Difficulty index Discrimination index Distractor effectiveness Mean ± SD 54.95 ± 13.4 0.31 ± 0.12 32.35 ± 31.3”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30–70%) with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 2
- It may state two separate claims as one. 5
In the paper
“In this study, Skewness and Kurtosis values (0.59 and 0.69) represent the moderately skewed data with tail on the right side of the distribution and data are lighttailed (lack of outliers) i.e distribution of data is approximately normal.17”
What the sources say
- Not checked yetcites [17] Mishra, 2019, Descriptive statistics and normality tests for statistical data, Ann Card Anaesth source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The SEM is directly related to the reliability of the test.”
What the sources say
- Not checked yetcites [11] McManus, 2012, The misinterpretation of the standard error of measurement in medical education: a primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement, Med Teach source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this study, the results showed that the overall validity of the test was good with a Cronbach alpha of 0.85, KR 20 = 0.71 and SEM 1.22.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, items were acceptably difficult and were good at differentiating students with higher and lower abilities.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“More NFD makes MCQ easy and vice versa with more functioning distractors making it difficult.23”
What the sources say
- Not checked yetcites [23] Tarrant, 2009, An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis, BMC Med Educ source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items analysed in this study were neither too easy nor too difficult which is acceptable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The present study shows weak correlation between the DIF I and DI (r = 0.140, p < .0001), indicating that with increasing the DIF I, the ability to discriminate between the high and low scorers decreased and fewer easy questions were used in MCQ tests.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
Any test score contains an error.
“Based on the assumption that any test score contains an error, SEM is used to estimate a band or interval within which a person's true score would fall, that is, the score if there were no error of measurement.”
The paper relies on this without support.
A non-functional distractor can reduce the discrimination power of an MCQ.
“A non-functional distractor can reduce the discrimination power of a MCQ.”
The paper relies on this without support.
The efficiency of MCQ assessment is solely based upon the quality of test MCQs.
“The efficiency of MCQ assessment is solely based upon the quality of test MCQs.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The DIS ranged from 1.0 to − 1.0.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredIn the citing paper
“Item analysis provides feedback about item construction and its validity and reliability after it appears in an examination [5, 35, 36].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
In the citing paper
“A positive DIS indicates that high achievers answered the item more correctly than lower achievers and vice versa [35, 44].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
In the citing paper
“Such findings were reported in anatomy exam [48], community Medicine [49], and multidisciplinary summative exams [50, 51] that were constructed traditionally.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Results
“We found a weak correlation between the DIF I and DI (r = 0.140, p < .0001).”
§ Discussion
“with increasing the DIF I, the ability to discriminate between the high and low scorers decreased”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Results
“The test score of class ranged from 27 to 82 (total score 90) with a mean test score of 47.35 ± 11.6 (mean ± SD).”
§ Results
“Percentage of mean test score ± SD 47.35 ± 11.6”
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Descriptive statistics and normality tests for statistical data
Gupta, A., Mishra, P., Pandey, C., Singh, U., Sahu, C., & Keshri, A. (2019) · Annals of Cardiac Anaesthesia · doi:10.4103/aca.aca_157_18 · Cited by 2,628 · cited 1×
“e distribution of data is approximately normal.(17) In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30e70%) with a mean DIF I of 55.”
BackgroundThe theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Assessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872
Cain, M. K., Zhang, Z. J., & Yuan, K. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0814-1 · Cited by 1,172
The selection of upper and lower groups for the validation of test items.
Kelley, T. L. (1939) · Journal of Educational Psychology · doi:10.1037/h0057123 · Cited by 777
Improving the fairness of multiple-choice questions: a literature review
McCoubrie, P. (2004) · Medical Teacher · doi:10.1080/01421590400013495 · Cited by 318
The Approximate Sampling Distribution of Kuder-Richardson Reliability Coefficient Twenty
Feldt, L. S. (1965) · Psychometrika · doi:10.1007/bf02289499 · Cited by 296
Tarrant, M., Ware, J. S., & Mohammed, A. M. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-40 · Cited by 238
Item analysis of multiple choice questions: Assessing an assessment tool in medical students
Rao, C., Kishan Prasad, H., Sajitha, K., Permi, H., & Shetty, J. (2016) · International Journal of Educational and Psychological Researches · doi:10.4103/2395-2296.189670 · Cited by 96
Correlation between difficulty & discrimination indices of MCQs in formative exam in Physiology
Pande, S., Pande, S. R., Parate, V., Nikam, A. P., & Sh, A. (2013) · South-East Asian Journal of Medical Education · doi:10.4038/seajme.v7i1.149 · Cited by 61
Karelia, B. (2013) · International e-Journal of Science Medicine & Education · doi:10.56026/imu.7.2.41 · Cited by 47
EditorialCompetency-based assessment: making it a reality
Davis, M. H., & Harden, R. M. (2003) · Medical Teacher · doi:10.1080/0142159032000153842 · Cited by 45
McManus, I. C. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.670318 · Cited by 32
Item analysis as tool to validate multiple choice question bank in pharmacology
Date, A. P., Borkar, A. S., Badwaik, R. T., Siddiqui, R. A., Shende, T. R., & Dashputra, A. V. (2019) · International Journal of Basic & Clinical Pharmacology · doi:10.18203/2319-2003.ijbcp20194106 · Cited by 27
Itemized Analysis of Questions of Multiple Choice Question (MCQ) Exam
25, S. S., Patel, K., & Mahajan, N. (2012) · International Journal of Scientific Research · doi:10.15373/22778179/feb2013/94 · Cited by 21
Christian, D. S., Prajapati, A. C., Rana, B. M., & Dave, V. (2017) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20172004 · Cited by 20
Study on item and test analysis of multiple choice questions amongst undergraduate medical students
Ingale, A. S., Giri, P., & Doibale, M. K. (2017) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20171764 · Cited by 18
The influence of distractor strength and response order on MCQ responding
Kiat, J. E., Ong, A. R., & Ganesan, A. (2018) · Educational Psychology · doi:10.1080/01443410.2017.1349877 · Cited by 15
Student assessment: Moving over to programmatic assessment
Singh, T. (2016) · International Journal of Applied and Basic Medical Research · doi:10.4103/2229-516x.186955 · Cited by 14
Item Analysis of Multiple Choice Questions in Medical Licensing Examination
Gomboo, A., Gombo, B., Munkhgerel, T., Sumberzul, N., & Badamdorj, O. (2019) · Central Asian Journal of Medical Sciences · doi:10.24079/cajms.2019.06.009 · Cited by 9
Difficulty Index, Discrimination Index and Distractor Efficiency in Multiple Choice Questions
Mahjabeen, W., Alam, S., Hassan, U., Zafar, T., Butt, R., Konain, S., & Rizvi, M. (2018) · Annals of PIMS-Shaheed Zulfiqar Ali Bhutto Medical University · Cited by 7
Research works printed but not linked: 1
- Patil, 2015, Item analysis of medicine multiple choice questions (MCQs) for undergraduate (3rd year MBBS) students, Res J Pharmaceut Biol Chem Sci, 6, 1242
Cited by
Sorted by citations
Cheung, B. H. H., Lau, G. K. K., Wong, G. T. C., Lee, E. Y. P., Kulkarni, D., Seow, C. S., Wong, R., & Co, M. T.-H. (2023) · PLOS ONE · doi:10.1371/journal.pone.0290691 · Cited by 195
Large language models for generating medical examinations: systematic review
Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05239-y · Cited by 128
Meo, S. A., Al-Masri, A. A., Alotaibi, M., Meo, M. Z. S., & Meo, M. O. S. (2023) · Healthcare · doi:10.3390/healthcare11142046 · Cited by 74
Agarwal, M., Sharma, P., & Goswami, A. (2023) · Cureus · doi:10.7759/cureus.40977 · Cited by 72
Rezigalla, A. A., Eleragi, A. M. S., Elhussein, A. B., Alfaifi, J. A., Alghamdi, M. A. A., Ameer, A., Yahia, A. I. O., Mohammed, O. A., & Adam, M. I. E. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05433-y · Cited by 52
Metadata
- Authors
- Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V.
- Affiliations
- Armed Forces Medical College (IN)
- Corresponding author
- Jaipurkar, Raksha
- Venue
- Medical Journal Armed Forces India
- Publisher
- Elsevier BV
- Published
- 2021-02
- Volume
- 77
- Issue
- Suppl 1
- Pages
- S85-S89
- ISSN
- 0377-1237 (ISSN-L)
- PMC
- PMC7873707
- OpenAlex
- W3127024416
- Cited by
- 109
- References
- 25
- Citations by year
- 2026: 24 · 2025: 19 · 2024: 33 · 2023: 21 · 2022: 9 · 2021: 3
- FWCI
- 14.27
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Journal Armed Forces India (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- item analysis, multiple-choice questions, item difficulty, item discrimination, distractor effectiveness, question bank, test validity, test reliability, medical education assessment, physiology education, MBBS students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
