The DOI matches its Crossref record; 22 of the 25 references it cites were found in a registry or an index; none of the 4 sources its claims cite is checked for support yet.
Item analysis of multiple choice questions: A quality assurance test for an assessment tool
Open accessAbstract
Background The item analysis of multiple choice questions (MCQs) is an essential tool that can provide input on validity and reliability of items. It helps to identify items which can be revised or discarded, thus building a quality MCQ bank.Methods The study focussed on item analysis of 90 MCQs of three tests conducted for 150 first year Bachelor of Medicine and Bachelor of Surgery (MBBS) physiology students. The item analysis explored the difficulty index (DIF I) and discrimination index (DI) with distractor effectiveness (DE). Statistical analysis was performed by using MS Excel 2010 and SPSS, version 20.0.Results Of total 90 MCQs, the majority, that is, 74 (82%) MCQs had a good/acceptable level of difficulty with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult and nine (10%) were too easy. A total of 72 (80%) items had an excellent to acceptable DI and 18 (20%) had a poor DI with an overall mean DI of 0.31 ± 0.12. There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001). The mean DE was 32.35 ± 31.3 with 73% functional distractors in all. The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good. The standard error of measurement was 1.22.Conclusion Our study helped teachers identify good and ideal MCQs which can be part of the question bank for future and those MCQs which needed revision. We recommend that item analysis must be performed for all MCQ-based assessments to determine validity and reliability of the assessment.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This study performed item analysis on 90 multiple choice questions from three physiology assessments for 150 first-year MBBS students to evaluate difficulty, discrimination, distractor effectiveness, and reliability. Most items had acceptable difficulty and discrimination, but only 20% met all criteria for an ideal MCQ.
Key findings
- Of 90 MCQs, 74 (82%) had good/acceptable difficulty (DIF I 30–70%; mean 55.32 ± 7.4), 7 (8%) were too difficult, and 9 (10%) were too easy; 34 (38%) were excellent/ideal (DIF I 50–60%).
- For discrimination, 72 (80%) items had acceptable-to-excellent DI (>0.20) and 18 (20%) had poor DI (≤0.20); 61 (68%) had good-to-excellent DI (≥0.25), with overall mean DI 0.31 ± 0.12. DIF I and DI showed a weak correlation (r = 0.140, p < .0001).
- Of 270 distractors, 198 (73%) were functional and 72 (27%) were non-functional; DE was poor in 2 (2%), moderate in 8 (9%), good in 50 (56%), and excellent in 30 (33%) items. Mean DE was 32.35 ± 31.3.
- Reliability was Cronbach alpha 0.85, KR-20 0.71, and SEM 1.22; 18 (20%) MCQs fulfilled all three criteria for an ideal MCQ (DIF I 30–70%, DI ≥0.25, 100% DE).
Methods
- Item analysis of 90 single-best-response MCQs from three internal assessments for 150 first-year MBBS physiology students; each MCQ had a stem and four options with one correct response and three distractors, scored 1 for correct and 0 for wrong, with no negative marking.
- The upper and lower 27% (41 students each) were classified as high and low achievers; DIF I was calculated as (H + L) × 100/N, DI as 2 × (H − L)/N, and DE was based on non-functional distractors selected by <5% of students.
- Internal consistency was assessed with Cronbach alpha, KR-20, and SEM; skewness and kurtosis were also measured; analysis used MS Excel 2010 and SPSS version 20.0.
Limitations
- Not stated in the paper; the text does not include an explicit limitations section.
- Evident from the text: the study was conducted in a single institution and single discipline (physiology) with first-year MBBS students across three internal assessments, so generalizability is limited.
- Evident from the text: the analysis is descriptive and does not report follow-up after revising or discarding flawed items, nor does it evaluate whether item revisions improved future assessments.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 4 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results of the DIF I showed that of 90 MCQs, 74 (82%) MCQs had good/acceptable levels of difficulty (DIF I 30–70%), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among all MCQs, 34 (38%) MCQs had excellent/ideal levels of difficulty (DIF I 50–60%) (Table 2).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of total 90 MCQs, 18 (20%) MCQ items had a poor DI (≤0.20), 11 (12%) items had an acceptable DI (0.21–0.24), 32 (36%) showed a good DI (0.25–0.34) and 29 (32%) items showed an excellent DI (≥0.35) (Table 3).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Item analysis parameters Difficulty index Discrimination index Distractor effectiveness Mean ± SD 54.95 ± 13.4 0.31 ± 0.12 32.35 ± 31.3”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of all items, two (2%) items had poor DE (0%), eight (9%) had moderate DE (33.3%), 50 (56%) had good DE (66.6%) and remaining 30 (33%) had excellent DE (100%) (Table 4).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001).”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30–70%) with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult (DIF I <30%) and nine (10%) were too easy (DIF I >70%).”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 2
- It may state two separate claims as one. 5
In the paper
“In this study, Skewness and Kurtosis values (0.59 and 0.69) represent the moderately skewed data with tail on the right side of the distribution and data are lighttailed (lack of outliers) i.e distribution of data is approximately normal.17”
What the sources say
- Not checked yetcites [17] Mishra, 2019, Descriptive statistics and normality tests for statistical data, Ann Card Anaesth source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The SEM is directly related to the reliability of the test.”
What the sources say
- Not checked yetcites [11] McManus, 2012, The misinterpretation of the standard error of measurement in medical education: a primer on the problems, pitfalls and peculiarities of the three different standard errors of measurement, Med Teach source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this study, the results showed that the overall validity of the test was good with a Cronbach alpha of 0.85, KR 20 = 0.71 and SEM 1.22.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, items were acceptably difficult and were good at differentiating students with higher and lower abilities.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“More NFD makes MCQ easy and vice versa with more functioning distractors making it difficult.23”
What the sources say
- Not checked yetcites [23] Tarrant, 2009, An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis, BMC Med Educ source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items analysed in this study were neither too easy nor too difficult which is acceptable.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The present study shows weak correlation between the DIF I and DI (r = 0.140, p < .0001), indicating that with increasing the DIF I, the ability to discriminate between the high and low scorers decreased and fewer easy questions were used in MCQ tests.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The authors state this as an assumption.
Any test score contains an error.
“Based on the assumption that any test score contains an error, SEM is used to estimate a band or interval within which a person's true score would fall, that is, the score if there were no error of measurement.”
The paper relies on this without support.
A non-functional distractor can reduce the discrimination power of an MCQ.
“A non-functional distractor can reduce the discrimination power of a MCQ.”
The paper relies on this without support.
The efficiency of MCQ assessment is solely based upon the quality of test MCQs.
“The efficiency of MCQ assessment is solely based upon the quality of test MCQs.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The DIS ranged from 1.0 to − 1.0.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
In the citing paper
“Item analysis provides feedback about item construction and its validity and reliability after it appears in an examination [5, 35, 36].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
In the citing paper
“A positive DIS indicates that high achievers answered the item more correctly than lower achievers and vice versa [35, 44].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [35]
In the citing paper
“Such findings were reported in anatomy exam [48], community Medicine [49], and multidisciplinary summative exams [50, 51] that were constructed traditionally.”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [51]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Positive correlation is described as a decrease
MajorA positive correlation indicates that the two indices tend to increase together, whereas the discussion says discrimination decreased as the difficulty index increased.
“We found a weak correlation between the DIF I and DI (r = 0.140, p < .0001).”
“with increasing the DIF I, the ability to discriminate between the high and low scorers decreased”
Mean test score is presented as both a score and a percentage
MinorThe results give 47.35 as a mean score out of 90, while Table 1 labels the same value a percentage. A score of 47.35 out of 90 is not 47.35%.
“The test score of class ranged from 27 to 82 (total score 90) with a mean test score of 47.35 ± 11.6 (mean ± SD).”
“Percentage of mean test score ± SD 47.35 ± 11.6”
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Descriptive statistics and normality tests for statistical data
Gupta, A., Mishra, P., Pandey, C., Singh, U., Sahu, C., & Keshri, A. (2019) · Annals of Cardiac Anaesthesia · doi:10.4103/aca.aca_157_18 · Cited by 2,628 · cited 1×
“e distribution of data is approximately normal.(17) In this study, of total 90 items, the majority, that is, 74 (82%) had a good/acceptable level of DIF I (30e70%) with a mean DIF I of 55.”
BackgroundThe theory of the estimation of test reliability
Kuder, G. F., & Richardson, M. W. (1937) · Psychometrika · doi:10.1007/bf02288391 · Cited by 1,953
Assessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872
Cain, M. K., Zhang, Z. J., & Yuan, K. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0814-1 · Cited by 1,172
The selection of upper and lower groups for the validation of test items.
Kelley, T. L. (1939) · Journal of Educational Psychology · doi:10.1037/h0057123 · Cited by 777
Improving the fairness of multiple-choice questions: a literature review
McCoubrie, P. (2004) · Medical Teacher · doi:10.1080/01421590400013495 · Cited by 318
The Approximate Sampling Distribution of Kuder-Richardson Reliability Coefficient Twenty
Feldt, L. S. (1965) · Psychometrika · doi:10.1007/bf02289499 · Cited by 296
Tarrant, M., Ware, J. S., & Mohammed, A. M. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-40 · Cited by 238
Item analysis of multiple choice questions: Assessing an assessment tool in medical students
Rao, C., Kishan Prasad, H., Sajitha, K., Permi, H., & Shetty, J. (2016) · International Journal of Educational and Psychological Researches · doi:10.4103/2395-2296.189670 · Cited by 96
Flagged first, then by citations
3 of the 25 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Patil, 2015, Item analysis of medicine multiple choice questions (MCQs) for undergraduate (3rd year MBBS) students, Res J Pharmaceut Biol Chem Sci, 6, 1242
Cited by
Sorted by citations
Items 101–107 of 107
Atreya, A., Acharya, D. R., Nepal, S., Sapkota, L. P., Menezes, R. G., & Luitel, B. C. (2026) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205261474117
Ullah, T., Salim, F. N., & Shah, J. A. (2026) · Research Mosaic · doi:10.64428/rm/v6.i2.5
Measuring Chemistry Students’ Ability through Entry Test Using the 1-PL IRT Model
Manoppo, Y., & Latuny, L. S. (2026) · Jurnal Penelitian dan Pengembangan Pendidikan · doi:10.23887/jppp.v10i1.102371
Regional Language Interference in Indonesian Phonology among Elementary School Students
Nahak, K. B., Nahak, M. M. N., Lasi, F., Seran, Y., Lein, A. L., & Verdial, J. D. C. (2026) · Jurnal Penelitian dan Pengembangan Pendidikan · doi:10.23887/jppp.v10i1.102894
Zhu, S.-R., Yusoff, H., Cheng, K. Y., Yusoff, D. M., Feng, X., Zheng, W., & Chen, H. (2026) · Nursing Open · doi:10.1002/nop2.70925
Ouh, E. L., Tan, K. W., Lo, S. L., Gan, B. K. S., & Zhang, Y. (2026) · ACM Transactions on Computing Education · doi:10.1145/3858331
Salahuddin, M., & Stupnisky, R. H. (2026) · Higher Education Quarterly · doi:10.1111/hequ.70179
Metadata
- Authors
- Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V.
- Affiliations
- Armed Forces Medical College (IN)
- Corresponding author
- Jaipurkar, Raksha
- Venue
- Medical Journal Armed Forces India
- Publisher
- Elsevier BV
- Published
- 2021-02
- Volume
- 77
- Issue
- Suppl 1
- Pages
- S85-S89
- ISSN
- 0377-1237 (ISSN-L)
- PMC
- PMC7873707
- OpenAlex
- W3127024416
- Cited by
- 109
- References
- 25
- Citations by year
- 2026: 24 · 2025: 19 · 2024: 33 · 2023: 21 · 2022: 9 · 2021: 3
- FWCI
- 14.27
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Journal Armed Forces India (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Educational Assessment and Pedagogy
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- item analysis, multiple-choice questions, item difficulty, item discrimination, distractor effectiveness, question bank, test validity, test reliability, medical education assessment, physiology education, MBBS students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
