Use of Item Analysis to Improve the Quality of Single Best Answer Multiple Choice Question in Summative Assessment of Undergraduate Medical Students in Malaysia
Open accessAbstract
Background: Single best answer (SBA) as multiple-choice items are often advantageous to use for its reliability and validity.However, SBA requires good number of plausible distractors to achieve reliability.Apart from psychometric evaluation of assessment it is important to perform item analysis to improve quality of items by analysing difficulty index (DIF I), discrimination index (DI) and distractor efficiency (DE) based on number of non-functional distractors (NFD).Objective: To evaluate quality of SBA items administered in professional examination to apply corrective measures determined by DIF I, DI and DE using students' assessment score.Method: An evaluation of post summative assessment (professional examination) of SBA items as part of psychometric assessment is performed after 86 weeks of teaching in preclinical phase of MD program.Forty SBA items and 160 distractors inclusive of key were assessed using item analysis.Hundred and thirty six students' score of SBA was analysed for mean and standard deviation, DIF I, DI and DE using MS Excel 2007.Unpaired t-test was applied to determine DE in relation to DIF I and DI with level of significance.Item-total correlation (r) and internal consistency by Cronbach's alpha and parallel-form method was also computed.Result: Fifteen items had DIF I = 0.31-0.61and 25 items had DIF I (≤ 0.30 or ≥ 0.61).Twenty six items had DI = 0.15 -≥ 0.25 compared to 14 items with DI (≤ 0.15).There were 26 (65%) items with 1-3 NFD and 14 (35%) items without any NFD.Thirty nine (32.50%) distractors were with choice frequency = 0. Overall mean DE was 65.8% and NFD was 49 (40.5%).DE in relation to DIF I and DI were statistically significant with p = 0.010 and 0.020 respectively.Item-total correlation for most items was < 0.3.Internal consistency by Cronbach's alpha in SBA Test 1 and 2 was 0.51 and 0.41 respectively and constancy by parallel-form method was 0.57 between SBA Test 1 and 2. Conclusion: The high frequency of difficult or easy items and moderate to poor discrimination suggest the need of items corrective measure.Increased number of NFD and low DE in this study indicates difficulty of teaching faculty in developing plausible distractors for SBA questions.This has been reflected in poor reliability established by alpha.Item analysis result emphasises the need of evaluation to provide feedback and to improve quality of SBA items in assessment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Values > 0.7, 0.6-0.7 is acceptable, 0.5-0.6 poor, < 0.5, and < 0.30 unreliable.
“It was reported that values > 0.7, 0.6–0.7 is acceptable, 0.5–0.6. poor, < 0.5, and < 0.30, unreliable [42].”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [42]
A KR20 of 0.68 is acceptable for in-class assessment [52].
“which is acceptable for in-class assessment [52]”
From AI in medical education: uses of AI in construction type A MCQs (2024) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Nursing Research: Principles and Methods
McFarland, G. K., Polit, D. F., & Hungler, B. P. (1987) · AJN American Journal of Nursing · doi:10.2307/3425916 · Cited by 6,942 · see citing works
Measurement and Assessment in Teaching
Linn, R. L., & Gronlund, N. E. (1994) · Medical Entomology and Zoology · Cited by 1,283 · see citing works
Developing and Validating Multiple-choice Test Items
Haladyna, T. M. (2004) · Routledge eBooks · doi:10.4324/9780203825945 · Cited by 591 · see citing works
Three Options Are Optimal for Multiple‐Choice Items: A Meta‐Analysis of 80 Years of Research
Rodriguez, M. C. (2005) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2005.00006.x · Cited by 360 · see citing works
Improving the fairness of multiple-choice questions: a literature review
McCoubrie, P. (2004) · Medical Teacher · doi:10.1080/01421590400013495 · Cited by 318 · see citing works
Setting standards on educational tests
Norcini, J. J. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01495.x · Cited by 315 · see citing works
Different written assessment methods: what can be said about their strengths and weaknesses?
Schuwirth, L. W. T., & Van Der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01916.x · Cited by 309 · see citing works
Tarrant, M., Ware, J. S., & Mohammed, A. M. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-40 · Cited by 238 · see citing works
Validity of a Taxonomy of Multiple-Choice Item-Writing Rules
Haladyna, T. M., & Downing, S. M. (1989) · Applied Measurement in Education · doi:10.1207/s15324818ame0201_4 · Cited by 217 · see citing works
How Many Options is Enough for a Multiple-Choice Test Item?
Haladyna, T. M., & Downing, S. M. (1993) · Educational and Psychological Measurement · doi:10.1177/0013164493053004013 · Cited by 196 · see citing works
Kumar, P., Gajjar, S., Sharma, R., & Rana, M. (2014) · Indian Journal of Community Medicine · doi:10.4103/0970-0218.126347 · Cited by 140 · see citing works
Further Investigation of Nonfunctioning Options in Multiple-Choice Test Items
Cizek, G. J., & O’Day, D. M. (1994) · Educational and Psychological Measurement · doi:10.1177/0013164494054004002 · Cited by 60 · see citing works
Assessment of Knowledge with Written Test Forms
Downing, S. M. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_25 · Cited by 59 · see citing works
THREE‐ALTERNATIVE MULTIPLE CHOICE TESTS: AN ATTRACTIVE OPTION
SIDICK, J. T., Barrett, G. V., & Doverspike, D. D. (1994) · Personnel Psychology · doi:10.1111/j.1744-6570.1994.tb01579.x · Cited by 53 · see citing works
The Effects of the Number of Options per Item and Student Ability on Test Validity and Reliability
Trevisan, M. S., Sax, G., & Michael, W. B. (1991) · Educational and Psychological Measurement · doi:10.1177/001316449105100404 · Cited by 49 · see citing works
Use of an Inclusive Option and the Optimal Number of Options for Multiple-Choice Items
Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993) · Educational and Psychological Measurement · doi:10.1177/0013164493053001027 · Cited by 48 · see citing works
Farley, J. K. (1989) · Nurse Educator · doi:10.1097/00006223-198911000-00003 · Cited by 33 · see citing works
Flagged first, then by citations
2 of the 19 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Hassan S, Mohd Amin R, Mohd Amin Rebuan H, Aung MMT. Item analysis, reliability statistics and standard error of measurement to improve the quality and impact of multiple choice questions in undergraduate medical education in faculty of medicine at UniSZA. Malaysian Journal of Public Health Medicine. 2016;16(3):7- 15.
- Ebel RL, Frisbie DA. Essentials of educational measurements. 5th ed. Englewood Clipps, NJ: Prentice Hall; 1991.
- Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non- functioning distractors in multiple choice questions: a descriptive analysis. BMC Medical Education. 2009;9(40):2-8. https:// doi.org/10.1186/1472-6920-9-40. · DOI
- Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distracter efficiency. J Pak Med Assoc. 2012;62:142-7.
Cited by
Sorted by citations
AI in medical education: uses of AI in construction type A MCQs
Rezigalla, A. A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05250-3 · Cited by 36 · see citing works
Item Analysis: Concept and Application
Rezigalla, A. A. (2022) · IntechOpen eBooks · doi:10.5772/intechopen.100138 · Cited by 18 · see citing works
Medical Education for the 21st Century
Firstenberg, M. S., & Stawicki, S. P. (2021) · IntechOpen eBooks · doi:10.5772/intechopen.95701 · Cited by 12 · see citing works
Šmida, D., Čipková, E., & Fuchs, M. (2023) · International Journal of Science Education · doi:10.1080/09500693.2023.2219811 · Cited by 11 · see citing works
Eleragi, A. M. S., Miskeen, E., Hussein, K., Rezigalla, A. A., Adam, M. I. E., Alfaifi, J. A., Alhalafi, A. H., Ameer, A., & Mohammed, O. A. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06700-2 · Cited by 8 · see citing works
Balaha, M. H. (2019) · Education in Medicine Journal · doi:10.21315/eimj2019.11.4.1 · Cited by 6 · see citing works
Chin, H., Meng, C. C., Yew, W. T., & Musa, M. (2022) · Participatory Educational Research · doi:10.17275/per.22.147.9.6 · Cited by 6 · see citing works
Shakurnia, A., Ghafourian, M., Khodadadi, A., Ghadiri, A., Amari, A., & Sharifat, M. (2022) · Education in Medicine Journal · doi:10.21315/eimj2022.14.4.5 · Cited by 5 · see citing works
Item Analysis in Multiple Choice Questions: A Study on Question Difficulty and Authors’ Evaluation
Konakcı, S. (2024) · Journal of Basic and Clinical Health Sciences · doi:10.30621/jbachs.1427464 · Cited by 4 · see citing works
Úroveň bádateľských zručností žiakov 6. ročníka základných škôl
Čipková, E., Fuchs, M., & Šmida, D. (2023) · Scientia in educatione · doi:10.14712/18047106.2276 · Cited by 3 · see citing works
Shammas, M., Nagda, S. J., Shah, C. J., Baxi, G. D., Gadde, P., Sachdeva, S., Gupta, D., Wali, O., Dhall, R. S., & Gajdhar, S. (2024) · Journal of Dental Education · doi:10.1002/jdd.13462 · Cited by 2 · see citing works
Terzi̇, B., & Ilgaz, A. (2024) · Journal of Nursology · doi:10.17049/jnursology.1420976 · Cited by 2 · see citing works
Pre and Post examination evaluation of MCQs and SEQs Items in Undergraduate MBBS mock examination.
Jaleel, A., & Khanum, Z. (2020) · The Professional Medical Journal · doi:10.29309/tpmj/2020.27.12.4474 · Cited by 2 · see citing works
Adam, S. K., Idris, F., Kassim, P. S. J., Zakaria, N. F., & Hod, R. (2021) · Journal of Medical Education · doi:10.5812/jme.116834 · Cited by 1 · see citing works
Self-citationUduafemhe, M. E., Danladi, U., John, S. O., & Karfe, R. Y. (2021) · Universal Journal of Educational Research · doi:10.13189/ujer.2021.090420 · Cited by 1 · see citing works
Adam, S. K., Idris, F., Kassim, P. S. J., Zakaria, N. F., & Hod, R. (2020) · doi:10.21203/rs.3.rs-38264/v1
Self-citationBalagueman, R. O., Assèdé, E. S. P., Moumouni-Moussa, I., AYEGNON, D. T. D., & Biaou, S. S. H. (2023) · Research Square · doi:10.21203/rs.3.rs-3360144/v1
Muntholib, M., Maudrey, L. I. A., Munzil, M., Muchson, M., & Setiawan, N. C. E. (2026) · E3S Web of Conferences · doi:10.1051/e3sconf/202669505005
BAKAR, ‘. M. A., Husain, N. R. N., Ismail, A. F., Kueh, Y. C., Muhamad, R., Naim, F., Zailani, N. A. A. C. M., Ismail, A. S., & AZNAM, F. A. (2026) · PeerJ · doi:10.7717/peerj.21271
Tanaka, Y., Yokoyama, A., KAMENO, J., & Tadaka, E. (2026) · Japan Journal of Nursing Science · doi:10.1111/jjns.70060
Zhu, S.-R., Yusoff, H., Cheng, K. Y., Yusoff, D. M., Feng, X., Zheng, W., & Chen, H. (2026) · Nursing Open · doi:10.1002/nop2.70925
Salahuddin, M., & Stupnisky, R. H. (2026) · Higher Education Quarterly · doi:10.1111/hequ.70179
- Authors
- Centre for Education (ICE), F. O. M. A. H. S., Hassan, S., Hod, R., & Faculty of Medicine and Health Sciences, U. P. M.
- Affiliations
- IMU University (MY)
- Universiti Putra Malaysia (MY)
- Corresponding author
- Hassan, Shahid
- Venue
- Education in Medicine Journal
- Publisher
- Universiti Sains Malaysia
- Published
- 2017-09-30
- Volume
- 9
- Issue
- 3
- Pages
- 33-43
- ISSN
- 2180-1932 (ISSN-L)
- OpenAlex
- W4239464535
- Cited by
- 23
- References
- 19
- Citations by year
- 2026: 5 · 2025: 1 · 2024: 4 · 2023: 3 · 2022: 3 · 2021: 3 · 2020: 2 · 2019: 2
- FWCI
- 1.02
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Education in Medicine Journal (publishedVersion)
- Universiti Putra Malaysia Institutional Repository (Universiti Putra Malaysia) (acceptedVersion)
- Education in Medicine Journal
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Educational Assessment and Pedagogy
- Keywords
- multiple-choice questions, item analysis, distractor efficiency, item difficulty, item discrimination, summative assessment, undergraduate medical students, preclinical teaching, psychometric assessment, internal consistency, item-total correlation, professional examinations, item difficulty distribution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
