Automatic Classification of Learning Objectives Based on Bloom's Taxonomy
Open accessThe DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index; none of the 51 sources its claims cite is checked for support yet.
Abstract
Learning objectives, especially those well defined by applying Bloom's taxonomy for Cognitive Objectives, have been widely recognized as important in various teaching and learning practices. However, many educators have difficulties developing learning objectives appropriate to the levels in Bloom's taxonomy, as they need to consider the progression of learners' skills with learning content as well as dependencies between different learning objectives. To remedy this challenge, we aimed to apply state-of-the-art computational techniques to automate the classification of learning objectives based on Bloom's taxonomy. Specifically, we collected 21,380 learning objectives from 5,558 different courses at an Australian university and manually labeled them according to the six cognitive levels of Bloom's taxonomy. Based on the labeled dataset, we applied five conventional machine learning approaches (i.e., naive Bayes, logistic regression, support vector machine, random forest, and XGBoost) and one deep learning approach based on pre-trained language model BERT to construct classifiers to automatically determine a learning objective's cognitive levels. In particular, we adopted and compared two methods in constructing the classifiers, i.e., constructing multiple binary classifiers (one for each cognitive level in Bloom's taxonomy) and constructing only one multi-class multi-label classifier to simultaneously identify all the corresponding cognitive levels. Through extensive evaluations, we demonstrated that: (i) BERT-based classifiers outperformed the others in all cognitive levels (Cohen's Kappa up to 0.93 and F1 score up to 0.95); (ii) three machine learning models – support vector machine, random forest, and XGBoost — delivered performance comparable to the BERT-based classifiers; and (iii) most of the binary BERT-based classifiers (5 out of 6) slightly outperformed the multi-class multi-label BERT-based classifier, suggesting that separating the characterization of different cognitive levels seemed to be a better choice than building only one model to identify all cognitive levels at one time.
Overview
CoChat Summary
## TL;DR
The paper builds machine learning and BERT-based classifiers to automatically assign Bloom’s taxonomy cognitive levels to learning objectives using 21,380 objectives from 5,558 courses at an Australian university. It finds that BERT, SVM, random forest, and XGBoost perform strongly, with the best models reaching Cohen’s κ up to 0.93 and F1 up to 0.95.
## Key findings
- The dataset contained 21,380 learning objectives from 5,558 courses across 10 faculties; some objectives had multiple labels, including 2,325 with two cognitive levels, 280 with three, and 2 with four.
- High-performing models included SVM, random forest, XGBoost, and BERT; across these models, Cohen’s κ ranged from 0.79 to 0.93, accuracy from 0.92 to 0.99, and F1 from 0.83 to 0.95.
- The paper reports that binary BERT models outperformed all binary machine learning models and achieved Cohen’s κ of 0.87 to 0.93, accuracy of 0.96 to 0.99, and F1 of 0.88 to 0.95.
- In the comparison of binary vs. multi-class multi-label approaches, binary classifiers generally performed better; for example, the paper notes binary BERT outperformed multi-class multi-label BERT in most cases, with evaluate as the main exception.
## Methods
- The study used 21,380 publicly available learning objectives scraped from course web pages at an Australian university in 2021, then manually labeled them with Bloom’s taxonomy cognitive levels by a trained human coder; a second coder independently checked 30% of the labels.
- The authors trained six model families: five traditional classifiers (SVM, logistic regression, naive Bayes, random forest, XGBoost) and one deep learning approach (BERT), implemented both as binary classifiers and as multi-class multi-label classifiers where appropriate.
- Data were split 80:20 into training and test sets; traditional models used 3-fold cross-validation with grid search for tuning, and evaluation used accuracy, Cohen’s κ, AUC, and F1.
## Limitations
- The paper states that some learning objectives might belong to other domains, such as the affective domain, rather than Bloom’s cognitive domain; it suggests future datasets should be labeled for other domains as well.
- The paper states that the supervised machine learning and deep learning methods require extensive labeled data, and that preparing such a dataset is costly and time-consuming.
- Evident from the text: the data come from one Australian university and were labeled by coders trained on Bloom’s taxonomy, so generalizability to other institutions or labeling practices is not tested here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 34 Not checked · 40 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we collected 21,380 learning objectives from 5,558 different courses at an Australian university and manually labeled them according to the six cognitive levels of Bloom's taxonomy.”
- Also stated in: § 3. METHODOLOGY 3.1 Data Collection and Labeling · plain
“We collected 21,380 learning objectives publicly available from 5,558 courses provided by the 10 faculties at an Australian university in 2021.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3. METHODOLOGY 3.1 Data Collection and Labeling · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“(ii) three machine learning models – support vector machine, random forest, and XGBoost — delivered performance comparable to the BERT-based classifiers;”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the major source of disagreement was because many learning objectives from the low-cognition category remember were wrongly categorized as the high-cognition category apply.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“but rarely outperformed binary BERT-based classifiers in terms of the prediction performance.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the multi-class multi-label BERT-based classifier performed better than all the binary and multiclass multi-label machine learning models from the same cognitive level”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
33 more claims
Create a free account to see all 76 claims and to verify the 51 references still open.
- Borderline: one of the tests for a substantial claim is unclear. 1
- It may state two separate claims as one. 12
- Probably not a substantial claim. 32
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 29
- [2][2] H. L. Andrade. A critical review of research on student self-assessment. In Frontiers in Education, volume 4, page 87. Frontiers, 2019. -
- [4][4] B. S. Bloom et al. Taxonomy of educational objectives. vol. 1: Cognitive domain. New York: McKay, 20(24):1, 1956. -
- [5][5] W.-C. Chang and M.-S. Chung. Automatic applying bloom's taxonomy to classify and analysis the cognition level of english question items. In 2009 Joint Conferences on Pervasive Computing (JCPC), pages 727–734, 2009. -
- [6][6] G. Chen, J. Yang, C. Hauff, and G.-J. Houben. Learningq: a large-scale dataset for educational question generation. In Twelfth International AAAI Conference on Web and Social Media, 2018. -
- [7][7] J. Chen, J. Feng, X. Sun, and Y. Liu. Co-training semi-supervised deep learning for sentiment classification of mooc forum posts. Symmetry, 12(1):8, 2020. -
- [8][8] T. Chen and C. Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pages 785–794, New York, NY, USA, 2016. ACM. -
- [11][11] V. Echeverría, J. C. Gomez, and M.-F. Moens. Automatic labeling of forums using bloom's taxonomy. In International Conference on Advanced Data Mining and Applications, pages 517–528. Springer, 2013. -
- [13][13] R. Gluga, J. Kay, R. Lister, Simon, and S. Kleitman. Mastering cognitive development theory in computer science education. Computer Science Education, 23(1):24–57, 2013. -
- [14][14] S. X. Guo, X. Sun, S. X. Wang, Y. Gao, and J. Feng. Attention-based character-word hybrid neural networks with semantic and structural information for identifying of urgent posts in mooc discussion forums. IEEE Access, 7:120522–120532, 2019. -
- [15][15] K. Jayakodi, M. Bandara, I. Perera, and D. Meedeniya. Wordnet and cosine similarity based classifier of exam questions using bloom's taxonomy. International Journal of Emerging Technologies in Learning, 11(4), 2016. -
- [18][18] M. Laddha, V. Lokare, A. Kiwelekar, and L. Netak. Classifications of the summative assessment for revised bloom's taxonomy by using deep learning. International Journal of Engineering Trends and Technology, 69:211–218, 03 2021. -
- [19][19] M. B. Larson and B. B. Lockee. Streamlined ID: A practical guide to instructional design. Routledge, 2019. -
- [20][20] C. Leistner, A. Saffari, J. Santner, and H. Bischof. Semi-supervised random forests. In 2009 IEEE 12th international conference on computer vision, pages 506–513. IEEE, 2009. -
- [21][21] S. Masapanta-Carrión and J. Á. Velázquez-Iturbide. A systematic review of the use of bloom's taxonomy in computer science education. In Proceedings of the 49th acm technical symposium on computer science education, pages 441–446, 2018. -
- [22][22] M. L. McHugh. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282, 2012. -
- [23][23] A. Miaschi and F. Dell'Orletta. Contextual and non-contextual word embeddings: an in-depth linguistic investigation. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 110–119, Online, July 2020. Association for Computational Linguistics. -
- [24][24] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, 2013. -
- [25][25] M. Mohammed and N. Omar. Question classification based on bloom's taxonomy cognitive domain using modified tf-idf and word2vec. PloS one, 15(3):e0230442, 2020. -
- [26][26] R. W. Morshead. Taxonomy of educational objectives handbook ii: Affective domain. Studies in Philosophy and Education, 4(1):164–170, 1965. -
- [27][27] V. Nikolovski, D. Kitanovski, D. Trajanov, and I. Chorbev. Case study: Predicting students objectivity in self-evaluation responses using bert single-label and multi-label fine-tuned deep-learning models. In V. Dimitrova and I. Dimitrovski, editors, ICT Innovations 2020. Machine Learning and Applications, pages 98–110, Cham, 2020. Springer International Publishing. -
- [28][28] A. Osadi, N. Fernando, and V. Welgama. Ensemble classifier based approach for classification of examination questions into bloom's taxonomy cognitive levels. International Journal of Computer Applications, 162:975–8887, 04 2017. -
- [29][29] A. Osman and A. A. Yahya. Classifications of exam questions using natural language syntatic features: A case study based on bloom's taxonomy. In Proc. 6th Int. Arab Conf. Quality Assurance Higher Edu., 2016. -
- [30][30] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011. -
- [32][32] C. Sammut and G. I. Webb. Encyclopedia of machine learning. Springer Science & Business Media, 2011. -
- [36][36] S. Shaikh, S. M. Daudpotta, and A. S. Imran. Bloom's learning outcomes' automatic classification using lstm and pretrained word embeddings. IEEE Access, 9:117887–117909, 2021. -
- [37][37] S. Sharma and D. Mehrotra. Comparative analysis of multi-label classification algorithms. In 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), pages 35–38, 2018. -
- [39][39] T. Tang, X. Tang, and T. Yuan. Fine-tuning bert for multi-label sentiment analysis in unbalanced code-switching text. IEEE Access, 8:193248–193256, 2020. -
- [42][42] R. Yarullin and P. Serdyukov. Bert for sequence-to-sequence multi-label text classification. In W. M. P. van der Aalst, V. Batagelj, D. I. Ignatov, M. Khachay, O. Koltsova, A. Kutuzov, S. O. Kuznetsov, I. A. Lomazova, N. Loukachevitch, A. Napoli, A. Panchenko, P. M. Pardalos, M. Pelillo, A. V. Savchenko, and E. Tutubalina, editors, Analysis of Images, Social Networks and Texts, pages 187–198, Cham, 2021. Springer International Publishing. -
- [44][44] J. Zhang, C. Wong, N. Giacaman, and A. Luxton-Reilly. Automated classification of computing education questions using bloom's taxonomy. In Australasian Computing Education Conference, ACE '21, page 58–65, New York, NY, USA, 2021. Association for Computing Machinery.
Cited by
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Doughty, J., Wan, Z., Bompelli, A., Qayum, J., Wang, T., Zhang, J., Zheng, Y., Doyle, A., Sridhar, P., Agarwal, A., et al. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636256 · Cited by 132
Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives
Sridhar, P., Doyle, A., Agarwal, A., Bogart, C., Savelka, J., & Sakr, M. (2023) · arXiv · doi:10.48550/arxiv.2306.17459
Gani, M. O., Ayyasamy, R. K., Sangodiah, A., & Fui, Y. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11842-1 · Cited by 50
Assessing AI-Generated Questions’ Alignment with Cognitive Frameworks in Educational Assessment
Yaacoub, A., Da-Rugna, J., & Assaghir, Z. (2025) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2025.v17.1374 · Cited by 41
Leveraging generative AI for course learning outcome categorization using Bloom's taxonomy
Almatrafi, O., & Johri, A. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100404 · Cited by 55
Li, Y., Raković, M., Dai, W., Lin, J., Khosravi, H., Galbraith, K. J., Lyons, K., Gašević, D., & Chen, G. (2023) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2023.100157 · Cited by 30
Self-citationTarget hierarchy-guided knowledge tracing : Fine-grained knowledge state modeling
Sun, X., Zhang, K., Shen, S., Wang, F., Guo, Y., & Liu, Q. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123898 · Cited by 22
Enhancing Feedback Quality at Scale: Leveraging Machine Learning for Learner-Centered Feedback
Aldino, A. A., Tsai, Y., Mello, R. F., Gašević, D., & Chen, G. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100332 · Cited by 24
Self-citationAlammary, A. S., & Masoud, S. (2025) · Electronics · doi:10.3390/electronics14122312 · Cited by 14
Ülker, S., & Kocakülah, M. S. (2023) · Buca eğitim fakültesi dergisi (Online)/Buca Eğitim Fakültesi dergisi · doi:10.53444/deubefd.12est66 · Cited by 8
Curriculum analytics: Exploring assessment objectives, types, and grades in a study program
Jovanović, J., Zamecnik, A., Barthakur, A., & Dawson, S. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13015-0 · Cited by 5
Revolutionizing Academic Evaluation: Bloom's Taxonomy Meets Deep Learning and NLP
Gavhane, J. M., & Pagare, R. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016632 · Cited by 5
Open Datasets in Learning Analytics: Trends, Challenges, and Best PRACTICE
Švábenský, V., Flanagan, B., Zapata, E. D. L., & Shimada, A. (2026) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3798096 · Cited by 8
Pınar, F. N., & Ocak, G. (2025) · Anadolu Journal Of Educational Sciences International · doi:10.18039/ajesi.1565450 · Cited by 4
Chor, W. T., Goh, K. M., Lim, L. L., Lum, K. Y., & Chiew, T. H. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-12180-y · Cited by 5
Assessing ChatGPT’s Capability to Generate Course Learning Outcomes
Almatrafi, O. (2024) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/icict62343.2024.00092 · Cited by 3
Misiejuk, K., López‐Pernas, S., Kaliisa, R., & Saqr, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5145012 · Cited by 4
Leveraging NLP-based tools for constructive alignment
Kilinc, C., Ranaweera, C., Ugon, J., Cain, A., & Pierce, C. (2025) · ASCILITE Publications · doi:10.65106/apubs.2025.2636 · Cited by 2
Puraivan, E., Cofré-Morales, C., Rodríguez, M., Godoy, T. L., Tapia, J. C. C., Hervás-Gómez, C., & Noguera, M. D. D. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-82073-1_23 · Cited by 1
Exploring Students’ Cognitive Engagement with Generative Artificial Intelligence
Alamray, F., Aljohani, N. R., Alfakeeh, A. S., Alhebaishi, N., & Gašević, D. (2026) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3785022.3785132 · Cited by 6
On Cognitive Level Classification of Assessment-items Using Pre-trained BERT-based Model
Dipto, A. S., Limon, M. M. R., Tuba, F. T., Uddin, M. M., Khan, M., & Tuhin, R. A. (2023) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3639233.3639331 · Cited by 1
A Semi-supervised Model for Automated Classification of AI-related Job Tasks using Bloom’s Taxonomy
Fongang, C. M., Vogelsang, M., Eichenberg, T., Rüschoff, B., & Stockem, A. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00019 · Cited by 1
VTU Question Classification System based on BERT and Bloom’s Taxonomy Ring’
Punyatoya, S., Vidya, N., Pradhan, C., Thejaswini, S., Sahoo, A. K., Shaista Afrin, Z., & Sandhya, S. (2025) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic64052.2025.10962783 · Cited by 2
NQF: A deep learning model for classifying course outcomes in the national qualification framework
Elhassan, A., Azzeh, M., & El-Rashdan, B. (2025) · International Journal of Speech Technology · doi:10.1007/s10772-025-10200-5 · Cited by 1
CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing EducationPreprint
Alonso-Carracedo, M., Fernandez-Boullon, R., Celard, P., Martínez, F. J. R. , & Otero-Cerdeira, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.00140 · Cited by 1
Exploring and Criticizing Students’ Artwork as a Learning Media in General English Instruction
Sukarni, S., Siswana, S., & Jumadil, J. (2024) · Journal of Languages and Language Teaching · doi:10.33394/jollt.v12i4.11021
Incorporating Cognitive Complexity of Text in Dense Retrieval for Personalized Search
Sokli, E., Raganato, A., & Pasi, G. (2024) · BOA (University of Milano-Bicocca)
Fiallos, B. (2024) · Repositori ObertUDL (University of Lleida)
Automatic Classification of Questions according to Bloom Taxonomy
Maharramov, A. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Puraivan, E., Cofré-Morales, C., Godoy, T. L., Hervás-Gómez, C., Noguera, M. D. D., & Rodríguez, M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-12885-0_7
Extending the Cognitive Domain of Bloom’s Taxonomy using Machine LearningPreprint
Talha, M., Shi, J., & Qureshi, A. J. (2026) · Research Square · doi:10.21203/rs.3.rs-8704766/v1 · Cited by 2
Turaev, S., Al-Dabet, S., John, M., Awad, M., Zaki, N., & Shuaib, K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7458498
Metadata
- Authors
- Li, Y., Rakovic, M., Poh, B. X., Gasevic, D., & Chen, G.
- Venue
- Zenodo
- Publisher
- Zenodo
- Published
- 2022-07-18
- OpenAlex
- W4285778212
- Cited by
- 22
- References
- 0
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 6 · 2023: 5
- FWCI
- 37.42
- Type
- conference paper
- Language
- en
- Open access
- green
- research.monash.edu
- Licence
- Locations
- Monash University Research Portal (Monash University) (submittedVersion, other-oa)
- HAL (submittedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (cc-by)
- Topics
- Social Sciences › Education › Educational Assessment and Pedagogy
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- learning objectives, Bloom's taxonomy, cognitive levels, automatic classification, BERT, multi-label classification, binary classifiers, course syllabus analysis, educational data mining, machine learning classifier comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.