CoChat
Sign Up
Cited by 4Artificial Intelligence in EducationTopic ModelingOnline Learning and Analytics

Generative AI-Powered Educational Alignment: A Framework for Matching Syllabus Course Topics with Web Description

Open access
Liu, Liyuan, Mendoza, Ruben A., Martin, Thomas R., Miori, Virginia M.·2024·Proceedings of the 2024 9th International Conference on Distance Education and Learning
DOI 10.1145/3675812.3675874
Open this paper in CoChat — free account

Abstract

The application of generative artificial intelligence (GAI) in the educational sector is increasingly gaining attention from researchers. This study explores the congruence between online course descriptions and actual course syllabi to improve course preparation and consistency for instructors. Alignment between course catalog descriptions and actual course content as detailed in the syllabus can lead to learning improvements, student satisfaction, and academic alignment in a program. Our research introduces a novel framework utilizing GAI to systematically evaluates and identifies mismatches and suggests content to close the gap between online course descriptions and syllabus content. We used OpenAI’s ChatGPT to extract key topics from course syllabi and assessed the congruence between results and course description content with embedding methods such as BERT, GPT-2, RoBERTa, and DistilBERT, coupled with cosine similarity metrics. Our framework also integrates an outlier detection algorithm to identify courses with significant misalignments and use GAI applications to refine and enhance course catalog descriptions. This approach helps higher education institutions update course offerings with cutting-edge technology and contributes to curriculum development, helping improve student learning efficiency and course design.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes a generative AI framework that extracts syllabus topics with ChatGPT, embeds them and online course descriptions with BERT, GPT-2, RoBERTa, and DistilBERT, and uses cosine similarity plus IQR outlier detection to find and revise misalignments. In a single business-school department case study of 413 syllabi and 67 course descriptions, it identified six misaligned courses and reported higher similarity scores after ChatGPT refinement.

Key findings

  • The framework identified 6 courses whose course descriptions were misaligned with syllabus topics; after ChatGPT revised those descriptions, similarity scores increased, especially with GPT and RoBERTa embeddings, with GPT-2 showing significant enhancement for the Advanced Business Analytics example.
  • Across embedding models, RoBERTa showed the highest average similarity; GPT-2 had the least standard deviation, indicating the most consistent performance; BERT had lower scores and a larger standard deviation.
  • IQR-based lower-outlier detection flagged GPT-2 and RoBERTa as identifying a higher number of misaligned courses, with GPT-2 noted for precise embedding despite a smaller standard deviation.
  • The case dataset included 413 syllabi from 67 different courses over 12 semesters (four full academic years), at both undergraduate and graduate levels, plus online descriptions for 67 courses.

Methods

  • Five-stage framework: data aggregation (syllabi plus web crawling with Python/Beautiful Soup), data extraction (ChatGPT/OpenAI API extracts course topics), data embedding (BERT, GPT-2, RoBERTa, DistilBERT), similarity and outlier detection (cosine similarity from 0 to 1; IQR method with k typically 1.5 or 3), and course description refinement (ChatGPT revises and reassesses descriptions).
  • Case study in one department of a business school at a comprehensive liberal arts university in Pennsylvania; 413 syllabi from 67 courses across 12 semesters, and 67 online course descriptions, after cleaning out unrelated template material and prerequisites.
  • Analysis compared model average similarity scores and standard deviations, applied IQR lower-outlier detection, and compared pre- and post-refinement similarity scores.

Limitations

  • Stated: extracted syllabus topics may not include all topics covered by instructors; outlier detection is based solely on data from one department, making outlier cutoffs specific to that dataset; processing a much larger dataset would require greater computational resources.
  • Evident: the single-department, single-business-school case limits generalizability; the authors state they plan to refine the detection algorithm and establish a similarity cutoff, so no final cutoff is reported; manual confirmation that ChatGPT-extracted topics matched syllabi is described but not quantified.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3675812.3675874

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 6 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 1 INTRODUCTION · p. 1

    Work at Aalto University's International Design Business Management program reveals the difficulty of aligning curriculum design with learning outcomes, due to different interpretations of course coverage by faculty and students.

    “Work at Aalto University's International Design Business Management program reveals the difficulty of aligning curriculum design with learning outcomes, due to different interpretations of course coverage by faculty and students [2, 48].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [2] [2] Jia Ying Chew. 2021. Siloed in breaking silos: A case study of interdisciplinary curriculum (mis) alignment. (2021). - source
    • cites [48] [48] Shasha Zhao. 2016. The problem of constructive misalignment in international business education: A three-stage integrated approach to enhancing teaching and learning. Journal of Teaching in International Business 27, 4 (2016), 179-196. - source
  • Findingcitation
    § 4.2 Experiment Results and Discussion · p. 4

    Employing BERT, GPT-2, RoBERTa, and DistilBERT achieved robust results.

    “Employing BERT, GPT-2, RoBERTa, and DistilBERT, we achieved robust results; BERT for deep contextual insights, GPT-2 for coherent text generation, RoBERTa for performance with large data, and DistilBERT for fast, cost-effective efficiency [39].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [39] [39] Thomas Wolf and Julien Chaumond. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Computational Linguistics, 38-45. https://www.aclweb.org/anthology/2020.emnlpdemos.6 - source

      The Library Design section (page not provided) lists BERT and RoBERTa as masked models, GPT-2 as autoregressive, and DistilBERT as efficient; the References also describe DistilBERT as faster and cheaper. These passages support several model-role descriptions, but do not establish RoBERTa’s performance with large data or GPT-2’s generation of coherent text. The citing authors’ statement that they achieved robust results is their own finding, not a result attributed to this source. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingown data
    § 4.2 Experiment Results and Discussion · p. 4

    GPT-2 had the least standard deviation in similarity scores among the embedding models tested.

    “GPT-2 demonstrated remarkable consistency with the least standard deviation, indicating stable performance ideal for analyses needing uniformity.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § 4.2 Experiment Results and Discussion · p. 5

    GPT-2 and RoBERTa flagged a higher number of misaligned courses than other models in the IQR outlier detection.

    “The boxplot reveals that GPT-2 and RoBERTa flagged a higher number of misaligned courses, with GPT-2 standing out due to its precise embedding technique despite a smaller standard deviation, indicating its nuanced detection capability.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § 4.2 Experiment Results and Discussion · p. 4

    RoBERTa achieved the highest average cosine similarity between syllabus topics and course descriptions among the embedding models tested.

    “RoBERTa showed the highest average similarity, suggesting it better captures the commonality between topics and descriptions.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 18
  • May combine two claims. 14

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 2.1 Role and Impact of Course Descriptions in Education · p. 2

    The importance of coherence, or alignment, between clear learning objectives, and teaching and assessment processes in learning outcomes is critical.

    “The importance of coherence, or alignment, between clear learning objectives, and teaching and assessment processes in learning outcomes is critical [34].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [34] [34] Mohammed A Shallal. 2018. Alignment of curriculum elements at the course level: Two useful tools. Academic Journal of Nawroz University 7, 3 (2018), 85-92. - source

  • May combine two claims.

    Findingown data
    § 4.2 Experiment Results and Discussion · p. 6

    After refining course descriptions with ChatGPT, similarity scores increased, especially with GPT and RoBERTa embeddings.

    “After refining course descriptions, we saw increased similarity scores, especially with GPT and RoBERTa embeddings.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Experiment Results and Discussion · p. 6

    For the Advanced Business Analytics course, the refined course description showed improved similarity, with GPT-2 showing significant enhancement.

    “A comparison for the Advanced Business Analytics course, shown in Figure 5, highlights the improvement, with GPT2 showing significant enhancement due to its advanced text representation abilities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 2.1 Role and Impact of Course Descriptions in Education · p. 2

    This approach enhances the precision of course material alignment and introduces a new AI application in education.

    “This approach enhances the precision of course material alignment and introduces a new AI application in education.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 1

    Consistency in course descriptions increases curricular alignment with program or departmental objectives and enables better content development and strategic planning.

    “Consistency in course descriptions increases curricular alignment with program or departmental objectives and enables better content development and strategic planning [31].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] [31] Josipa Roksa, Teniell L Trolian, Charles Blaich, and Kathleen Wise. 2017. Facilitating academic performance in college: understanding the role of clear and organized instruction. Higher Education 74 (2017), 283-300. - source

  • May combine two claims.

    Interpretationreasoning
    § ACMReference Format: · p. 2

    The framework integrates topic extraction from syllabi using ChatGPT and transformation of text into numerical data with generative AI models, effectively identifying misalignment of course descriptions with syllabi.

    “Integrates topic extraction from syllabi using ChatGPT and transformation of text into numerical data with generative AI models, effectively identifying misalignment of course descriptions with syllabi.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 3 INTRODUCTION TO THE FRAMEWORK · p. 3

    The proposed framework is designed to validate and refine the alignment between online course descriptions and syllabus content.

    “Our proposed framework is designed to validate and refine the alignment between online course descriptions and syllabus content.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § ACMReference Format: · p. 2

    The framework employs similarity calculations and outlier detection for evaluating congruence between course topics and web descriptions to improve alignment.

    “Employs similarity calculations and outlier detection for evaluating congruence between course topics and web descriptions to improve alignment”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning
    § ACMReference Format: · p. 2

    The framework demonstrates practical applications, simplifying course content alignment with online descriptions, thereby enhancing educational effectiveness for educators and students.

    “Demonstrates practical applications of the framework we proposed, simplifying course content alignment with online descriptions, thereby enhancing educational effectiveness for educators and students.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning
    § ACMReference Format: · p. 2

    This method saves time by allowing a clearer, more precise, and consistent articulation of online course descriptions.

    “This method saves time by allowing a clearer, more precise, and consistent articulation of online course descriptions.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § ACMReference Format: · p. 2

    The framework pioneers the use of Generative AI, specifically ChatGPT and pre-trained models, to align online course descriptions with syllabus course topics, narrowing the gap in online course representation.

    “Pioneers the use of Generative AI, specifically ChatGPT and pre-trained models, to align online course descriptions with syllabus course topics, narrowing the gap in online course representation.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: ChatGPT can extract key topics from course syllabi.
    • Rests on an assumption: Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.
    • Rests on an assumption: Outlier detection via IQR can identify courses with significant misalignment.
    • Rests on an assumption: Generative AI can refine course descriptions to improve alignment.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.2 Experiment Results and Discussion · p. 6

    GPT-2's significant enhancement in similarity for the Advanced Business Analytics course is due to its advanced text representation abilities.

    “A comparison for the Advanced Business Analytics course, shown in Figure 5, highlights the improvement, with GPT2 showing significant enhancement due to its advanced text representation abilities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5 CONCLUSION · p. 6

    Processing a much larger dataset would require greater computational resources.

    “Processing a much larger dataset would require greater computational resources (processing, network, storage, etc.).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Experiment Results and Discussion · p. 4

    GPT-2's similarity assessments are stable and ideal for analyses needing uniformity.

    “GPT-2 demonstrated remarkable consistency with the least standard deviation, indicating stable performance ideal for analyses needing uniformity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 5 CONCLUSION · p. 6

    The course topics extracted from syllabi may not include all topics covered by instructors.

    “We used course topics extracted from syllabi, which may not include all topics covered by instructors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § 5 CONCLUSION · p. 6

    The outlier detection and its cutoffs are based solely on data from one department and are specific to this dataset.

    “Additionally, our outlier detection is based solely on data from one department, making the outlier cutoffs specific to our dataset.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5 CONCLUSION · p. 6

    The GAI framework aids students in understanding course content and making informed choices.

    “Our experiments show that this approach not only improves the alignment between course content and descriptions, as reflected in higher similarity scores but also aids students in understanding course content and making informed choices.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data
    § 5 CONCLUSION · p. 6

    The GAI framework offers a practical solution for improving curriculum development, alignment, and consistency.

    “Our framework offers a practical solution for improving curriculum development, alignment, and consistency.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 4.2 Experiment Results and Discussion · p. 4

    GPT-2's unsupervised learning is key for generating new, contextually relevant text.

    “GPT-2's unsupervised learning is key for generating new, contextually relevant text.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Experiment Results and Discussion · p. 5

    GPT-2 has a precise embedding technique and nuanced detection capability for misalignments.

    “The boxplot reveals that GPT-2 and RoBERTa flagged a higher number of misaligned courses, with GPT-2 standing out due to its precise embedding technique despite a smaller standard deviation, indicating its nuanced detection capability.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 4.2 Experiment Results and Discussion · p. 4

    DistilBERT's efficiency suits limited-resource or quick-deployment needs.

    “DistilBERT's efficiency suits limited-resource or quick-deployment needs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 4.2 Experiment Results and Discussion · p. 4

    BERT excels in contextual understanding due to its bidirectional training, making it ideal for nuanced tasks.

    “The embedding model choice significantly impacts task outcomes, with BERT excelling in contextual understanding due to its bidirectional training, making it ideal for nuanced tasks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 4.2 Experiment Results and Discussion · p. 4

    RoBERTa is the go-to model for large-scale tasks with ample computational resources, offering enhanced tuning.

    “RoBERTa is the go-to for large-scale tasks with ample computational resources, offering enhanced tuning.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 5 CONCLUSION · p. 6

    The GAI framework supports instructors in course preparation.

    “This method also supports instructors in course preparation and enhances university branding with clearer, more accurate course information.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data
    § 5 CONCLUSION · p. 6

    The integration of AI tools provides a significant opportunity for academic institutions to enhance course descriptions and align them more closely with actual syllabus content, meeting accreditation standards and documenting continuous course refinement.

    “The integration of AI tools provides a significant opportunity for academic institutions to enhance course descriptions and align them more closely with actual syllabus content, meeting accreditation standards and documenting continuous course refinement.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 5 CONCLUSION · p. 6

    The GAI framework enhances university branding with clearer, more accurate course information.

    “This method also supports instructors in course preparation and enhances university branding with clearer, more accurate course information.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.
26 of 26

  • The paper relies on this without support.

    ChatGPT can extract key topics from course syllabi.

    “This is achieved using OpenAI's capabilities, where tools like ChatGPT or the OpenAI API play a pivotal role in isolating course topics from syllabi.”

  • The paper relies on this without support.

    Transformer-based embedding models with cosine similarity can measure congruence between course topics and web descriptions.

    “Post-embedding, the course descriptions, and topics, now in vector form, are analyzed for congruence using cosine similarity measures.”

  • The paper relies on this without support.

    Outlier detection via IQR can identify courses with significant misalignment.

    “Extreme discrepancies, indicative of misalignment, are identified via outlier detection algorithms.”

  • The paper relies on this without support.

    Generative AI can refine course descriptions to improve alignment.

    “OpenAI tools help edit course descriptions with significant misalignment with syllabi. The revised content is reevaluated to verify improved alignment.”

  • The paper relies on this without support.

    Alignment between course catalog descriptions and actual course content as detailed in the syllabus can lead to learning improvements, student satisfaction, and academic alignment in a program.

    “Alignment between course catalog descriptions and actual course content as detailed in the syllabus can lead to learning improvements, student satisfaction, and academic alignment in a program.”

  • The paper relies on this without support.

    Clear course descriptions help align curricular goals, improve teaching methods, and influence enrollment, playing a strategic role in an increasingly competitive higher education landscape.

    “Clear course descriptions help align curricular goals, improve teaching methods, and influence enrollment, playing a strategic role in an increasingly competitive higher education landscape.”

  • The paper relies on this without support.

    Aligning course descriptions with syllabi improves students' preparedness and engagement and ensures a cohesive teaching and learning experience, effectively aligning course coverage with curriculum design.

    “Aligning course descriptions with syllabi not only improves students' preparedness and engagement but also ensures a cohesive teaching and learning experience, effectively aligning course coverage with curriculum design.”

  • The paper relies on this without support.

    AI tools have significantly improved the ability to identify misalignments between syllabi, textbooks, and student reviews.

    “AI tools have significantly improved the ability to identify misalignments between syllabi, textbooks, and student reviews.”

  • The paper relies on this without support.

    ChatGPT permits the analysis of complex language tasks, including syllabus and course content analysis through the use of transformer architecture.

    “ChatGPT permits the analysis of complex language tasks, including syllabus and course content analysis through the use of transformer architecture.”

  • The paper relies on this without support.

    Challenges like uneven/inconsistent course syllabi between instructors of a course, and the need for standardized topics in curriculum design remain a challenge that cannot be overcome with these tools alone.

    “Challenges like uneven/inconsistent course syllabi between instructors of a course, and the need for standardized topics in curriculum design remain a challenge that cannot be overcome with these tools alone.”

  • The paper relies on this without support.

    Cosine similarity between embeddings of syllabus topics and course descriptions is a valid measure of their alignment.

    “We measured the alignment between syllabi topics and course descriptions using cosine similarity, where values range from 0 (no similarity) to 1 (perfect similarity).”

  • The paper relies on this without support.

    Course descriptions and syllabus topics for the same course are expected to align.

    “To assess alignment, we applied the IQR method, focusing on lower outliers to spot misalignments between course descriptions and syllabi, as depicted in Figure 4.”

  • The paper relies on this without support.

    ChatGPT-generated revisions to course descriptions preserve or improve their accuracy while aligning them with syllabus topics.

    “We used ChatGPT to revise these course descriptions using their syllabus topics.”
13 of 13

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Frontiers in Education. 2023. Improving Undergraduates’ and Postgraduates’ Academic Writing Skills with Strategy Training and Feedback. Frontiers in Education (2023). https://www.frontiersin.org/ Accessed: 2023-01-16.
  2. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  3. Anthony Wotring, Honglin Chen, and Mark Fraser. 2021. Exploring Curriculum Alignment through Syllabus Document Analysis: From National Language Policy to Local ELT Practice.Iranian Journal of Language Teaching Research 9, 2 (2021), 57–72.

Cited by

Sorted by citations

Related

Authors
Liu, L., Mendoza, R. A., Martin, T. R., & Miori, V. M.
Affiliations
  • Saint Joseph's University (US)
  • Department of Decision and System Sciences, Saint Joseph's University, USA
ORCID
Venue
Proceedings of the 2024 9th International Conference on Distance Education and Learning
Publisher
ACM
Published
2024-06-14
Pages
340-346
DOI
10.1145/3675812.3675874
OpenAlex
W4401408996
Semantic Scholar
f293a33efe50ad70636f4ff547c738f9e3d397dc
Cited by
4
References
19
Citations by year
2026: 4
FWCI
1.54
Type
conference paper
Language
en
Open access
Licence
Locations
International Conference on Distance Education and Learning (ICDEL) (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
generative AI, course syllabus, course descriptions, curriculum alignment, ChatGPT, topic extraction, text embeddings, cosine similarity, outlier detection, curriculum development, higher education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-25.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-25.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review