Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The expert panel size (N = 10) is consistent with established Delphi methodology in educational program development but remains modest.

“First, the expert panel size (N = 10), while consistent with established Delphi methodology in educational program development [24,33], remains modest”

• cites [24] Tang, 2011, Optimizing an immersion ESL curriculum using analytic hierarchy process, Eval. Program Plan. sourceNot checked — no full text of the source
• cites [33] Yao, Y., Cao, J., Tan, Z., and Chen, K. (2025). Developing a digital health competency assessment framework for public health services: A Delphi-AHP approach. Front. Public Health, 13. sourceDisputed

  The Methods > Expert selection passage says research suggests a minimum of 12 panellists for Delphi consensus, contrary to citing this paper as support for a panel of 10 being consistent with that methodology. The source used 15 experts; it does not establish a different standard for educational program development. No page number is available in our copy. (LLM, conf=0.89)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

May combine two claims.

University-level AI literacy goes beyond foundational AI knowledge typical of K-12 settings and requires deeper engagement with data and algorithms, problem-solving capacities, and career-relevant competencies.

“Chee, Ahn, and Lee [3], through a systematic review, demonstrated that university-level AI literacy goes beyond the foundational AI knowledge typical of K-12 settings and requires deeper engagement with data and algorithms, problem-solving capacities, and career-relevant competencies.”

• cites [3] Chee, 2025, A competency framework for AI literacy: Variations by different learner groups and an implied learning pathway, Br. J. Educ. Technol. sourceSupported

Classified as probably not a substantial claim.

AI literacy must encompass practical application and critical evaluation capacities, not merely declarative knowledge.

“Ng et al. [9] distinguished between knowing and applying, arguing that AI literacy must encompass practical application and critical evaluation capacities, not merely declarative knowledge.”

• cites [9] Ng, 2021, Conceptualizing AI literacy: An exploratory review, Comput. Educ. Artif. Intell. sourceSupported

May combine two claims.

More recent studies on curriculum-linked extracurricular programs have shown significant improvements in knowledge application, logical thinking, creative problem-solving, and ethical competencies.

“More recent studies on curriculum-linked extracurricular programs have shown significant improvements in knowledge application, logical thinking, creative problem-solving, and ethical competencies [16,17].”

• cites [16] Choi, 2020, The effect of co-curriculum on students’ major competency and satisfaction, J. Educ. Cult. sourceNot checked — no full text of the source
• cites [17] Yoon, 2020, Effects of curriculum-linked project-based extracurricular education based on industry-university cooperation: Focusing on D University, J. Educ. Innov. sourceCould not verify

  The available indexed copy contains a KCI bibliographic listing (no section or page), identifying a study of a curriculum-linked extracurricular program but reporting no outcomes. Searches for knowledge application, logical thinking, creative problem-solving, ethical competencies, and their Korean equivalents found no passage establishing significant improvements. (LLM, conf=0.72)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The number of Level 1 criteria (n = 6) remained within Saaty's recommended range (7 ± 2) for cognitively tractable pairwise comparison.

“The number of Level 1 criteria (n = 6) remained within Saaty's [23] recommended range (7 ± 2) for cognitively tractable pairwise comparison.”

• cites [23] Saaty, 1987, The analytic hierarchy process—What it is and how it is used, Math. Model. sourcePartly supported

  Section 4 (page unavailable) says the number of elements compared should be small and specifically fewer than 10, which includes six criteria. It does not recommend a 7 ± 2 range or give cognitive tractability as the reason; its stated concern is that errors can distort priorities when many elements are compared. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

AI literacy is a multidimensional competency extending beyond technical knowledge to encompass ethical judgment and creative application.

“The authors emphasized that AI literacy is a multidimensional competency extending beyond technical knowledge to encompass ethical judgment and creative application.”

• cites [9] Ng, 2021, Conceptualizing AI literacy: An exploratory review, Comput. Educ. Artif. Intell. sourceSupported

May combine two claims.

Panel composition adhered to Green's recommendation that Delphi panels comprise individuals with substantive expertise and to the suggested minimum panel size for educational Delphi studies (8-15 experts).

“Panel composition adhered to Green's [22] recommendation that Delphi panels comprise individuals with substantive expertise on the topic, and to the suggested minimum panel size for educational Delphi studies (8-15 experts) [24,33].”

• cites [22] Green, 2014, The Delphi technique in educational research, SAGE Open sourceNot checked — no full text of the source
• cites [24] Tang, 2011, Optimizing an immersion ESL curriculum using analytic hierarchy process, Eval. Program Plan. sourceNot checked — no full text of the source
• cites [33] Yao, Y., Cao, J., Tan, Z., and Chen, K. (2025). Developing a digital health competency assessment framework for public health services: A Delphi-AHP approach. Front. Public Health, 13. sourcePartly supported

  The Methods > Expert selection passage says panel size should reflect project scale and disciplinary coverage, cites a suggested minimum of 12 panellists, and reports that this study engaged 15 experts. It does not give an 8–15-expert recommendation specifically for educational Delphi studies. The expertise recommendation is attributed to Green [22], not to this source. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Bartkus et al. distinguished extra-curricular activities (those proceeding independently of academic coursework) from co-curricular activities (those intentionally aligned with the learning outcomes of formal curriculum).

“Bartkus, Nemelka, Nemelka, and Gardner [27], through a comprehensive literature review, distinguished extra-curricular activities (those proceeding independently of academic coursework) from co-curricular activities (those intentionally aligned with the learning outcomes of formal curriculum).”

• cites [27] Bartkus, 2012, Clarifying the meaning of extracurricular activity: A literature review of definitions, Am. J. Bus. Educ. sourcePartly supported

  In “An analysis of” (page not provided), Bartkus et al. distinguish the two categories and say co-curricular activities are aligned with curriculum objectives. Their proposed definition also requires participation to meet a curricular requirement; they note that a voluntary activity can be closely aligned with a student's major yet still have characteristics of an extracurricular activity. The citing sentence omits that requirement and makes alignment sound like the decisive distinction. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim. May combine two claims.

AHP decomposes complex problems into hierarchical structures of goal, criteria, and alternatives, and quantifies relative importance through pairwise comparisons.

“The Analytic Hierarchy Process (AHP), developed by Saaty [23], is a multi-criteria decision-making technique that decomposes complex problems into hierarchical structures of goal, criteria, and alternatives, and then quantifies relative importance through pairwise comparisons.”

• cites [23] Saaty, 1987, The analytic hierarchy process—What it is and how it is used, Math. Model. sourceSupported

Classified as probably not a substantial claim. May combine two claims.

AHP enables the conversion of qualitative expert judgments into numerical weights, with built-in consistency verification through the Consistency Ratio (CR).

“AHP enables the conversion of qualitative expert judgments into numerical weights, with built-in consistency verification through the Consistency Ratio (CR).”

• cites [23] Saaty, 1987, The analytic hierarchy process—What it is and how it is used, Math. Model. sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

AI literacy has been increasingly framed as a precondition for lifelong learning and adaptive citizenship in a rapidly evolving technological environment.

“From the standpoint of sustainable higher education, AI literacy has been increasingly framed as a precondition for lifelong learning and adaptive citizenship in a rapidly evolving technological environment [6,7,26].”

• cites [6] Cardona, M., Rodriguez, R.J., and Ishmael, K. (2023). Artificial Intelligence and the Future of Teaching and Learning.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [7] Knox, 2023, The political economy of education and the post-digital condition, Postdigit. Sci. Educ.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [26] Holmes, 2022, State of the art and practice in AI in education, Eur. J. Educ. sourcePartly supported

  The Introduction says AI literacy has received increasing attention in policy debates and describes a need for it in education. It does not frame AI literacy as a precondition for lifelong learning or adaptive citizenship. Section 3.1.11 discusses AI-assisted lifelong learning assistants, not AI literacy as a prerequisite for lifelong learning. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

May combine two claims.

The resulting six-criterion structure remained within Saaty's recommended range (7 ± 2), allowing cognitively tractable pairwise comparison while preserving full domain coverage.

“The resulting six-criterion structure remained within Saaty's [23] recommended range (7 ± 2), allowing cognitively tractable pairwise comparison while preserving full domain coverage.”

• cites Saaty's [23] Saaty, 1987, The analytic hierarchy process—What it is and how it is used, Math. Model. sourcePartly supported

  Section 4 (page unavailable) says the number of elements compared should be small, specifically fewer than 10, so six criteria are consistent with its guidance for pairwise comparison. The paper does not give a recommended range of 7 ± 2 or establish that the citing study’s six criteria preserve full domain coverage. Section 2 (page unavailable) instead advises making a hierarchy complex enough to capture the situation while keeping it small and nimble. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

The same expert panel participated in all three phases, introducing a potential circularity in which the panel that generated the framework also evaluated its validity.

“the same expert panel participated in all three phases of the study-Delphi consensus-building, AHP prioritization, and CVR verification.While this design choice preserved judgmental consistency across phases and reflects common practice in Delphi-AHP research [24,33], it introduces a potential circularity in which the panel that generated the framework also evaluated its validity.”

• cites [24] Tang, 2011, Optimizing an immersion ESL curriculum using analytic hierarchy process, Eval. Program Plan. sourceNot checked — no full text of the source
• cites [33] Yao, Y., Cao, J., Tan, Z., and Chen, K. (2025). Developing a digital health competency assessment framework for public health services: A Delphi-AHP approach. Front. Public Health, 13. sourcePartly supported

  The Methods sections on expert selection and AHP describe a 15-expert Delphi consultation whose ratings were used to construct the AHP judgment matrix. This supports the paper as an example of using expert input across Delphi and AHP, but our copy does not describe a third, CVR-verification phase or establish that the same panel participated in one. The potential circularity is the citing authors’ interpretation, not a finding attributed to this source. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

The final integrated framework comprised 24 consensus components organized into seven domains, plus the AHP priority architecture.

“The final integrated framework-comprising 24 consensus components organized into seven domains, plus the AHP priority architecture-was subjected to expert content validity verification using Lawshe's [35] CVR.”

• cites [35] Lawshe, 1975, A quantitative approach to content validity, Pers. Psychol. sourceNot checked — no full text of the source

May combine two claims.

Of 30 verification items organized into 10 domains, 22 items (73.3%) achieved CVR ≥ 0.62 in the initial verification, with an initial CVI of 0.71.

“Of the 30 verification items organized into 10 domains, 22 items (73.3%) achieved CVR ≥ 0.62 in the initial verification, with an initial CVI of 0.71.”

May combine two claims.

Weekly topic mapping between curriculum and extracurricular activities (M = 4.90) and AI tool practical application competency (M = 4.90) tied for the highest mean scores.

“Second, weekly topic mapping between curriculum and extracurricular activities (M = 4.90) and AI tool practical application competency (M = 4.90) tied for the highest mean scores, indicating that experts view curriculum-extracurricular alignment and applied competency as the twin pillars of program design.”

May combine two claims.

Round 2 evaluated all 33 candidate components against the three consensus criteria; 24 components met all three thresholds and were retained, while nine items were eliminated.

“Round 2 evaluated all 33 candidate components against the three consensus criteria (CVR ≥ 0.62, CV ≤ 0.5, consensus ≥ 0.75).Of these, 24 components met all three thresholds and were retained, while nine items were eliminated.”

The quote only partly states this claim.

The credit-bonus item was universally rejected (CVR = -1.00).

“the universal rejection of the credit-bonus item (CVR = -1.00)”

Probably not a substantial claim: may not pass S3 (load-bearing).

The final 24 components were organized into seven domains.

“The final 24 components were organized into seven domains, as shown in Table 1.”

May combine two claims.

AI tool workshops excel in curricular linkage (local weight 0.600) and mode of operation (0.500).

“AI tool workshops excel in curricular linkage (local weight 0.600) and mode of operation (0.500)”

May combine two claims.

Mode of operation (C2 = 0.112), assessment system (C5 = 0.088), and support system (C6 = 0.059) received progressively lower weights.

“Mode of operation (C2 = 0.112), assessment system (C5 = 0.088), and support system (C6 = 0.059) received progressively lower weights, suggesting that operational and infrastructural considerations-while consensually endorsed as essential components (Table 1)-are viewed by experts as secondary to the pedagogical core of practice integration, curriculum alignment, and learning objectives.”

May combine two claims.

AI practice integration (C4 = 0.286) and curricular linkage (C3 = 0.286) emerged jointly as the two highest-weighted criteria, sharing the top rank with identical weights, and jointly accounting for 57.2% of the total weight.

“AI practice integration (C4 = 0.286) and curricular linkage (C3 = 0.286) emerged jointly as the two highest-weighted criteria, sharing the top rank with identical weights, and jointly accounting for 57.2% of the total weight.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Items with lower frequency of mention (e.g., individual self-directed learning mode and course credit bonus for extracurricular participation) emerged as potentially contentious and were flagged for closer scrutiny in Round 2.

“Items with lower frequency of mention (e.g.,”

May combine two claims.

AI ethics/citizenship debate was eliminated as a stand-alone program type (CVR = 0.60) but retained as a learning objective (CVR = 0.80).

“Notably,”

May combine two claims.

Learning objectives (C1) had a weight of 0.169; mode of operation (C2) 0.112; assessment system (C5) 0.088; support system (C6) 0.059.

“The intermediate weight assigned to learning objectives (C1 = 0.169) and the lower weights assigned to mode of operation (C2 = 0.112), assessment system (C5 = 0.088), and support system (C6 = 0.059)”

May combine two claims.

AI tool workshops (A1) had a global weight of 0.414 and AI-based problem-solving challenges (A2) had a global weight of 0.390, a difference of 0.025.

“The narrow margin between the top two program types in the AHP results (AI tool workshops at 0.414 and AI-based problem-solving challenges at 0.390, ∆ = 0.025)”

May combine two claims.

AI practice integration (C4) and curricular linkage (C3) each received a weight of 0.286 and together accounted for 57.2% of the total criterion weight.

“These two criteria emerged jointly at the top of the priority ranking, each receiving a weight of 0.286 and together accounting for 57.2% of the total criterion weight.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

The item course credit bonus for extracurricular participation received a CVR of -1.00 (i.e., universal rejection).

“The item”

May combine two claims.

AI-based problem-solving challenges excel in AI practice integration (0.540), learning objectives (0.540), and assessment system (0.500).

“AI-based problem-solving challenges, by contrast, excel in AI practice integration (0.540), learning objectives (0.540), and assessment system (0.500)”

May combine two claims.

After revisions, all 30 items met the CVR threshold in a re-verification round, with CVI = 0.84, exceeding Davis's threshold of 0.80.

“After these revisions, all 30 items met the CVR threshold in a re-verification round (CVI = 0.84, exceeding Davis's [38] threshold of 0.80).”

• cites Davis's [38] Davis, 1992, Instrument review: Getting the most from a panel of experts, Appl. Nurs. Res. sourceNot checked — no full text of the source

May combine two claims.

Notable items achieving full consensus (mentioned by all 10 panel members) included AI tool practical application competency, generative AI tool utilization, and weekly curriculum-extracurricular topic mapping.

“Notable items achieving full consensus (mentioned by all 10 panel members) included”

May combine two claims.

Learning Content (0.67), Final Presentation (0.70), and Performance Management (0.73) had lower initial CVRs.

“Domains with lower initial CVRs-including Learning Content (0.67), Final Presentation (0.70), and Performance Management (0.73)-were strengthened through targeted revisions following expert feedback.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

AI competency is a self-reflective, confidence-bearing capacity to apply knowledge in beneficial ways.

“Chiu et al. [10] further developed the construct of AI competency as a self-reflective, confidencebearing capacity to apply knowledge in beneficial ways.”

• cites [10] Chiu, 2024, What are artificial intelligence literacy and competency? A comprehensive framework to support them, Comput. Educ. Open sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

AI literacy has been positioned as a universal competency required of all citizens, regardless of major or professional background.

“AI literacy-the capacity to critically evaluate, communicate with, and meaningfully use AI tools-has accordingly been positioned as a universal competency required of all citizens, regardless of major or professional background [1,2].”

• cites [1] Long, D., and Magerko, B. (2020, January 25–30). What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA. sourceNot checked — no full text of the source
• cites [2] Lee, 2021, Establishing a definition of AI literacy and designing a liberal arts education program, J. Lang. Lit.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Formal coursework primarily addresses abstract conceptualization and reflective observation, while concrete experience and active experimentation require additional instructional mechanisms.

“From the lens of Kolb's experiential learning theory [11], formal coursework primarily addresses abstract conceptualization and reflective observation, while concrete experience and active experimentation-the other half of the learning cycle-require additional instructional mechanisms.”

• cites [11] Kolb, D.A. (2014). Experiential Learning: Experience as the Source of Learning and Development, FT Press. [2nd ed.].Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The deliberate inclusion of experts affiliated with non-metropolitan institutions was intended to ensure that the resulting framework reflects the diverse institutional realities of contemporary higher education, including resource-constrained contexts.

“The deliberate inclusion of experts affiliated with non-metropolitan institutions (n = 4)-defined here as universities located outside the Seoul Capital Area (Seoul, Incheon, and Gyeonggi Province), commonly referred to as regional universities-was intended to ensure that the resulting framework reflects the diverse institutional realities of contemporary higher education, including resource-”

Probably not a substantial claim: may not pass S4 (based).

The sustainable implementation of national-level investments in AI education depends on universities' ability to design AI/data extracurricular programs that are evidence-based, systematically integrated with coursework, and adaptable to diverse institutional contexts, including resource-constrained settings.

“The sustainable implementation of these national-level investments depends on universities' ability to design AI/data extracurricular programs that are evidence-based, systematically integrated with coursework, and adaptable to diverse institutional contexts, including resource-constrained settings.”

May combine two claims.

Six of the seven Delphi-consensus domains were mapped to Level 1 evaluation criteria, while the seventh (program type) was mapped to Level 2 alternatives.

“Specifically, six of the seven Delphi-consensus domains (learning objectives, mode of operation, curricular linkage, AI practice integration, assessment system, and support system) were mapped to Level 1 evaluation criteria, while the seventh domain (program type) was mapped to Level 2 alternatives, as program types constitute design choices rather than evaluative criteria.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The rapid evolution of AI tools demands flexible learning environments that can adapt more quickly than formal curriculum revision cycles permit.

“Second, the rapid evolution of AI tools (e.g., generative AI platforms) demands flexible learning environments that can adapt more quickly than formal curriculum revision cycles permit.”

May combine two claims.

A meta-analysis of Korean studies up to 2017 reported an overall effect size of 0.534 (medium magnitude) for extracurricular participation, with particularly strong effects on community-mindedness, foundational competencies, and college life adaptation.

“A meta-analysis of Korean studies up to 2017 [12] reported an overall effect size of 0.534 (medium magnitude), with particularly strong effects on communitymindedness, foundational competencies, and college life adaptation.”

• cites [12] Kim, 2017, The analysis of effectiveness and review of university extra-curricular programs, J. Learn.-Cent. Curric. Instr. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Face validity was addressed qualitatively: panel members reviewed the wording, clarity, and surface appropriateness of each item and provided open-ended feedback.

“In addition to the quantitative CVR criterion, face validity was addressed qualitatively: panel members reviewed the wording, clarity, and surface appropriateness of each item and provided open-ended feedback, which informed the item revisions reported in Section 4.4.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Effective AI competency formation requires authentic engagement with AI tools, opportunities for critical evaluation, and reflective practice across diverse contexts.

“Effective competency formation requires authentic engagement with AI tools, opportunities for critical evaluation, and reflective practice across diverse contexts.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The first author and one of the corresponding authors independently coded the open-ended responses and consolidated semantically equivalent items; coding discrepancies were reviewed and adjudicated by the other corresponding author.

“The first author and one of the corresponding authors independently coded the open-ended responses and consolidated semantically equivalent items; coding discrepancies were reviewed and adjudicated by the other corresponding author.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

AI/data competency formation inherently requires hands-on engagement with tools, real datasets, and authentic problems.

“First, AI/data competency formation inherently requires hands-on engagement with tools, real datasets, and authentic problems.”

May combine two claims.

The ten experts represented nine distinct organizations, with each of the eight academically affiliated experts drawn from a different university.

“In total, the ten experts represented nine distinct organizations (eight universities-four metropolitan and four nonmetropolitan-and one company), with each of the eight academically affiliated experts drawn from a different university.”

May combine two claims.

All panel members held a master's degree or higher (eight held doctoral degrees), and all had a minimum of three years of relevant professional experience.

“All panel members held a master's degree or higher (eight held doctoral degrees), and all had a minimum of three years of relevant professional experience.”

Classified as probably not a substantial claim.

A methodologically important feature of well-designed Delphi-AHP studies is the direct continuity between phases: the consensus domains identified through Delphi should serve as the structural foundation for the AHP hierarchy.

“A methodologically important feature of well-designed Delphi-AHP studies is the direct continuity between phases: the consensus domains identified through Delphi should serve as the structural foundation for the AHP hierarchy, ensuring that quantitative prioritization is grounded in qualitative consensus rather than being driven by externally imposed criteria.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Sustainable AI education in higher education requires alignment with the equity dimension of SDG 4: resource-constrained institutions must be able to participate meaningfully in AI literacy development without prohibitive infrastructure investments.

“Third, sustainable AI education in higher education requires alignment with the equity dimension of SDG 4: resource-constrained institutions must be able to participate meaningfully in AI literacy development without prohibitive infrastructure investments, a goal that extracurricular programs-especially those leveraging cloud-based and generative AI tools-can directly support.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Kolb's experiential learning theory posits that effective learning proceeds through a four-stage cycle: concrete experience, reflective observation, abstract conceptualization, and active experimentation.

“According to this theory, effective learning proceeds through a four-stage cycle: concrete experience, reflective observation, abstract conceptualization, and active experimentation.”

• cites [11] Kolb, D.A. (2014). Experiential Learning: Experience as the Source of Learning and Development, FT Press. [2nd ed.].Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Meaningful learning requires students to integrate experiences across curricular and extracurricular contexts and to apply learning across diverse situations.

“Kuh [14] extended Kolb's framework to higher education through the concept of high-impact educational practices, emphasizing that meaningful learning requires students to integrate experiences across curricular and extracurricular contexts and to apply learning across diverse situations.”

• cites [14] Kuh, 2008, High-impact educational practices, Peer Rev.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The Delphi technique is a structured communication method that elicits expert consensus through iterative, anonymous questionnaires with controlled feedback.

“The Delphi technique is a structured communication method that elicits expert consensus through iterative, anonymous questionnaires with controlled feedback [31].”

• cites [31] Linstone, H.A., and Turoff, M. (1975). The Delphi Method: Techniques and Applications, Addison-Wesley.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The content validity index (CVI) was computed as the mean CVR across all retained items, with CVI ≥ 0.80 set as the threshold for overall framework validity.

“The content validity index (CVI) was computed as the mean CVR across all retained items, with CVI ≥ 0.80 set as the threshold for overall framework validity [38].”

• cites [38] Davis, 1992, Instrument review: Getting the most from a panel of experts, Appl. Nurs. Res. sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The Delphi technique's principal strength lies in its capacity to aggregate expert judgment while mitigating social-psychological biases (e.g., groupthink, dominance effects) that can compromise face-to-face deliberation.

“The technique's principal strength lies in its capacity to aggregate expert judgment while mitigating the social-psychological biases (e.g., groupthink, dominance effects) that can compromise face-to-face deliberation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Response rates for both Delphi rounds were 100%.

“Response rates for both rounds were 100%.”

May combine two claims.

Classroom instruction typically engages abstract conceptualization and reflective observation, while well-designed extracurricular programs provide opportunities for concrete experience and active experimentation.

“Classroom instruction typically engages the latter two stages (abstract conceptualization and reflective observation), while well-designed extracurricular programs provide opportunities for concrete experience and active experimentation.”

• cites [11] Kolb, D.A. (2014). Experiential Learning: Experience as the Source of Learning and Development, FT Press. [2nd ed.].Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

AI competency is distinct from AI literacy and is defined as the confident application of knowledge in beneficial ways.

“Chiu et al. [10] introduced the broader construct of AI competency, distinguishing it from AI literacy by defining competency as the confident application of knowledge in beneficial ways.”

• cites [10] Chiu, 2024, What are artificial intelligence literacy and competency? A comprehensive framework to support them, Comput. Educ. Open sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

Consensus was determined using three indicators: content validity ratio (CVR) with a minimum threshold of 0.62 for a panel of 10, coefficient of variation (CV ≤ 0.5), and degree of consensus (≥0.75 based on the interquartile range to upper quartile ratio).

“Consensus was determined using three indicators: (a) content validity ratio (CVR) [35] with a minimum threshold of 0.62 for a panel of 10 (i.e., at least nine panel members rating the item ≥ 4); (b) coefficient of variation (CV ≤ 0.5); and (c) degree of consensus (≥0.75 based on the interquartile range to upper quartile ratio).”

• cites [35] Lawshe, 1975, A quantitative approach to content validity, Pers. Psychol. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The combined emphasis on both criteria reflects an expert vision of extracurricular programs as integrative bridges that connect classroom learning with authentic AI engagement.

“The combined emphasis on both criteria reflects an expert vision of extracurricular programs as integrative bridges that connect classroom learning with authentic AI engagement.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Such an architecture aligns with UNESCO's developmental Understand-Apply-Create framework and with Kuh's high-impact practices typology.

“Such an architecture aligns with UNESCO's [4] developmental Understand-Apply-Create framework and with Kuh's [14] high-impact practices typology.”

• cites UNESCO's [4] UNESCO (2024). AI Competency Framework for Teachers, UNESCO.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kuh's [14] Kuh, 2008, High-impact educational practices, Peer Rev.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The AHP weights reflect relative emphasis when allocating attention and resources across criteria.

“Rather, the AHP weights reflect relative emphasis when allocating attention and resources across criteria.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The three-tier structure resonates with UNESCO's Understand-Apply-Create progression and Kuh's tiered high-impact practice typology.

“So, when interpreted, the structure resonates with UNESCO's [4] Understand-Apply-Create progression and with Kuh's [14] tiered high-impact practice typology.”

• cites UNESCO's [4] UNESCO (2024). AI Competency Framework for Teachers, UNESCO.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kuh's [14] Kuh, 2008, High-impact educational practices, Peer Rev.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The AHP results provide robust empirical support for two foundational design principles of sustainable AI/data extracurricular programs in higher education: authentic AI practice integration and systematic curriculum-extracurricular linkage.

“The AHP results provide robust empirical support for two foundational design principles of sustainable AI/data extracurricular programs in higher education: (i) authentic AI practice integration and (ii) systematic curriculum-extracurricular linkage.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Delphi anonymity was maintained throughout to mitigate social influence effects.

“Fifth, Delphi anonymity was maintained throughout to mitigate social influence effects.”

Classified as probably not a substantial claim.

AHP consistency was strictly enforced (CR ≤ 0.1) to exclude logically inconsistent judgments.

“Fourth, AHP consistency was strictly enforced (CR ≤ 0.1) to exclude logically inconsistent judgments.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All phases were documented in an audit trail to support traceability and replicability.

“Sixth, all phases were documented in an audit trail to support traceability and replicability.”

Probably not a substantial claim: may not pass S2 (checkable).

The narrow margin between the top two program types suggests that these formats should be conceived as complementary rather than competing.

“suggests that these formats should be conceived as complementary rather than competing.”

May combine two claims.

The key contribution of the study is the empirical demonstration that these conceptual tiers can be operationalized through three specific program types, with quantified priority rankings to guide institutional resource allocation.

“The key contribution of the present study is the empirical demonstration that these conceptual tiers can be operationalized through three specific program types, with quantified priority rankings to guide institutional resource allocation.”

May combine two claims.

The hierarchy does not imply that lower-weighted criteria are dispensable; experts view the support-system components as essential foundations.

“Importantly, this hierarchy does not imply that lower-weighted criteria are dispensable; the Delphi consensus confirmed all four support-system components with CVR = 1.00, indicating that experts view these components as essential foundations.”

Classified as probably not a substantial claim.

The 24 components identified in this study provide a concrete operationalization of sustainable AI literacy education.

“The 24 components identified in this study provide a concrete operationalization of sustainable AI literacy education.”

May combine two claims.

This complementarity supports a tiered program architecture: workshops as the curriculum-aligned core, challenges as the advanced practice-integrated extension, and data analytics projects as a specialized supplementary track.

“This complementarity supports a tiered program architecture: workshops as the curriculum-aligned core, challenges as the advanced practice-integrated extension, and data analytics projects as a specialized supplementary track.”

May combine two claims.

Practice integration without curricular linkage risks producing fragmented experiential learning disconnected from formal coursework, while curricular linkage without practice integration risks reducing extracurricular programs to mere extensions of classroom instruction.

“practice integration without curricular linkage risks producing fragmented experiential learning disconnected from formal coursework, while curricular linkage without practice integration risks reducing extracurricular programs to mere extensions of classroom instruction.”

May combine two claims.

The present study structured the AHP hierarchy to inherit directly from the Delphi consensus: six of the seven Delphi-consensus domains served as Level 1 evaluation criteria, while the seventh (program type) served as Level 2 alternatives.

“The present study instead structured the AHP hierarchy to inherit directly from the Delphi consensus: six of the seven Delphi-consensus domains served as Level 1 evaluation criteria, while the seventh (program type) served as Level 2 alternatives.”

May combine two claims.

The narrow margin between A1 and A2 suggests a complementary architecture in which workshops serve as the curriculum-aligned core, challenges serve as the advanced practice-integrated extension, and data analytics projects function as a specialized supplementary track.

“rather than indicating a single dominant program type, the AHP results suggest a complementary architecture in which workshops serve as the curriculum-aligned core, and challenges serve as the advanced practice-integrated extension.Data analytics projects function as a specialized supplementary track.”

May combine two claims.

These findings indicate strong expert consensus regarding two foundational design principles: the centrality of generative AI in contemporary AI/data education and the necessity of systematic curriculum-extracurricular alignment.

“These findings indicate strong expert consensus regarding two foundational design principles: the centrality of generative AI in contemporary AI/data education and the necessity of systematic curriculum-extracurricular alignment.”

May combine two claims.

This finding empirically confirms the two foundational design principles for sustainable AI/data extracurricular programs in higher education: authentic integration of AI practice and systematic linkage with formal coursework.

“This finding empirically confirms the two foundational design principles for sustainable AI/data extracurricular programs in higher education: (i) authentic integration of AI practice (including cloud-based and generative AI tools) and (ii) systematic linkage with formal coursework.”

Classified as probably not a substantial claim.

The study adds to the Delphi-AHP methodology literature demonstrating the methodology's value for educational program design.

“the study adds to the growing body of applications demonstrating the methodology's value for educational program design”

• cites [22][23][24]33Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The panel's strong endorsement of industry mentor linkage and public-data-based analytics practice positions the proposed framework within a wider ecosystem of stakeholders.

“the panel's strong endorsement of industry mentor linkage and public-data-based analytics practice positions the proposed framework within a wider ecosystem of stakeholders”

Probably not a substantial claim: may not pass S4 (based).

Essential AI practice integration may be achievable through cloud-based environments and generative AI tools without prohibitive infrastructure investments.

“the framework suggests that essential AI practice integration may be achievable through cloud-based environments and generative AI tools without prohibitive infrastructure investments”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

These findings carry implications for the sustainable advancement of AI literacy in higher education in alignment with SDG 4.

“These findings carry implications for the sustainable advancement of AI literacy in higher education in alignment with SDG 4.”

Classified as probably not a substantial claim.

The three-program-type priority architecture provides a phased-implementation roadmap suitable for institutions with varying levels of resources.

“The three-program-type priority architecture (workshops as core, challenges as advanced, analytics projects as specialized) provides a phased-implementation roadmap suitable for institutions with varying levels of resources.”

May combine two claims.

Procedural safeguards mitigated the circularity risk: anonymity was maintained throughout, the phases were temporally separated, and the CVR phase employed a distinct judgment task.

“Several procedural safeguards mitigated this risk: anonymity was maintained throughout to reduce social conformity pressure, the phases were temporally separated so that each round required independent re-engagement with the items, and the CVR phase employed a distinct judgment task (essentiality rating) rather than a repetition of prior ratings.”

May combine two claims.

The design cannot rule out that the three phases jointly reproduced rather than independently tested the panel's initial shared assumptions; the reported CVI of 0.84 should be read as evidence of internal consensus coherence rather than external validity.

“Even with these safeguards, the design cannot rule out the possibility that the three phases jointly reproduced, rather than independently tested, the panel's initial shared assumptions; accordingly, the reported CVI of 0.84 should be read as evidence of internal consensus coherence rather than of external validity”

Classified as probably not a substantial claim.

The 24-component framework offers a concrete checklist for AI/data extracurricular program design and self-assessment.

“the 24-component framework offers a concrete checklist for AI/data extracurricular program design and self-assessment.”

May combine two claims.

The study identifies and validates structural components but does not empirically demonstrate the effectiveness of programs designed according to the framework.

“the present study identifies and validates structural components but does not empirically demonstrate the effectiveness of programs designed according to the framework.”

May combine two claims.

The panel's resource-constrained perspective derives from non-metropolitan institutions within a developed higher education system; institutions in low-income countries or without basic AI infrastructure were not represented.

“the panel's resource-constrained perspective derives from non-metropolitan institutions within a developed higher education system; institutions in low-income countries or without basic AI infrastructure were not represented”

May combine two claims.

The study extends existing extracurricular and co-curricular learning frameworks by applying them to AI/data and by quantifying the relative weights of design criteria.

“the study extends existing frameworks by applying them to a specific emerging competency domain (AI/data) and by quantifying the relative weights of design criteria.”

• cites [14][15][16][17]27Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The linkage with digital badges and micro-credentials connects extracurricular learning to broader lifelong-learning ecosystems.

“the linkage with digital badges and micro-credentials connects extracurricular learning to broader lifelong-learning ecosystems, supporting the SDG 4 emphasis on continuous learning opportunities.”