The Digital Health Competencies in Medical Education Framework

May combine two claims.

Two recently published digital health frameworks for medical students outlined 27 artificial intelligence-specific competencies and 40 digital health topics, respectively.

“Two recently published digital health frameworks for medical students outlined 27 artificial intelligence-specific competencies 47 and 40 digital health topics, 48 respectively.”

• cites [47] sourceDisputed

  The Abstract and Introduction identify this as a framework for digital health equity intended for digital health tool creators and developers, not a framework for medical students. The paper describes digital determinants of health, not 40 digital health curriculum topics; the 27 AI competencies are attached to a separate citation in the citing sentence. (LLM, conf=0.87)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [48] sourceCould not verify

  The Introduction mentions artificial intelligence as part of digital health, and Results > Research Topics categorizes digital health literacy publications. Neither passage addresses frameworks for medical students or lists 27 AI-specific competencies or 40 digital health topics. Searches for both counts and their associated terms found no supporting passage in our copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

Many medical students and junior physicians do not feel equipped to use digital health technologies and are keen to receive more training in this field.

“This report is consistent with findings from multiple countries showing that many medical students and junior physicians do not feel equipped to use digital health technologies and are keen to receive more training in this field.[7][8][9][10][11][12]”

• cites [7] sourceNot checked — no full text of the source
• cites [8] sourcePartly supported

  The Abstract Results report that most surveyed medical students in China said they urgently needed digital-health knowledge and skills, and the Discussion says more than 80% wanted to learn more in their curriculum. This supports the claim about students wanting training and feeling underprepared, but the paper’s survey does not establish the same finding for junior physicians. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [9] sourcePartly supported

  The Discussion reports that medical students lack an overview of digital health applications and that two-thirds request additional digital communication training. The study surveyed Austrian medical students, not junior physicians, so it does not establish that part of the claim. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The application of digital health has rapidly grown worldwide over the past 4 decades and accelerated particularly during the COVID-19 pandemic.

“The application of digital health has rapidly grown worldwide over the past 4 decades, and accelerated particularly during the COVID-19 pandemic.[1][2][3]”

• cites [1] sourcePartly supported

  The Results section describes uses of telehealth during the COVID-19 outbreak, and the Conclusion calls for expanded use. But the paper does not establish that digital health grew rapidly worldwide over the preceding four decades or that its rate of adoption accelerated during the pandemic; its focus is telehealth’s potential and use during COVID-19. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [2] sourcePartly supported

  The Results (Patient-facing technologies; page unavailable) describe UC San Diego Health rapidly expanding telemedicine during COVID-19, and the Discussion describes pandemic-related uses elsewhere. The paper does not establish that digital health grew worldwide over the preceding four decades or measure a worldwide acceleration during the pandemic. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [3] sourcePartly supported

  The Introduction (page unavailable) says US telemedicine use increased steadily over the preceding decade, then expanded rapidly during COVID-19. The paper supports the pandemic acceleration, but not the claim of rapid worldwide growth in digital health over four decades: its account concerns the narrower field of telemedicine, chiefly in the United States. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S2 (checkable).

Effective training in digital health is vital for delivering safe, efficient, and high-quality health care.

“Effective training in digital health is vital for delivering safe, efficient, and high-quality health care.13”

• cites [13] sourcePartly supported

  The Introduction says postgraduate health-informatics training is crucial to bridging doctors’ skills gap, and the Conclusion argues that a digitally ready workforce is needed to realise the benefits of digital healthcare. But the Discussion says evidence that such training improves digital programme success is lacking; the paper does not establish that training delivers safer, more efficient, or higher-quality care. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

Country-specific national medical curricula that incorporate digital health learning objectives vary widely and overlook more recently introduced digital health tools such as symptom checkers and conversational agents.

“There are several country-specific national medical curricula that incorporate digital health learning objectives.[43][44][45][46] However, these curricula vary widely and overlook more recently introduced digital health tools such as symptoms checkers and conversational agents (eg, chat bots).”

• cites [43] sourceCould not verify

  The cited sentence attaches the references to the existence of country-specific national medical curricula with digital health learning objectives. The paper’s 'Strengthen health systems' section discusses digital health infrastructure, telehealth and training, but the passages found when searching for medical curricula and learning objectives do not address national medical curricula. The subsequent claim about symptom checkers and conversational agents has no citation attached in the printed sentence. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [44] sourceNot checked — no full text of the source
• cites [45] sourceNot checked — no full text of the source

May combine two claims.

These 2 frameworks involved experts from a single country in their development, which may limit their wider use.

“Nevertheless, these 2 frameworks involved experts from a single country in their development, which may limit their wider use.”

• cites [47] sourceCould not verify

  The Abstract and the section on the NIMHD Research Framework describe this paper’s framework and the earlier framework it expands. Neither those passages nor the Discussion identifies the countries of the experts involved in developing the frameworks or says that single-country involvement may limit wider use. Searches for expert involvement, country, and international applicability found no passage establishing the claim. (LLM, conf=0.74)

  Experimental — accuracy not yet measured
• cites [48] sourceCould not verify

  The Discussion > Limitations section discusses database selection, English-language publications, and the lack of detailed study-content analysis—not how two frameworks were developed. Searches for single-country expert panels, framework development, and wider applicability found no passage addressing the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Expert agreement percentage is commonly used as the cutoff for consensus in Delphi, with studies using thresholds ranging from 67% to 80%.

“Expert agreement percentage is commonly used as the cutoff for consensus in Delphi, 50 with studies using different thresholds ranging from 67% to 80%.[51][52][53]”

• cites [50] sourceCould not verify

  The Methods > Data Analyses section gives thresholds for scale reliability and factor analysis, not expert agreement in a Delphi study. Searches for Delphi, expert-agreement cutoffs, and 67%–80% consensus thresholds found no passage addressing the claim in our copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [51] sourceNot checked — no full text of the source
• cites [52] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Of the 149 participants who took part in both rounds, 63 (42.6%) were based in high-income countries and 19 (12.8%) in low-income countries.

“Of those who took part in both rounds (149 participants), 63 (42.6%) were based in high-income countries and 19 (12.8%) in low-income countries (World Bank country income classification).54”

• cites [54] sourceCould not verify

  The Data availability section says no data were generated for this article, and the passages retrieved from our copy do not report participants taking part in two rounds. Searches for the participant counts and the World Bank income classifications found no passage supporting the stated figures. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

May combine two claims.

Of the 162 learning outcomes retained from round 1 and the 19 newly added learning outcomes, 30 attained more than 70% agreement rates for the mandatory category, while 151 did not reach the 70% agreement rate for any single category.

“Of the 162 learning outcomes retained from round 1 and the 19 newly added learning outcomes, 30 attained more than 70% agreement rates for the mandatory category, while 151 did not reach the 70% agreement rate for any single category (ie, mandatory, elective, or supplementary).”

May combine two claims.

The group identified barriers to adopting a universal DHC framework in four categories: geographic heterogeneity, variation in resources between regions and between institutions in the same region, variance in local needs within a given institution, and the zero-sum nature of the curriculum.

“Our group identified the following categories: geographic heterogeneity, variation in resources between regions and between institutions in the same region, variance in local needs within a given institution, and the zero-sum nature of the curriculum (ie, adopting new content requires removal of other content).”

May combine two claims.

Following differentiation of learning outcomes in round 2, 30 learning outcomes emerged as mandatory and 148 as discretionary.

“Following differentiation of learning outcomes in round 2, 30 learning outcomes emerged as mandatory and 148 as discretionary.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Qualitative analysis of 169 free-text suggestions led to rewording of 14 learning outcomes and merging of 6 learning outcomes into 3.

“Qualitative analysis of 169 free-text suggestions led to rewording of 14 learning outcomes and merging of 6 learning outcomes into 3.”

May combine two claims.

After the 2-round Delphi survey and the consensus meeting, the final DECODE framework included 4 domains, 19 competencies, 33 mandatory learning outcomes, and 145 discretionary learning outcomes.

“After the 2-round Delphi survey and the consensus meeting, a total of 4 domains, 19 competencies, 33 mandatory learning outcomes, and 145 discretionary learning outcomes were included in the final DECODE framework (eAppendix 2 in the Supplement).”

• AssumptionRests on an assumption: The initial list of domains, competencies, and learning outcomes derived from the scoping review and expert consultations is comprehensive enough to cover all relevant digital health competencies.

May combine two claims.

In round 2, all newly added items reached the predefined level of consensus and were included in the framework.

“In round 2, all newly added items reached the predefined level of consensus and were included in the framework.”

May combine two claims.

There was a high number of participating nations with representations from both lower middle-income and low-income countries within the expert panel.

“Next, there was a high number of participating nations with representations from both lower middleincome and low-income countries within the expert panel.”

May combine two claims.

In this framework, 13 of the 19 competencies have both mandatory and discretionary learning outcomes, while the remaining 6 have only discretionary learning outcomes.

“In this framework, 13 of the 19 competencies have both mandatory and discretionary learning outcomes, while the remaining 6 have only discretionary learning outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Twelve learning outcomes that did not reach the a priori consensus level were removed from the list.

“Twelve learning outcomes that did not reach the a priori consensus level were removed from the list.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A total of 446 free-text suggestions were received.

“A total of 446 free-text suggestions were received.”

May combine two claims.

Following qualitative analysis, 1 competency and 7 learning outcomes associated with this competency were removed, 16 learning outcomes were reorganized and merged into 8, 64 learning outcomes were reworded, and 2 new competencies and 19 new learning outcomes were added.

“Following qualitative analysis, 1 competency and 7 learning outcomes associated with this competency were removed, 16 learning outcomes were reorganized and merged (into 8 learning outcomes), 64 learning outcomes were reworded, and 2 new competencies and 19 new learning outcomes were added.”

Classified as probably not a substantial claim.

There was consideration that some discretionary learning outcomes might already be addressed in other facets of medical curricula, such as within the purview of public health or bioinformatics.

“There was consideration that some of these might already be addressed in other facets of medical curricula, such as within the purview of public health or bioinformatics.”

Classified as probably not a substantial claim.

The consensus panel concurred that the interpretation of the nomenclature used within this framework may vary due to geographical, language, and other differences.

“The consensus panel concurred that the interpretation of the nomenclature used within this framework may vary due to geographical, language, and other differences.”

Classified as probably not a substantial claim.

The consensus panel expressed concern about the potential oversight of these critical domains if not adequately covered in a specific digital health curriculum.

“Nevertheless, the consensus panel expressed concern about the potential oversight of these critical domains if not adequately covered in a specific digital health curriculum.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Several experts recognized the tremendous potential of digital diagnostics in LMICs, highlighting that regions with the scarcest resources stand to gain substantially from the remote capabilities of digital diagnostics.

“Nevertheless, several experts recognized the tremendous potential of digital diagnostics in LMICs, highlighting that regions with the scarcest resources stand to gain substantially from the remote capabilities of digital diagnostics.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The consensus panel agreed that the use of working definitions in the preface of the framework would help circumvent these issues.

“The consensus panel agreed that the use of working definitions in the preface of the framework would help circumvent these issues.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The consensus panel acknowledged that digital health literacy (of health consumers) is a concept that is not universally agreed upon and is difficult to measure and therefore challenging to operationalize.

“The consensus panel acknowledged that digital health literacy (of health consumers) is a concept that is not universally agreed upon and is difficult to measure and therefore challenging to operationalize.”

May combine two claims.

With most of its associated learning outcomes included as discretionary, the findings aptly reflect the present state of digital diagnostics, wherein most experts acknowledged its relevance in medical education but did not consider it mandatory across all settings.

“With most of its associated learning outcomes included as discretionary, our findings aptly reflect the present state of digital diagnostics, wherein most experts acknowledged its relevance in medical education but did not consider it mandatory across all settings.”

Classified as probably not a substantial claim.

The consensus panel highlighted the need for a focused approach to distinguish digital health from its nondigital counterparts during the curriculum development process.

“The consensus panel highlighted the need for a focused approach to distinguish digital health from its nondigital counterparts during the curriculum development process.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Several experts raised concerns on the challenges in incorporating all competencies within this framework into the existing crowded medical curricula and the potential pushback that might arise.

“From the curriculum implementation perspective, several experts raised concerns on the challenges in incorporating all competencies within this framework into the existing crowded medical curricula and the potential pushback that might arise.”

May combine two claims.

Existing digital health frameworks focus mainly on more traditional biomedical informatics-related competencies and do not cover all aspects of digital health.

“While useful, these frameworks focus mainly on more traditional biomedical informatics-related competencies and do not cover all aspects of digital health.”

Probably not a substantial claim: may not pass S2 (checkable).

There is a clear and urgent need for integration of digital health education into all medical school programs.

“There is a clear and urgent need for integration of digital health education into all medical school programs.”

May combine two claims.

Other existing digital health frameworks largely focus on nursing staff in high-income countries settings, limiting their adoption in preregistration medical education and in low- and middle-income countries.

“Other existing digital health frameworks largely focus on nursing staff in high-income countries settings, 29 limiting their adoption in preregistration medical education (which can be both undergraduate or postgraduate) and in low-and middle-income countries.”

• cites [29] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

Digital health education and training should follow a clearly defined framework of digital health competencies.

“Digital health education and training should follow a clearly defined framework of DHCs.29”

• cites [29] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

A further criticism of the Delphi approach is its potential constraint when experts are confined to voting on a predetermined list.

“A further criticism of the Delphi approach lies in its potential constraint when experts are confined to voting on a predetermined list.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The majority of participants reported having a teaching role (125 [83.9%]) and/or research role (109 [73.2%]), while one-half had a clinical role (76 [51.0%]).

“The majority reported having a teaching role (125 participants [83.9%]) and/or research role (109 participants [73.2%]), while one-half of them had a clinical role (76 participants [51.0%]).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Consensus meeting panelists were selected to represent a range of roles within medical education, diverse medical fields, and global geographic locations.

“They were selected to represent a range of roles within medical education, diverse medical fields, and global geographic locations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Qualitative analysis of free-text suggestions was performed independently by 2 reviewers, and discrepancies were resolved through discussion with a third reviewer.

“Qualitative analysis of the free-text suggestions was performed independently by 2 reviewers (O.C.O. and T.E.F.), and discrepancies were resolved through discussion with a third reviewer (J.C.).”

Classified as probably not a substantial claim.

Consensus was reached through discussion.

“Consensus was reached through discussion.”

May combine two claims.

A total of 30 DHC frameworks and 28 common digital health domains were identified in the scoping review, with only 1 framework developed specifically for preregistration medical education.

“A total of 30 DHC frameworks and 28 common digital health domains were identified, and a varied focus across health professions was noted with only 1 framework developed specifically for preregistration medical education.”

• cites [29] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two pilots were carried out to review the survey questionnaire and the identified list of domains, competencies, and learning outcomes.

“Subsequently, 2 pilots were carried out.The survey questionnaire and the identified list of domains, competencies, and learning outcomes were distributed among the core group for an extensive review.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study followed the ACCORD reporting guideline for consensus methods in biomedicine developed via a modified Delphi.

“and followed the Accurate Consensus Reporting Document (ACCORD) reporting guideline for consensus methods in biomedicine developed via a modified Delphi.”

May combine two claims.

The involvement of experts from lower middle-income and low-income countries enhanced the framework's broader applicability and led to advocacy for elevating and incorporating digital health equity within the competencies.

“Because priorities in digital health training of future physicians in these settings may differ, the involvement of experts from these countries enhanced the framework's broader applicability and led to advocacy for elevating and incorporating digital health equity within the competencies (eg, digital determinants of health).”

Probably not a substantial claim: may not pass S3 (load-bearing).

This study was approved by the Nanyang Technological University institutional review board.

“This study was approved by Nanyang Technological University institutional review board”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The substantial presence of institutional leadership among Delphi participants enriched the framework with their strategic insights, ensured its alignment with broader educational and organizational needs, and may facilitate its adoption within their respective institutions.

“The substantial presence of institutional leadership among Delphi participants enriched the framework with their strategic insights, ensured its alignment with broader educational and organizational needs, and may facilitate its adoption within their respective institutions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Written informed consent was obtained online before participation.

“Detailed information on the purpose of the study and the roles of each panelist were provided before written informed consent was obtained online.”

Probably not a substantial claim: may not pass S4 (based).

To the authors' knowledge, this is the first attempt to develop a curriculum for digital health medical education at a global scale.

“To our knowledge, this is the first attempt to develop a curriculum for digital health medical education at a global scale.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In total, 211 individuals participated in round 1 of the survey, and 149 participated in round 2 (70.6% of participants from round 1).

“In total, 211 individuals participated in round 1 of the survey, and 149 participated in round 2 (70.6% of participants from round 1).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Almost two-thirds of the participants (91 [61.1%]) reported clinical medicine as their primary professional background.

“Almost two-thirds of the participants (91 participants [61.1%]) reported clinical medicine as their primary professional background.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Items with a marginal percentage of agreement (60.0%-69.9%) were highlighted and discussed in the consensus meeting.

“Items with a marginal percentage of agreement (ie, 60.0%-69.9%)were highlighted and discussed.”

Probably not a substantial claim: may not pass S3 (load-bearing).

During the Delphi survey and consensus meeting, several areas of debate arose, suggesting highly relevant topics for stakeholders in medical education and health care to consider when developing or implementing digital health curricula in medical institutions.

“During the Delphi survey and consensus meeting, several areas of debate arose.These discussions suggest highly relevant topics that stakeholders in medical education and health care should consider when developing or implementing digital health curricula in medical institutions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The majority (140 [94.0%]) worked in a university, and more than one-half (83 [55.7%]) worked in a clinical setting.

“The majority (140 participants [94.0%]) also worked in a university, and more than one-half of them (83 participants [55.7%]) worked in a clinical setting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Approximately one-third (47 [31.5%]) reported holding an institutional leadership position.

“Approximately one-third (47 participants [31.5%]) reported holding an institutional leadership position, such as president or chancellor of university, dean or vice dean of medical school, or chief medical officer of hospital.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A virtual consensus meeting was held in September 2023 and involved 12 participants (16 invited) who had taken part in rounds 1 and 2, and 2 members of the core group.

“A virtual consensus meeting was held in September 2023 and involved 12 participants (16 invited) who had taken part in rounds 1 and 2, and 2 members of the core group (meeting chair and notetaker).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Four digital health researchers screened competencies against predefined eligibility criteria.

“Four digital health researchers screened competencies against predefined eligibility criteria (eTable 1 in the Supplement).”

Classified as probably not a substantial claim.

For an item to be included in the DHC framework, a prespecified combined rating of agree and strongly agree of at least 70% of participants, excluding blank votes and abstentions, had to be achieved.

“For an item to be included in the DHC framework, a prespecified combined rating (agree and strongly agree) of at least 70% of the participants (excluding blank votes and abstentions) had to be achieved.”

Probably not a substantial claim: may not pass S2 (checkable).

There is a need for a comprehensive and actionable international digital health competencies framework for medical education.

“Therefore, there is a need for a comprehensive and actionable international DHC framework for medical education.”

Probably not a substantial claim: may not pass S3 (load-bearing).

An iterative sampling method was used to create a diverse expert panel for the Delphi study.

“An iterative sampling method was used to create a diverse expert panel for the Delphi study.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The absence of a neutral option in the Likert scale may have inadvertently forced respondents to select either a positive or negative response despite their ambivalence, potentially introducing response bias.

“Third, the absence of a neutral option in the Likert scale may have inadvertently forced the respondents to select either a positive or negative response despite their ambivalence; this could have introduced response bias.”

May combine two claims.

The framework is based on findings of a scoping review, expert consultations, a 2-round Delphi survey, a consensus meeting, and postmeeting qualitative feedback.

“The framework is based on findings of a scoping review, 29 expert consultations, a 2-round Delphi survey, a consensus meeting, and postmeeting qualitative feedback.”

• cites [29] sourceNot checked — no full text of the source

May combine two claims.

Survey participants were from 79 countries and territories in round 1 and 68 countries in round 2, with representation from all 6 WHO regions and all 4 World Bank country income groups.

“Survey participants were from 79 countries and territories in round 1 and 68 countries in round 2, with representation from all 6 WHO regions and all 4 World Bank country income groups (eFigure in the Supplement).”

Not yet classified.

The assumption that this curriculum would cover datarelated domains comprehensively could possibly lead to displacement and neglect of these essential aspects of data handling in other educational domains, such as population health.

“The assumption that this curriculum would cover datarelated domains comprehensively could possibly lead to displacement and neglect of these essential aspects of data handling in other educational domains, such as population health.”

Classified as probably not a substantial claim. May combine two claims.

The international composition of the Delphi expert panel provided a balanced perspective on digital health technologies and digital health inequities, leading to the segregation of learning outcomes into mandatory and discretionary categories to enhance their applicability in diverse educational settings.

“The international composition of the Delphi expert panel provided a balanced perspective on digital health technologies and digital health inequities, leading to the segregation of learning outcomes into mandatory and discretionary categories to enhance their applicability in diverse educational settings.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Digital health literacy was acknowledged as a superdeterminant of health with civic, digital, and health literacies as its 3 building blocks.

“More recently, digital health literacy was acknowledged as a superdeterminant of health with civic, digital, and health literacies as its 3 building blocks.70”

• cites [70]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable).

Focusing on foundational concepts and skills that will support medical students through clinical practice in lieu of technical skills that are likely to change as technology evolves could achieve this balance.

“This could be achieved by focusing on foundational concepts and skills that will support medical students through clinical practice in lieu of technical skills that are likely to change as technology evolves.”

Classified as probably not a substantial claim. May combine two claims.

The framework is adaptable to each country's specific needs and contexts, ensuring both relevance and flexibility in curriculum design.

“Adaptable to each country's specific needs and contexts, this approach ensures both relevance and flexibility in curriculum design.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Introducing these competencies might result in redundancy.

“As such, introducing these competencies might result in redundancy.”

Probably not a substantial claim: may not pass S2 (checkable).

The framework will play an enabling role in assisting individual medical institutions in developing and introducing digital health learning outcomes and appropriate learning opportunities into curricula for medical students.

“This framework will play an enabling role in assisting individual medical institutions in developing and introducing digital health learning outcomes and appropriate learning opportunities into curricula for medical students.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Free-text responses were allowed in the survey for experts to propose additional items, which has proven to be invaluable.

“To mitigate this, free-text responses were allowed in the survey for experts to propose additional items, which has proven to be invaluable.”

Probably not a substantial claim: may not pass S2 (checkable).

Integration of this framework into curriculum design can assist in better preparing future physicians for the ongoing digital transformation in health care.

“Integration of this framework into curriculum design can assist in better preparing future physicians for the ongoing digital transformation in health care.”

Classified as probably not a substantial claim.

Within the scope of this study, the authors included all domains deemed pertinent to the domain of digital health, irrespective of their potential intersection with other disciplines, to provide medical educators globally with a comprehensive framework for consideration.

“Therefore, within the scope of this study, we included all domains deemed pertinent to the domain of digital health, irrespective of their potential intersection with other disciplines, to provide medical educators globally with a comprehensive framework for consideration.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some argue that digital health literacy is the convergence of digital literacy and health literacy, which are 2 different constructs, but they are equally important in the context of digital health.

“While some argue that digital health literacy is the convergence of digital literacy and health literacy, 66 which are 2 different constructs, they are nonetheless equally important in the context of digital health.”

• cites [66]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The presumption that certain competencies might already be encompassed elsewhere or an optimistic anticipation of their coverage within other curricula poses the risk of inadvertent oversight of these competencies.

“However, the presumption that certain competencies might already be encompassed elsewhere or an optimistic anticipation of their coverage within other curricula poses the risk of inadvertent oversight of these competencies.”

Probably not a substantial claim: may not pass S2 (checkable).

The conundrum lies within balancing the enthusiasm to cover all relevant domains with the practicality of curriculum implementation to ensure that all essential items are effectively integrated into the curriculum.

“The conundrum lies within balancing the enthusiasm to cover all relevant domains with the practicality of curriculum implementation to ensure that all essential items are effectively integrated into the curriculum.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Given the pervasiveness of digitalization in health care, digital health curricula might become the default standard for anything related to data.

“Given the pervasiveness of digitalization in health care, digital health curricula might become the default standard for anything related to data.”

May combine two claims.

Health and digital literacy are both well-established determinants of health.

“Health and digital literacy are both well-established determinants of health.[67][68][69]”

• cites [67]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [68]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [69] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The use of an integrated or cross-domain curriculum approach, such as in planetary health, may also facilitate implementation by expanding existing content to accommodate digital health domains.

“The use of an integrated or cross-domain curriculum approach, such as in planetary health, may also facilitate implementation by expanding existing content to accommodate digital health domains.74”

• cites [74]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

One of the challenges in creating digital health competencies for a global audience is the need to use meaningful terminology, given the diverse national and local approaches to curriculum design, terminology, and practices.

“One of the challenges in creating digital health competencies for a global audience is the need to use meaningful terminology, given the diverse national and local approaches to curriculum design, terminology, and practices.”

Classified as probably not a substantial claim.

Placing too much emphasis on general concepts applicable to both digital and nondigital contexts risks diluting the distinctiveness of digital health.

“Placing too much emphasis on general concepts applicable to both digital and nondigital contexts risks diluting the distinctiveness of digital health.”

• AssumptionRests on an assumption: Some fundamental principles, such as data privacy and security, have already existed in the predigital era and are not exclusive to digital health.

May combine two claims.

There are several terminology schemes used in curriculum and instructional design in medical education globally, and these vary depending on the region, location, and organization.

“There are several terminology schemes used in curriculum and instructional design in medical education globally, and these vary depending on the region, location, and organization.”

• cites [59] sourceNot checked — no full text of the source
• cites [60]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [61]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Digital health literacy affects patients' access to digital health services, health-related behaviors, and health outcomes.

“This speaks to the importance of digital health literacy as a concept that should be understood and considered by all who practice modern medicine because it affects patients' access to digital health services, 71 health-related behaviors, 72 and health outcomes.73”

• cites [71]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [72]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [73]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

With limited exposure and little hands-on practice, medical students may have difficulty grasping the use of these platforms.

“With limited exposure and little hands-on practice, medical students may have difficulty grasping the use of these platforms.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Educators who lack clinical experience in using digital diagnostics may not be confident and competent to educate students on these technologies.

“In addition, educators who lack clinical experience in using digital diagnostics may not be confident and competent to educate students on these technologies.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The discourse around the feasibility and costs associated with setting up these platforms for educational purposes, along with how learners can effectively demonstrate the requisite competencies, is likely to persist for years to come.

“Notwithstanding the potential benefits of digital diagnostics, the discourse around the feasibility and costs associated with setting up these platforms for educational purposes, along with how learners can effectively demonstrate the requisite competencies, is likely to persist for years to come.”

Classified as probably not a substantial claim.

This variation emphasizes the need for clarity, refinement, and consensus around language in the field of digital health.

“This variation emphasizes the need for clarity, refinement, and consensus around language in the field of digital health.”

• AssumptionRests on an assumption: The use of terms such as competencies, outcomes, and objectives is widespread, but the way these terms are defined, used, and understood varies around the world.

Probably not a substantial claim: may not pass S3 (load-bearing).

This is also relevant for institutions that offer dual-degree programs such as MD-MBA, MD-JD, MD-MPH, and MD-PhD, where some aspects of the DECODE framework might be covered in the combined degree program but not within the medical curriculum.

“This is also relevant for institutions that offer dual-degree programs such as MD-MBA (business), MD-JD (law), MD-MPH (public health), and MD-PhD, where some aspects of the DECODE framework might be covered in the combined degree program but not within the medical curriculum.”

May combine two claims.

In published literature, the terms digital health literacy, digital literacy, and eHealth literacy have been used to denote a common or substantially similar concept, and these terms are sometimes used interchangeably.

“In published literature, the terms digital health literacy, digital literacy, and eHealth literacy have been used to denote a common or substantially similar concept, [62][63][64] and these terms are sometimes used interchangeably.65”

• cites [62]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [63]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [64] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Exhaustiveness of the initial item list could not be guaranteed.

“Although the research team attempted to identify all possible options through the preliminary work (scoping review, iterative expert consultations, and piloting) leading up to the initial item list, exhaustiveness of the list could not be guaranteed.”