Establishing and implementing a responsible artificial intelligence framework: a 1-year review

Probably not a substantial claim: may not pass S3 (load-bearing).

Saenz et al developed a system-wide governance model at Mass General Brigham, operationalizing responsible AI principles through a structured intake process, shadow deployments, and a real-world case study in generative AI documentation.

“Similarly, Saenz et al developed a system-wide governance model at Mass General Brigham, operationalizing responsible AI (RAI) principles through a structured intake process, shadow deployments, and a real-world case study in generative AI documentation.”

• cites [4] Saenz, 2024, Establishing responsible use of AI guidelines: a comprehensive case study for healthcare institutions, NPJ Digit Med sourceSupported

Classified as probably not a substantial claim.

Vendors claimed negligible differences in magnetic resonance imaging or computed tomography scans across race.

“Vendors claimed negligible differences in magnetic resonance imaging or computed tomography scans across race, but a stratified analysis is necessary to unambiguously demonstrate not only fairness but also transparency, trustworthiness, and applicability across all populations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Most of the intake survey questions align closely with ONC's source attributes guided by fair, appropriate, valid, effective, and safe principles.

“While our framework was developed before ONC's guidance, most of our intake survey questions align closely with their source attributes guided by fair, appropriate, valid, effective, and safe principles.”

May combine two claims.

The remaining pillars showed similar score ranges: 60.4% to 89.6% for transparency, 62.0% to 87.2% for accountability, and 61.3% to 92.8% for trustworthiness.

“The remaining pillars showed a similar range in scores: 60.4% to 89.6% for transparency, 62.0% to 87.2% for accountability, and 61.3% to 92.8% for trustworthiness.”

May combine two claims.

Most respondents did not stratify validation performance by demographic groups, and none did so for underserved populations.

“Most respondents did not stratify validation performance by demographic groups, and none did so for underserved populations.”

May combine two claims.

Lower accountability scores were commonly due to a lack of monitoring processes, often placing responsibility on users to monitor and report errors to the vendor, and a lack of retraining processes without a regular cadence for model updates.

“Lower accountability scores were commonly due to a lack of monitoring processes in place, often placing the responsibility on users to monitor and report errors to the vendor, as well as a lack of retraining processes in place without a regular cadence for model updates.”

The quote only partly states this claim.

Through the UNC Health investigation, four concepts emerged as key pillars of responsible AI models: fairness, transparency, accountability, and trustworthiness.

“Through this investigation, four concepts emerged as key pillars of RAI models.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

One of the two rejections was a rapid rejection, which differed from a traditional rejection as no official scoring could be done.

“One of the 2 rejections was a”

Probably not a substantial claim: may not pass S3 (load-bearing).

One exemplary intake survey response (evaluation L) provided important interpretation.

“However, one exemplary intake survey response (evaluation L) provided such important interpretation.”

May combine two claims.

The volunteer structure did not provide protected administrative time, resulting in minimal engagement and long lead times for a decision.

“Though the formation of a multidisciplinary committee to perform RAI evaluations was met with enthusiasm, the volunteer structure did not provide protected administrative time, resulting in minimal engagement and long lead times for a decision.”

May combine two claims.

Fairness scores dipped the lowest amongst the pillars, ranging from 55.8% to 90.0%, mainly due to insufficient consideration for fairness throughout model building.

“Fairness scores dipped the lowest amongst the pillars, ranging from 55.8% to 90.0%, mainly due to insufficient consideration for fairness throughout model building.”

• Also stated in: § Abstract · plain

  “Twelve survey evaluations revealed low fairness scores and resulted in 83% conditional approvals.”

May combine two claims.

Many vendors failed to provide performance metrics stratified by patient demographics and did not consider or address training data bias.

“Given many vendors failed to provide performance metrics stratified by patient demographics and did not consider or address training data bias, fairness appears to be an under-considered model attribute, which should prompt implementation apprehension.”

May combine two claims.

Ten of the survey evaluations (83%) were conditionally approved, and the remaining two were rejected.

“Ten of the survey evaluations (83%) were conditionally approved, and the remaining two were rejected.”

• Also stated in: § Abstract · plain

  “Twelve survey evaluations revealed low fairness scores and resulted in 83% conditional approvals.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Each AI tool must have a minimum of five evaluators but cannot consist of only IT professionals.

“Each tool must have a minimum of five evaluators but cannot consist of only IT professionals.”

Probably not a substantial claim: may not pass S4 (based).

Continuous monitoring ensures that models perform accurately and consistently across all patient demographics, maintaining the integrity and reliability of AI applications within the health system.

“Continuous monitoring ensures that models perform accurately and consistently across all patient demographics, maintaining the integrity and reliability of AI applications within the health system.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Collaboration between vendors and healthcare delivery organizations is essential, and monitoring responsibility should ideally be shared.

“As such, this exercise has reinforced our perspective that collaboration between vendors and healthcare delivery organizations is essential and that monitoring responsibility should ideally be shared.”

May combine two claims.

The intake survey responses illuminated a need for all vendors to expand upon fairness investigations and to improve transparency efforts for certain algorithm types.

“Additionally, the intake survey responses illuminated a need for all vendors to not only expand upon fairness investigations but to improve transparency efforts for certain algorithm types.”

Probably not a substantial claim: may not pass S3 (load-bearing).

As proposed by the Food and Drug Administration, AI tool vendors or manufacturers should be responsible for post-market surveillance through real-world performance monitoring.

“As proposed by the Food and Drug Administration, AI tool vendors or manufacturers should be responsible for post-market surveillance through real-world performance monitoring.29”

• cites [29] Food and Drug Administration (FDA), Department of Health and Human Services (HHS), 2019Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The World Health Organization presented 6 governance principles and elements that governance frameworks should include, yet fails to offer a specific, enforceable governance framework to systematically and comprehensively assess AI tools in medicine.

“Internationally, the World Health Organization presented 6 governance principles, and elements that governance frameworks should include, yet similarly fails to offer specific, enforceable governance framework to systematically and comprehensively assess AI tools in medicine.”

• cites [2] World Health Organization, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Managing a centralized AI review required balancing enterprise-wide resources with individual hospital needs, primarily accomplished by AAA.

“Managing a centralized AI review required balancing enterprise-wide resources with individual hospital needs, primarily accomplished by AAA, the RAI oversight body.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The survey evaluation is completed independently by each member, with each intake survey response assessed for sufficiency on a 5-point Likert scale.

“The survey evaluation is completed independently by each member whereby each intake survey response is assessed for sufficiency according to a 5-point Likert scale (1 ¼”

Probably not a substantial claim: may not pass S2 (checkable).

Given the variety of data ownership policies, system resource constraints, and application specifics, nuanced monitoring responsibility could fall to the vendor or health system.

“However, given the variety of data ownership policies, system resource constraints, and application specifics, nuanced monitoring responsibility could fall to the vendor or health system.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Evaluators convene to discuss responses with inter-evaluator disagreement or low average score to establish consensus on deficits, followed by a vote to recommend approval, conditional approval, or rejection based on RAI merits.

“Evaluators then convene to discuss responses with inter-evaluator disagreement or low average score to establish consensus on deficits, followed by a vote to recommend approval, conditional approval, or rejection of the tool based on its RAI merits, where conditional approval offers a pathway to approval if specific requirements are met.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The RAI intake survey responses are evaluated by a group of volunteer technical advisors from twelve disciplines, including medical specialties and health system departments.

“The RAI intake survey responses are evaluated by a group of volunteer technical advisors from twelve disciplines including medical specialties (eg, cardiology, radiology, internal medicine, pathology, and laboratory medicine, emergency medicine) and health system departments (eg, nursing, community health, population health, data sciences, IT, ambulatory services, analytics solutions management) (”

Probably not a substantial claim: may not pass S3 (load-bearing).

Having procedures to manage one or more rejections was necessary to reconcile all review outcomes.

“Having procedures to manage one or more rejections was necessary to reconcile all review outcomes.”

May combine two claims.

The community approach allowed leveraging diverse expertise but also led to variability in the thoroughness of model survey evaluations.

“The community approach allowed us to leverage diverse expertise but also led to variability in the thoroughness of model survey evaluations.”

Probably not a substantial claim: may not pass S4 (based).

The RAI framework formed at UNC Health ensures only models with adequate fairness, transparency, accountability, and trustworthiness will be implemented.

“The RAI framework formed at UNC Health ensures only models with adequate fairness, transparency, accountability, and trustworthiness will be implemented.”

• AssumptionRests on an assumption: Health systems face challenges in scaling responsible AI oversight across diverse clinical domains.

Probably not a substantial claim: may not pass S2 (checkable).

A stratified analysis is necessary to unambiguously demonstrate fairness, transparency, trustworthiness, and applicability across all populations.

“Vendors claimed negligible differences in magnetic resonance imaging or computed tomography scans across race, but a stratified analysis is necessary to unambiguously demonstrate not only fairness but also transparency, trustworthiness, and applicability across all populations.”

Probably not a substantial claim: may not pass S2 (checkable).

Given the limited AI model details in those areas, health systems may need to further consider whether internal post-implementation monitoring infrastructure is necessary.

“Given the limited AI model details in those areas, health systems may need to further consider whether internal post-implementation monitoring infrastructure is necessary.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A survey evaluation was not considered complete until at least five evaluators had submitted scores.

“A survey evaluation was not considered complete until at least five evaluators had submitted scores; however, the median number of evaluators was 5.5.”