Organizational Governance of Emerging Technologies: AI Adoption in Healthcare

Classified as probably not a substantial claim. May combine two claims.

Participants hoped to build trust in AI software to a degree where clinical end users trusted the technology and found it useful, but not to a degree where they would fully rely on it in making decisions.

“Participants hoped to build trust in AI software to a degree where clinical end users trusted the technology and found it useful, but not to a degree where they would fully rely on it in making decisions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants cautioned that model evaluation may have to change over time, as the model performance may drift due to changes made in clinical care.

“The participants cautioned that model evaluation may have to change over time, as the model performance may drift due to changes made in clinical care.”

• AssumptionRests on an assumption: Model performance may drift due to changes made in clinical care.

Probably not a substantial claim: may not pass S3 (load-bearing).

One of the most common practices of building a user-friendly UI was to make it simple and customize it for different groups of users and different sites of users.

“One of the most common practices of building a user-friendly UI was to make it simple and customize it for different groups of users and different sites of users.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants felt it was important to define which metrics organizations should measure to determine the success of the AI solution.

“In addition to the model performance metrics, participants felt it was important to define which metrics the organizations should measure to determine the success of the solution.”

May combine two claims.

Participants felt motivated to build AI in-house when it did not exist in the market or they were skeptical about an existing model.

“Participants felt motivated to build AI in-house when it did not exist in the market or they were skeptical about an existing model.”

May combine two claims.

Vendors often did not have or were not willing to share material needed for quality assurance.

“Vendors often did not have or were not willing to share the material.”

Classified as probably not a substantial claim.

Participants reported that a new workflow involving AI software should be designed before the software was integrated into clinical practice.

“The participants reported that a new workflow involving AI software should be designed before the software was integrated into clinical practice.”

May combine two claims.

Common methods of prospective validation included conducting a silent trial, comparing the model outcomes to patient charts in the medical record, and conducting a pilot on a subset of patients.

“Common methods of prospective validation included conducting a silent trial, comparing the model outcomes to patient charts in the medical record, and conducting a pilot on a subset of patients.”

May combine two claims.

In communications leading up to clinical integration, it was critical to explain short-term and long-term benefits of using the tool and to answer potential questions from end users.

“In communications leading up to clinical integration, it was also critical to explain the short-term and long-term benefits of using the tool and to answer potential questions from end users.”

Classified as probably not a substantial claim.

Participants emphasized the importance of well-defined, narrow intended uses of AI.

“Despite the potential benefits of using AI, participants emphasized the importance of well defined, narrow intended uses of AI.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants reported that assessing the quality of data used to train and validate the model was an important step for ensuring the safety and efficacy of the model.

“The participants reported that assessing the quality of data used to train and validate the model was an important step for ensuring the safety and the efficacy of the model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants described that the launch of the tool was disseminated via various communication platforms, including email and website, depending on the target audience.

“The participants described that the launch of the tool was disseminated via various communication platforms, including email and website, depending on the target audience.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants highlighted the importance of recognizing changes caused by AI implementation and providing necessary support to front-line workers reconciling these changes.

“Given that implementation of the AI software would create inevitable changes in the existing workflow, they also highlighted the importance of recognizing the changes and providing necessary support to front-line workers reconciling these changes.”

• AssumptionRests on an assumption: Implementation of AI software would create inevitable changes in the existing workflow.

Classified as probably not a substantial claim. May combine two claims.

To mitigate risks, participants suggested organizations should deeply understand regulatory requirements and work with a group of people with expertise in the full breadth of relevant domains when evaluating the model.

“To mitigate these risks, participants suggested that organizations should deeply understand regulatory requirements and work with a group of people with expertise in the full breadth of relevant domains when evaluating the model.”

Classified as probably not a substantial claim.

Participants pointed out that business administrators who do not have sufficient knowledge in patient care were typically the ones who evaluated AI algorithms.

“The participants pointed out that business administrators who do not have sufficient knowledge in patient care were typically the ones who evaluated AI algorithms.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

If communicating directly with end users was not feasible, developers should at least speak with frontline clinicians or clinical leaders who were trusted by the end users.

“If communicating directly with the end users was not feasible, the developers should at least speak with frontline clinicians or clinical leaders who were trusted by the end users.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

One participant suggested that simulation-based training would be the most effective method of education and training.

“One participant suggested that simulation-based training would be the most effective method of education and training.”

May combine two claims.

After evaluating safety, efficacy, and equity concerns, organizations decide to move forward with adopting AI software or to stop the process.

“After evaluating the safety, efficacy, and equity concerns of the model, the organizations decide to move forward with adopting the AI software or to stop the process.”

Probably not a substantial claim: may not pass S3 (load-bearing).

After evaluation, an organization decides to clinically integrate or abandon the AI software.

“After the evaluation, an organizations decides to clinically integrate or abandon the AI software.”

Classified as probably not a substantial claim.

Participants shared that having a standardized structure or guideline for risk assessment and clinical integration decisions would be helpful for making informed decisions.

“The participants shared that having a standardized structure or guideline for risk assessment and clinical integration decisions would be helpful for making informed decisions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants highlighted the importance of understanding the context in which problems were identified, especially when organizational leaders raised the problems.

“The participants highlighted the importance of understanding the context in which the problems were identified, especially when the organizational leaders raised the problems.”

May combine two claims.

Participants noted that organizations were frequently approached by vendors, and there could be external pressures to procure AI software.

“The participants also noted that the organizations were frequently approached by vendors, and there could be external pressures to procure AI software.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The factors were grouped into 8 thematic areas that described key decision points across the entire process of adopting AI.

“The factors were grouped into 8 thematic areas that described key decision points across the entire process of adopting AI.”

Classified as probably not a substantial claim. May combine two claims.

It was critical that end users of the AI software were involved in developing the new workflow and provided context into the current state.

“As algorithm developers tend to have limited insight into an existing workflow, it was critical that end users of the AI software were involved in the process of developing the new workflow and provided context into the current state.”

• AssumptionRests on an assumption: Algorithm developers tend to have limited insight into an existing workflow.

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants proposed to design the new workflow in line with the existing one to minimize the changes made in implementation.

“The participants proposed to design the new workflow in line with the existing one to minimize the changes made in implementation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For AI software to cause minimal friction, designing an intuitive user interface was an important component for optimizing technical aspects of workflow.

“For the AI software to cause minimal friction, designing an intuitive user interface (UI) was also an important component for optimizing technical aspects of workflow.”

May combine two claims.

Challenges in aligning affected parties included identifying key affected parties and aligning different priorities across all affected parties.

“Challenges in aligning affected parties included identifying key affected parties who should be involved in making the decision and aligning different priorities across all affected parties.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Once the model was ready for clinical integration, participants proposed to implement it incrementally in clinical settings to minimize potential harm in patient care caused by unforeseen safety issues.

“Then, once the model was ready for clinical integration, participants proposed to implement it incrementally in clinical settings to minimize the potential harm in patient care that could be caused by unforeseen safety issues.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants noted that model validation could be a time-consuming process, which could take many months.

“The participants noted that model validation could be a time-consuming process, which could take many months.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The target audience of the communication included end users of the tool, those who would oversee change management, and those who would be impacted by the change.

“The target audience of the communication included not only end users of the tool but also those who would oversee change management and those who would be impacted by the change.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants reported that it is important to verify whether AI is indeed an appropriate solution to a problem.

“The participants also reported that it is important to verify whether AI is indeed an appropriate solution to a problem.”

May combine two claims. The quote only partly states this claim.

The participants reported two dominant ways problems were initially identified as meriting investment: frontline workers surfaced daily pain points, or organizational leadership led problem identification.

“The participants reported two dominant ways problems were initially identified as meriting investment.”

May combine two claims.

When valid AI software already existed in the market or organizations lacked resources to build, participants felt motivated to buy AI from external vendors.

“On the other hand, when the valid AI software already existed in the market or the organizations lacked the resources to build, including data for validation and internal expertise, the participants felt motivated to buy AI from external vendors.”

Classified as probably not a substantial claim.

Validating the model on real-time data prior to clinical integration was seen as a key component of assessing the safety, efficacy, and equity concerns of the model.

“Validating the model on real-time data prior to clinical integration was seen as a key component of assessing the safety, efficacy, and equity concerns of the model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants agreed that many operational and clinical parties have heard of AI but do not have much knowledge in it.

“Participants agreed that many operational and clinical parties have heard of AI but do not have much knowledge in it.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Outcomes commonly captured in success metrics included improvements in patient outcomes, reductions in burden on healthcare providers, and reductions in cost.

“Outcomes commonly captured in the success metrics included improvements in patient outcomes, reductions in burden on healthcare providers, and reductions in cost.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants suggested that being aware of the potential risks would be the first step.

“They suggested that being aware of the potential risks would be the first step.”

May combine two claims.

Participants described how assessments of requirements and feasibility were often inefficient, with resources and capabilities decentralized and duplicated across the organization.

“Yet, the participants described how these assessments were often inefficient, with resources and capabilities decentralized and duplicated across the organization.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants reported that organizations often decide not to move forward with adoption when the model was no longer valuable, not implementable, not interoperable, too expensive, or unsuccessful in a pilot launch.

“The participants reported that organizations often decide not to move forward with the adoption process when the model was no longer valuable, not implementable, not interoperable, too expensive, or unsuccessful in a pilot launch.”

Classified as probably not a substantial claim.

Participants reported that in making decisions to develop or adapt AI software, all affected parties from various disciplines must be aligned.

“The participants reported that in making decisions to develop or adapt AI software, all affected parties from various disciplines, including AI specific committees and specialists in clinical, operational, and technical areas, must be aligned.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Simplicity in the user interface could be achieved by standardizing the visualization of the model, conducting usability tests with users, and using technology that was already familiar to users.

“Simplicity in the user interface could be achieved by standardizing the visualization of the model, conducting usability tests with users, and using technology that was already familiar to users.”

Probably not a substantial claim: may not pass S2 (checkable).

The governance body should work closely with frontline workers knowledgeable and experienced in clinical care and community members to promote equity in healthcare.

“For instance, the governance body should work closely with frontline workers who are knowledgeable and experienced in clinical care and community members to promote equity in healthcare.”

Classified as probably not a substantial claim.

Participants emphasized that clinical personnel must be part of the evaluation process.

“To ensure the safe, effective, and equitable use of AI software, participants emphasized that clinical personnel must be part of the evaluation process.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many participants recommended validating the model on underserved populations as well as a large representative sample of populations to ensure equal model performance across different groups of patients.

“Many also recommended validating the model on underserved populations as well as a large representative sample of populations to ensure equal model performance across different groups of patients.”

Classified as probably not a substantial claim. May combine two claims.

Participants raised a need for a standardized structure of quality assurance and active communication across organizations.

“Given the lack of documentation and the amount of due diligence required, the participants raised a need for a standardized structure of quality assurance and active communication across organizations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants described the need to tailor education and training to the target audience, their role, and the scale of change management.

“Participants described the need to tailor education and training to the target audience, their role, and the scale of change management.”

Classified as probably not a substantial claim. May combine two claims.

To ensure safe, effective, and equitable use of AI software, participants indicated organizations should be aware of potential risks and have plans to mitigate them from early on.

“To ensure the safe, effective, and equitable use of AI software, the participants indicated that organizations should be aware of the potential risks and have plans to mitigate them from early on.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Involving members of underserved communities from the early stage of problem identification was raised as an aspirational and impactful practice.

“Involving members of underserved communities from the early stage of problem identification was raised as an aspirational and impactful practice.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants suggested making algorithm developers have conversations with end users to provide transparent explanations about the model with trustworthy data and make them familiar with AI and its use.

“The first approach was to make algorithm developers have conversations with end users to provide transparent explanations about the model with trustworthy data and make them become familiar with AI and its use.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Common practices of data quality assurance were ensuring minimal bias in data, creating a diverse data set for training the model, and cleaning data before building the model.

“Some of the common practices of data quality assurance were ensuring that there was minimal bias present in data, creating a diverse data set for training the model, and cleaning data before building the model.”

Classified as probably not a substantial claim.

It was important to perform thorough data quality assurance, even though it required significant resources.

“Given that healthcare data is messy and may contain bias, it was important to perform thorough data quality assurance, even though it required significant resources.”

• AssumptionRests on an assumption: Healthcare data is messy and may contain bias.

Probably not a substantial claim: may not pass S3 (load-bearing).

Gaining trust from frontline clinicians or clinical leaders was found to be an efficient way to slowly build trust with the end user community.

“Gaining trust from them was found to be an efficient way to slowly build trust with the end user community.”

Probably not a substantial claim: may not pass S3 (load-bearing).

When all interested parties were satisfied with the model, they decided to integrate the model into clinical practice.

“When all interested parties were satisfied with the model, they decided to integrate the model into clinical practice.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Model validation was an essential process, especially when testing model accuracy on retrospective labels might not be sufficient due to inaccuracies in ground truth labels.

“Yet, it was an essential process to go through, especially when testing model accuracy on retrospective labels might not be sufficient for model validation due to inaccuracies in ground truth labels.”

Classified as probably not a substantial claim. May combine two claims.

Participants suggested that a decision to clinically integrate or abandon the model should be made based on evidence supported by data and that organizations should be open to making an abandonment decision at any given point.

“The participants suggested that a decision to clinically integrate or abandon the model should be made based on evidence supported by data and that the organizations should be open to making an abandonment decision at any given point.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A commonly described unique contribution of AI was the use of data to promote efficiency in clinical care.

“Yet, a commonly described unique contribution of AI was the use of data to promote efficiency in clinical care.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants reported that the biggest challenge in operational monitoring was staff turnover.

“The participants reported that the biggest challenge in operational monitoring was staff turnover.”

• AssumptionRests on an assumption: Staff turnover happens frequently in clinical care.

May combine two claims.

Organizations conduct a comprehensive review of data and evaluate AI software before routinely using it in clinical practice.

“This thematic area describes how the organizations conduct a comprehensive review of data & evaluate the AI software before they routinely use the software in clinical practice.”

Probably not a substantial claim: may not pass S3 (load-bearing).

If disparities were detected in model performance among different patient populations, participants suggested addressing these issues by adjusting thresholds based on the disparities in model performance.

“If there were any disparities detected in the model performance among different patient populations, participants suggested addressing these issues by adjusting thresholds based on the disparities in the model performance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In defining model metrics, it was seen to be important to ensure that the model performs similarly on subgroups stratified by important demographic factors so that patients receive equitable care.

“In defining the model metrics, it was seen to be important to ensure that the model performs similarly on subgroups stratified by important demographic factors so that patients receive equitable care.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants suggested showing end users how clinically valuable it would be to use the tool by showing them real-world evidence from the local context.

“The second approach was to show end users how clinically valuable it would be to use the tool by showing them real-world evidence from the local context.”

Classified as probably not a substantial claim. May combine two claims.

The results identified challenges and risks that organizations experience in the process of AI adoption as well as real-world processes and practices.

“The results identified challenges and risks that organizations experience in the process of AI adoption as well as real-world processes and practices.”

Probably not a substantial claim: may not pass S3 (load-bearing).

From a technical standpoint, participants proposed to set clear eligibility criteria and model performance thresholds for a given model.

“From a technical standpoint, the participants proposed to set clear eligibility criteria and model performance thresholds for a given model.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

All safety, effectiveness, and equity concerns should be taken into consideration throughout the entire process of AI adoption.

“While prospective model validation takes place once the model is integrated into IT infrastructure, it is important to note that all other safety, effectiveness, and equity concerns should be taken into consideration throughout the entire process of AI adoption.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants described many instances in which software with simple decision logic can be a viable solution with greater transparency and lower resource requirements than AI or machine learning.

“Participants described many instances in which software with simple decision logic can be a viable solution with greater transparency and lower resource requirements than AI or machine learning.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Education and training materials were disseminated in various ways, frequently including online learning management systems and webinars.

“Education and training materials were disseminated in various ways, frequently including online learning management systems and webinars.”

May combine two claims.

Bi-directional communication between developers and users was seen as key, both to allow adaptations needed to accommodate end users' needs and to build a sense of accountability.

“Bi-directional communication between developers and users was seen as key, both to allow adaptations needed to accommodate end users' needs on the ground and to build a sense of accountability that AI developers were listening to end users and respecting their agency.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants suggested that if the model drifts, the entire team of technical, clinical, and operational parties should get together to potentially retune the model and continue monitoring and revalidating the model over time.

“They suggested that if the model drifts, the entire team of technical, clinical, and operational parties should get together to potentially retune the model and continue monitoring and revalidating the model over time.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants suggested that for successful model validation, algorithm developers should collaborate closely with clinical partners.

“The participants suggested that for successful model validation, algorithm developers should collaborate closely with clinical partners.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A primary benefit of building AI internally was having full transparency and visibility into the model development process.

“A primary benefit of building AI internally was having full transparency & visibility into the model development process.”

May combine two claims.

Cybersecurity risk, bias and disparities in care, and unexpected harms to patients were the most prevalent risks that participants identified.

“Cybersecurity risk, bias and disparities in care, and unexpected harms to patients were the most prevalent risks that the participants identified.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants recommended centralizing roles and capabilities to think broadly about the optimal use of AI within the organization.

“To enhance efficiencies, participants recommended centralizing roles and capabilities to think broadly about the optimal use of AI within the organization.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants suggested giving end users agency and engaging them from early stages of the AI software adoption process.

“Another approach was to give end users agency and engage them from early stages of the AI software adoption process.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One challenge raised was how to engage clinicians, particularly end users, as product owners, as they were often stretched between too many responsibilities already.

“One challenge that they raised was how to engage clinicians, particularly end users, as product owners, as they were often stretched between too many responsibilities already.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Monitoring patient outcomes could address a limitation of technical monitoring when model performance might not be a reliable measure for patients receiving appropriate care.

“For example, monitoring patient outcomes could address a limitation of technical monitoring when model performance might not be a reliable measure for patients receiving appropriate care.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the second stage of AI adoption ends with a decision to use the algorithm in patient care.

“The second stage starts with developing an algorithm internally or adapting an externally developed one for internal use and ends with a decision to use the algorithm in patient care.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the second stage of AI adoption starts with developing an algorithm internally or adapting an externally developed one for internal use.

“The second stage starts with developing an algorithm internally or adapting an externally developed one for internal use and ends with a decision to use the algorithm in patient care.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants recommended conducting user studies and quantitative analyses to understand potential problems to address before deciding on model updates.

“Participants also recommended conducting user studies and quantitative analyses to understand potential problems to address before deciding on model updates.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants emphasized the importance of creating new roles dedicated to overseeing operational monitoring.

“Participants also emphasized the importance of creating new roles dedicated to overseeing operational monitoring.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the first stage of AI adoption starts with identifying a pain point.

“The first stage starts with identifying a pain point and ends with allocating a budget to pursue an opportunity.”

Classified as probably not a substantial claim.

Participants often recommended a centralized group with dedicated resources be accountable for monitoring the AI software.

“Participants often recommended a centralized group with dedicated resources be accountable for monitoring the AI software.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the third stage of AI adoption starts with a rollout of the algorithm, including the associated training, communication, and change management.

“The third stage starts with a rollout of the algorithm, including the associated training, communication, and change management, and ends with a decision to make the algorithm use standard operations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants noted that when a serious safety issue is identified, even with little notice, a decision to decommission the model should be made urgently.

“However, they noted that when a serious safety issue is identified, even with little notice, a decision to decommission the model should be made urgently.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Algorithmic brokers working for AI vendors were viewed as playing a more pernicious role.

“Algorithmic brokers working for AI vendors, however, were viewed as playing a more pernicious role.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants appreciated that often a model cannot be simply retrained, because the prior clinical integration changed the environment.

“They appreciated that often a model cannot be simply retrained, because the prior clinical integration changed the environment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the fourth stage of AI adoption ends with a decision to either continue using the algorithm, update the algorithm, or decommission the algorithm.

“The fourth stage starts with the algorithm used in steady state and ends with a decision to either continue using the algorithm, update the algorithm, or decommission the algorithm.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants reported that they would consider updating the model when changes in clinical care, intervention outcome, and IT environment were detected.

“The participants reported that they would consider updating the model when changes in clinical care, intervention outcome, and IT environment were detected.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

This project is the first multi-organizational collaboration with leaders from a range of major health systems across the United States to investigate organizational governance processes of AI adoption.

“To our knowledge, this project is the first multi-organizational collaboration with leaders from a range of major health systems across the United States to investigate organizational governance processes of AI adoption.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the third stage of AI adoption ends with a decision to make the algorithm use standard operations.

“The third stage starts with a rollout of the algorithm, including the associated training, communication, and change management, and ends with a decision to make the algorithm use standard operations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants reported that while it would not be practical to constantly have clinicians monitor model performance, periodic review of AI software monitoring and its use was important.

“The participants reported that while it would not be practical to constantly have clinicians monitor model performance, periodic review of AI software monitoring and its use was important.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants noted that frequent monitoring would require significant resources, as more models were put into practice.

“They noted that frequent monitoring would require significant resources, as more models were put into practice.”

May combine two claims.

This articulation work was viewed as essential to success but difficult to fully identify prior to clinical integration.

“This articulation work was viewed as essential to success but difficult to fully identify prior to clinical integration.”

Classified as probably not a substantial claim. May combine two claims.

Our interviewees underscored the need for evaluation and integration methods that foster calibrated clinician trust, while emphasizing that AI must be viewed as a trustworthy but not infallible source of information.

“Specifically, our interviewees underscored the need for evaluation and integration methods that foster calibrated clinician trust, while emphasizing that AI must be viewed as a trustworthy but not infallible source of information.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Monitoring staff satisfaction with the tool would inform why end users might use or not use the tool and identify areas of improvement.

“Monitoring staff satisfaction with the tool would inform why end users might use or not use the tool and identify areas of improvement.”

Classified as probably not a substantial claim. May combine two claims.

The current research identifies missing resources and suggests that a range of new professionals and practices are considered necessary to produce knowledge that informs the actions taken by clinicians.

“The current research identifies missing resources and suggests that a range of new professionals and practices are considered necessary to produce knowledge that informs the actions taken by clinicians.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

Participants proposed that joint accountability between clinical, statistical, and technical providers would be ideal for continued use of the model.

“They proposed that joint accountability between clinical, statistical, and technical providers would be ideal for continued use of the model.”

May combine two claims.

Interviewees described that the introduction of AI systems may require the creation of new teams to interact with or monitor the system, new communication strategies to maintain professional boundaries, and new expertise.

“Interviewees described that the introduction of AI systems may require the creation of new teams to interact with or monitor the system, new communication strategies to maintain professional boundaries, and new expertise.”

Probably not a substantial claim: may not pass S3 (load-bearing).

To make the monitoring process more feasible, participants suggested the creation of channels for users to report concerns related to the models and potential adverse events.

“To make the monitoring process more feasible, the participants suggested the creation of channels for users where they could report any concerns related to the models and potential adverse events that happen in clinical care.”

May combine two claims.

Overall, we found that health systems are developing individual institutional practices and principles to guide their process of AI adoption but acknowledge an urgent need for a move towards shared standards.

“Overall, we found that health systems are developing individual institutional practices and principles to guide their process of AI adoption but acknowledge an urgent need for a move towards shared standards.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the fourth stage of AI adoption starts with the algorithm used in steady state.

“The fourth stage starts with the algorithm used in steady state and ends with a decision to either continue using the algorithm, update the algorithm, or decommission the algorithm.”

Classified as probably not a substantial claim.

Participants reported that there was no clear ownership of lifecycle management activities.

“The participants reported that there was no clear ownership of lifecycle management activities.”

May combine two claims.

Participants reported that they would consider decommissioning a model if the model performed poorly, when safety concerns emerged, when the model would be replaced by a newly available better model, when there was no value in the clinical use case, and when the model was no longer financially supported.

“Participants reported that they would consider decommissioning a model if the model performed poorly, when safety concerns emerged, when the model would be replaced by a newly available better model, when there was no value in the clinical use case, and when the model was no longer financially supported.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Outcomes were measured across various domains, including patient health outcomes, equity in patient care, financial outcomes, and staff satisfaction with the tool.

“Outcomes were measured across various domains, including patient health outcomes, equity in patient care, financial outcomes, and staff satisfaction with the tool.”

Probably not a substantial claim: may not pass S3 (load-bearing).

To mitigate operational monitoring challenges, participants suggested creating robust documentation standards and governance rubrics for every tool clinically integrated.

“To mitigate these challenges, the participants suggested creating robust documentation standards and governance rubrics for every tool clinically integrated.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants emphasized that additional monitoring might be required when any updates or unexpected events took place in clinical settings.

“The participants emphasized that additional monitoring might be required when any updates or unexpected events (e.g.implementation of new medical equipment within the relevant workflow) took place in clinical settings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In healthcare, the first stage of AI adoption ends with allocating a budget to pursue an opportunity.

“The first stage starts with identifying a pain point and ends with allocating a budget to pursue an opportunity.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All participants provided verbal consent to be interviewed and to have anonymized data used in qualitative analyses.

“All participants provided verbal consent to be interviewed and to have anonymized data used in qualitative analyses.”

May combine two claims.

Health systems are increasingly called upon to enhance organizational governance of algorithms, but concrete guidance and capacity building programs are largely missing.

“Health systems are increasingly called upon to enhance organizational governance of algorithms, but concrete guidance and capacity building programs are largely missing [11].”

• cites [11]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Data collection was terminated when no new themes or insights emerged (thematic saturation).

“We analyzed data alongside the recruitment and decided to terminate data collection when no new themes or insights emerged (i.e.thematic saturation).”

May combine two claims.

Taken together, our findings show how health systems currently make decisions to adopt AI into clinical practice and are in need of shared approaches to governing the process.

“Taken together, our findings show how health systems currently make decisions to adopt AI into clinical practice and are in need of shared approaches to governing the process.”

Probably not a substantial claim: may not pass S2 (checkable).

Achieving a healthy governance ecosystem requires a process to align and expand the competencies of the existing actors-regulators and standard setting bodies-in a manner that best furthers the policy goals of safety, efficacy and equity.

“Achieving a healthy governance ecosystem requires a process to align and expand the competencies of the existing actors-regulators and standard setting bodies-in a manner that best furthers the policy goals of safety, efficacy and equity.”

May combine two claims.

Burden shifting is inefficient as health systems assessing the same AI products must engage in duplicate work and less well-resourced health systems will be further disadvantaged.

“This burden shifting is wildly inefficient as health systems assessing the same AI products must engage in duplicate work and less well-resourced health systems will be further disadvantaged in their struggle to invest in adequate assessment and validation capabilities.”

May combine two claims.

The absence of shared frameworks impedes adoption of AI and undermines safety, efficacy, and equity.

“Not only does the absence of shared frameworks create social and economic costs on individual health systems, but it also impedes adoption of AI and undermines safety, efficacy, and equity.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Even better resourced health systems will eventually shift away from engaging with external vendors to prioritize internally developed AI products because it allows them to better assess and manage risk.

“Furthermore, even better resourced health systems will eventually shift away from engaging with external vendors, in order to prioritize internally developed AI products because it allows them to better assess and manage risk.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Developing documentation and explanatory artifacts of the AI model and data may build clinician knowledge throughout the integration process.

“Developing documentation and explanatory artifacts of the AI model and data may build clinician knowledge throughout the integration process.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Health system organizations must exchange information, foster dialogue, and contribute best practices that complement and strengthen a rapidly evolving regulatory landscape.

“Health system organizations must exchange information, foster dialogue, and contribute best practices that complement and strengthen a rapidly evolving regulatory landscape.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Respecting clinician's agency and expertise may reduce unwarranted skepticism about AI outputs.

“Respecting clinician's agency and expertise may reduce unwarranted skepticism about AI outputs.”

Probably not a substantial claim: may not pass S2 (checkable).

Governance processes are required at the organizational, not algorithmic, level for analyzing the quality of AI and clinical outcomes produced by socio-technical systems reconfigured by AI.

“Governance processes are required at the organizational, not algorithmic, level for analyzing the quality of AI and clinical outcomes produced by socio-technical systems reconfigured by AI.”

Probably not a substantial claim: may not pass S2 (checkable).

Successfully governing AI requires a detailed understanding of the competencies and capacities of different actors.

“Successfully governing AI requires a detailed understanding of the competencies and capacities of different actors.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Each health system must develop governance frameworks along with assigning responsibilities to entities with the independence, expertise, and operational capacity.

“As an important building block to produce organizational governance, each health system must develop governance frameworks along with assigning responsibilities to entities with the independence, expertise, and operational capacity.”

Probably not a substantial claim: may not pass S2 (checkable).

Our research identified an immediate opportunity to improve organizational governance of AI in healthcare.

“Our research identified an immediate opportunity to improve organizational governance of AI in healthcare.”