The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; none of the 32 sources its claims cite is checked for support yet.
Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors
Open accessAbstract
Objective: The objective was to identify barriers and facilitators to the implementation of artificial intelligence (AI) applications in clinical radiology in The Netherlands. Materials and methods: Using an embedded multiple case study, an exploratory, qualitative research design was followed. Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals. The analysis of barriers and facilitators was guided by the recently published Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework for new medical technologies in healthcare organizations. Results: Among the most important facilitating factors for implementation were the following: (i) pressure for cost containment in the Dutch healthcare system, (ii) high expectations of AI’s potential added value, (iii) presence of hospital-wide innovation strategies, and (iv) presence of a “local champion.” Among the most prominent hindering factors were the following: (i) inconsistent technical performance of AI applications, (ii) unstructured implementation processes, (iii) uncertain added value for clinical practice of AI applications, and (iv) large variance in acceptance and trust of direct (the radiologists) and indirect (the referring clinicians) adopters. Conclusion: In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications. Key Points: • Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians. • Implementation of AI in radiology is facilitated by the presence of a local champion. • Evidence on the clinical added value of AI in radiology is needed for successful implementation.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This qualitative multiple-case study in seven Dutch hospitals identifies facilitators and barriers to implementing AI applications in radiology using the NASSS framework. It finds that local champions, cost pressure, expected benefits, and innovation strategies facilitate implementation, while inconsistent technical performance, unstructured implementation, uncertain added value, and variable acceptance hinder it.
Key findings
- Facilitating factors included expected added benefit for diagnostic practice (18 interviewees), expected operational benefits (17), easy integration into PACS (15), minimizing workflow changes (14), and local champions (10); hospital-wide innovation strategies were present in 4/7 hospitals and innovation managers in 3/7.
- Hindering factors included uncertain funding (14 interviewees), inconsistent acceptance/trust among radiologists (14), inconsistent technical performance (10), doubting quality and safety (10), technical knowledge necessary (10), unstructured planning and monitoring (10), unstructured workflow implementation (9), and no empirical evidence on AI app validation (9).
- Acceptance and trust varied among direct adopters (radiologists) and indirect adopters (referring clinicians); in three hospitals, referring clinicians did not trust the AI output and redid manual bone age analysis for every scan.
- BoneXpert was the only AI application in clinical use across several Dutch hospitals; its smooth PACS integration was cited by 10 interviewees, but inconsistent acceptance of BoneXpert was also cited by 10.
Methods
- Embedded multiple case study with an exploratory qualitative design, investigating seven Dutch hospitals and focusing on departments using BoneXpert, a commercial software-only medical device for automated bone maturity assessment from pediatric hand X-rays.
- Data collection consisted of 24 semi-structured interviews from February to June 2019—20 case interviews across the seven hospitals plus 4 key informant interviews—complemented by document analysis; interviews lasted 20–80 min, 21 were face-to-face and 3 by telephone, and thematic saturation was reached.
- Analysis was guided by the NASSS framework across seven domains; concepts from interviews were compared with the original framework, which was refined for AI in radiology, especially the adopter system with local champions, radiologists as direct adopters, and referring clinicians as indirect adopters.
Limitations
- The paper states that the research focuses only on AI applications in Dutch radiology departments and cannot be generalized to other healthcare systems; varied interviewee roles and positions limit generalizability to other hospitals in The Netherlands and beyond.
- The paper states that interviewees varied in experience and knowledge because AI implementation in radiology is at an early stage, so the sample may be biased toward individuals with particular interest and above-average positive attitudes toward AI.
- The paper states that further research should examine applications with higher technical complexity than BoneXpert and representing a larger part of radiologists’ diagnostic work; evident from the text, the study relies on qualitative interviews and documents, with no quantitative measurement of clinical or organizational outcomes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1 The NASSS framework [20], specified for AI applications in clinical radiology in The Netherlands](/paper-figures/3ddd0db2c682403dab1a2a322163de05/implementation-of-artificial-intelligence-ai-applications-in-radiology-hindering-figure-1.jpg)
Verification
Claims the paper makes
25 Own result · 16 Not checked yet · 13 No basis given · 7 Own interpretation · 4 Own method
None of the sources cited for this claim have been checked yet.
In the paper
“First, pressure on healthcare budgets stimulates Dutch hospitals and radiology departments to develop and implement innovative technologies that promise efficiency and/or quality gains.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“although there is little empirical evidence, radiologists, members of hospital management, and technology developers expect a large added value of the AI applications in clinical practice.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in order for AI applications to be perceived as user-friendly by radiologists, they need to be easily integrated into existing IT systems used by radiologists, such as picture archiving and communication systems (PACSs).”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such leadership is also manifested by the presence or absence of a designated innovation manager (present in three of the seven cases).”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“interviewees mentioned the need to reframe their professional identity and responsibilities as a consequence of the arrival of AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, just like the radiologists, the referring clinicians showed varying levels of acceptance of AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“our interviewees expressed concerns about the unresolved question of legal responsibility for damage occurred due to e.g. false negatives and false positives resulting from an AI-generated diagnosis [34].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
What the sources say
- Not checked yetcites [34] Neri E, Coppola F, Miele V, Bibbolino C, Grassi R (2020) Artificial intelligence: who is responsible for the diagnosis? Radiol Med 0123456789. https://doi.org/10.1007/s11547-020-01135-9 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“lacking jurisprudence from the European General Data Protection Regulation and the new Medical Device Regulation (MDR), which will come into effect in May 2020, leads to several regulatory and legal uncertainties for AI applications in radiology [32, 33].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
What the sources say
- Not checked yetcites [32] Parikh RB, Obermeyer Z, Navathe AS (2019) Regulation of predictive analytics in medicine. Science 363(6429):810–812. https://doi.org/10.1126/science.aaw0029 source
- Not checked yetcites [33] Tsang L, Kracov DA, Mulryne J et al (2017) The impact of artificial intelligence on medical innovation in the European Union and United States. Intellect Prop Technol Law J 29(8):3–12
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Interestingly, none of the interviewees expressed fear of being replaced by AI.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To overcome resistance by radiologists, developers and hospital management are framing AI applications as “co-pilots” enabling radiologists to perform better while staying in control.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In four of the seven cases, a hospital-wide innovation strategy including AI was present, reflecting innovation leadership among the hospital management.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plainn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, concerns remain about the integration of other AI applications into the PACSs.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“users perceived the technical performance of most AI applications as inconsistent.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the failure to include all relevant stakeholders in the planning, execution, and monitoring phase of the implementation of AI applications was found to be a major hindering factor.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found local champions to play a crucial role in overcoming lacking acceptance of technology users.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“in all cases, the work done to monitor existing practices or the impact of the implementation of novel technologies on the level of the hospital is currently limited.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on the empirical findings, the NASSS framework was adapted and refined for the case of the implementation of AI applications in radiology.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Radiologists’ perspectives on AI applications range from outright enthusiasm to curiosity, skepticism, and fear [31].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
What the sources say
- Not checked yetcites [31] Pinto dos Santos D, Giese D, Brodehl S et al (2019) Medical students’ attitude towards artificial intelligence: a multicentre survey. Eur Radiol 29(4):1640–1646. https://doi.org/10.1007/s00330-018-5601-1 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The main adaptation of the original NASSS model to form the NASSS for AI in radiology concerns the “adopter system” (Fig. 1). It is elaborated with local champions, radiologists as direct adopters of the technology, and referring clinicians as indirect adopters.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“planning and monitoring of AI implementation tend to be unstructured.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the acceptance and trust of direct (radiologists) and indirect adopters (referring clinicians) in AI applications differ greatly.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“funding for AI applications is uncertain due to the lacking evidence on the added value of AI applications needed to back adoption and funding decisions.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the Radiological Society of The Netherlands (NVvR) serves as a knowledge-exchange platform among its members facilitating the implementation of AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These differences in opinion across radiologists were also visible for BoneXpert.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The interviews showed that computer science and programming knowledge required in the development of AI algorithms are not present-day competencies of radiologists.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The unstructured nature of implementation processes can be explained by the absence of official guidelines or best practices.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We identified lacking acceptance as one of the most important causes for non-adoption, abandonment, and thus a barrier to the successful implementation of AI applications in radiology.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Earlier forms of AI applications, such as the first computerized aided diagnosis (CAD) systems, have failed to achieve widespread adoption.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Abstract > Key Points
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 28
- Probably not a substantial claim. 13
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The interviewees mentioned two main benefits: (1) improved diagnostic practice due to more precise and objective diagnoses, avoidance of mistakes, and the automation of cumbersome tasks; and (2) operational benefits, such as diminishing workloads, time-saving, more consistent reporting across radiologists, and advanced service availability.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The interviewees mentioned two main benefits: (1) improved diagnostic practice due to more precise and objective diagnoses, avoidance of mistakes, and the automation of cumbersome tasks; and (2) operational benefits, such as diminishing workloads, time-saving, more consistent reporting across radiologists, and advanced service availability.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These local champions are radiologists that show a particularly strong interest in AI applications and usually have better than average understanding of the technical aspects of AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the benefits and costs of using AI may be unequally divided over departments, which complicates funding decisions.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“clinical benefits or organizational goals that might be achieved by using AI applications are not clearly established ex-ante and therefore hard to assess after implementation.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When evidence on the technical performance is available (such as with BoneXpert), interviewees noted that publications on the validation of the algorithms are based on laboratory rather than clinical settings.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Against a background of lacking technical understanding of AI, some radiologists are doubting the quality and safety of an application and fail to adopt or abandon AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To overcome the opposition of potential skeptical colleagues, local champions appear to follow two strategies: (1) providing general information on AI or on an AI application in particular through scientific articles, books, and presentations; and (2) promoting opportunities for experimentation with an application, e.g., by organizing showcases or installing a test version of the application.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Also, AI applications should be implemented without large changes to routines and workflow practices, i.e., by avoiding additional steps for reporting the result of the AI application.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While the possibility of AI replacing radiologists, and thereby threatening their professional identity, was extensively mentioned in many recent radiology publications [5, 9, 16, 44], none of the interviewees in this research identified with this threat.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Discussion
What the sources say
- Not checked yetcites [5] Sogani J, Allen B Jr, Dreyer K, McGinty G (2019) Artificial intelligence in radiology: the ecosystem essential to improving patient care. Clin Imaging 59(July 2019):8–11. https://doi.org/10.1016/j.clinimag.2019.08.001 source
- Not checked yetcites [9] Chockley K, Emanuel E (2016) The end of radiology? Three threats to the future practice of radiology. J Am Coll Radiol 13(12):1415–1420. https://doi.org/10.1016/j.jacr.2016.07.010 source
- Not checked yetcites [16] Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL (2018) Artificial intelligence in radiology. Nat Rev Cancer 18(8):500–510. https://doi.org/10.1038/s41568-018-0016-5 source
- Not checked yetcites [44] Nawrocki T, Maldjian PD, Slasky SE, Contractor SG (2018) Artificial intelligence and radiology: have rumors of the radiologist’s demise been greatly exaggerated? Acad Radiol 25(8):967–972. https://doi.org/10.1016/j.acra.2017.12.027 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“they envision radiologists of the future as “imaging consultants” who play an active role in an interdisciplinary patient-focused hospital environment [28].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
What the sources say
- Not checked yetcites [28] Nederlandse Vereniging voor Radiologie (2016) De Rol van de Radioloog in 2020. Utrecht. https://www.radiologen.nl/system/files/bestanden/documenten/de_rol_van_de_radioloog_in_2020.pdf. Accessed 17 Nov 2019
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“If other departments are to cover parts of the additional costs, they need to learn about the technology and the potential benefits of using AI applications. This requires efficient communication between departments, a shortcoming in several of the studied cases.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“there is a lack of empirical evidence on the effect of AI applications on the radiological workflow, as well as their added value for clinical radiology practice.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“in three hospitals, we found that the referring clinicians did not trust the output of the AI application and redid a manual bone age analysis for every scan.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“implementation plans do not specify how the AI application should be integrated into the clinical workflow, which leads to significant variations in the way the application is used in different departments.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“interviewees and document analysis [28, 29] show that local champions are vital in initiating and stimulating implementation within their department and taking the lead during the entire process.”
- Also stated in: § Conclusion · plain
“One of the important facilitating factors is the presence of a “local champion,” an individual with a strong personal interest in AI applications who most often initiates and actively advances AI implementation in the organization.”
- Also stated in: § Key Points · plain
“Implementation of AI in radiology is facilitated by the presence of a local champion.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
What the sources say
- Not checked yetcites [28] Nederlandse Vereniging voor Radiologie (2016) De Rol van de Radioloog in 2020. Utrecht. https://www.radiologen.nl/system/files/bestanden/documenten/de_rol_van_de_radioloog_in_2020.pdf. Accessed 17 Nov 2019
- Not checked yetcites [29] Nederlandse Vereniging van Ziekenhuizen (2018) Ziekenhuiszorg in Cijfers 2018. Utrecht. https://ziekenhuiszorgincijfers.nl/assets/uploads/NVZ-Brancherapport-2018.pdf
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the interviewed users experienced the integration of BoneXpert into the PACS as very smooth, being the main reason for its perceived user-friendliness.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On radiology department-level, only one hospital had a formalized innovation strategy regarding AI. However, four more hospitals were developing such a strategic approach at the time of this research.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“BoneXpert is one of the first commercial applications of AI in radiology [25] and appeared to be the only AI application in clinical use across several hospitals in The Netherlands.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
What the sources say
- Not checked yetcites [25] Lee H, Tajmir S, Lee J et al (2017) Fully automated deep learning system for bone age assessment. J Digit Imaging 30(4):427–441. https://doi.org/10.1007/s10278-017-9955-8 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The Dutch radiology departments vary strongly in the number and types of internally available AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The Dutch healthcare system is confronted with a constant rise in demand accompanied by a strong pressure to limit associated costs [27].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified facilitating factors for AI implementation in radiology
What the sources say
- Not checked yetcites [27] Ministerie van Volksgezondheid Welzijn en Sport (2018) Kamerbrief over Aanbieding Bestuurlijk Akkoord (Hoofdlijnenakkoord) Medisch-Specialistische Zorg 2019-2022. https://www.rijksoverheid.nl/documenten/kamerstukken/2018/06/04/kamerbrief-over-hoofdlijnenakkoord-medisch-specialistische-zorg-2019-2022. Accessed 21 Nov 2019
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the most prominent hindering factors is the uncertain added value for clinical practice, which causes low acceptance of AI applications among adopters and complicates the mobilization of funds to acquire AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Having a large number of false negatives is even more dangerous, because it means that a potential lesion might get overlooked.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, some technical understanding is imperative for quality and safety assessment and therefore create trust in the AI application’s reliability [8, 21, 30].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Results > Identified hindering factors for AI implementation in radiology
What the sources say
- Not checked yetcites [8] Char DS, Shah NH, Magnus D (2018) Implementing machine learning in health care ’ addressing ethical challenges. N Engl J Med 378(11):981–983. https://doi.org/10.1056/NEJMp1714229 source
- Not checked yetcites [21] Recht MP, Dewey M, Dreyer K et al (2020) Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol:1–9. https://doi.org/10.1007/s00330-020-06672-5 source
- Not checked yetcites [30] Rubin DL (2019) Artificial intelligence in imaging: the radiologist’s role. J Am Coll Radiol 16(9):1309–1317. https://doi.org/10.1016/j.jacr.2019.05.036 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Interviewees varied with regard to their related experience and knowledge, due to the early stage of implementation of AI in radiology practice. Therefore, the sample of interviewees is possibly biased towards individuals with a particular interest and above-average positive attitude towards AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Abstract > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evidence on the clinical added value of AI in radiology is needed for successful implementation.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Abstract > Key Points
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“Technology implementation in hospital settings involves a large variety of stakeholders and organizational procedures, with strong routines and professional identities, as well as strict legal and regulatory standards [18–20].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
What the sources say
- Not checked yetcites [18] Pope C, Halford S, Turnbull J, Prichard J, Calestani M, May C (2013) Using computer decision support systems in NHS emergency and urgent care: Ethnographic study using normalisation process theory. BMC Health Serv Res 13(1) source
- Not checked yetcites [19] Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O (2004) Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x source
- Not checked yetcites [20] Greenhalgh T, Wherton J, Papoutsi C et al (2017) Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 19(11):e367. https://doi.org/10.2196/jmir.8775 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Literature has mainly blamed this on low technical performance of these early applications [5, 14–16], while other potential barriers to successful implementation, such as organizational or social aspects, have been largely ignored [17].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
What the sources say
- Not checked yetcites [5] Sogani J, Allen B Jr, Dreyer K, McGinty G (2019) Artificial intelligence in radiology: the ecosystem essential to improving patient care. Clin Imaging 59(July 2019):8–11. https://doi.org/10.1016/j.clinimag.2019.08.001 source
- Not checked yetcites [14] van Ginneken B, Schaefer-Prokop CM, Prokop M (2011) Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology. 261(3):719–732. https://doi.org/10.1148/radiol.11091710 source
- Not checked yetcites [15] Kohli A, Jha S (2018) Why CAD failed in mammography. J Am Coll Radiol 15(3):535–537. https://doi.org/10.1016/j.jacr.2017.12.029 source
- Not checked yetcites [16] Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL (2018) Artificial intelligence in radiology. Nat Rev Cancer 18(8):500–510. https://doi.org/10.1038/s41568-018-0016-5 source
- Not checked yetcites [17] Nishikawa RM, Bae KT (2018) Importance of better human-computer interaction in the era of deep learning: mammography computer-aided diagnosis as a use case. J Am Coll Radiol 15(1):49–52. https://doi.org/10.1016/j.jacr.2017.08.027 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Considering that AI applications in radiology are in an emerging phase, it is too early to evaluate their implementation [21].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
What the sources say
- Not checked yetcites [21] Recht MP, Dewey M, Dreyer K et al (2020) Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol:1–9. https://doi.org/10.1007/s00330-020-06672-5 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“AI is expected to cause large changes in clinical work practices and requires complementary skills from radiologists [5, 7, 8].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
What the sources say
- Not checked yetcites [5] Sogani J, Allen B Jr, Dreyer K, McGinty G (2019) Artificial intelligence in radiology: the ecosystem essential to improving patient care. Clin Imaging 59(July 2019):8–11. https://doi.org/10.1016/j.clinimag.2019.08.001 source
- Not checked yetcites [7] Jha S, Topol EJ (2016) Adapting to artificial intelligence: Radiologists and pathologists as information specialists. JAMA 316(22):2353–2354. https://doi.org/10.1001/jama.2016.17438 source
- Not checked yetcites [8] Char DS, Shah NH, Magnus D (2018) Implementing machine learning in health care ’ addressing ethical challenges. N Engl J Med 378(11):981–983. https://doi.org/10.1056/NEJMp1714229 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Interviews were conducted until thematic saturation was reached, meaning when no new themes appeared during additional interviews.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The NASSS framework aims to detect the determinants of implementation processes of complex technologies in healthcare on seven domains: the condition, technology, value proposition, the adopter system (patient, lay caregivers, individual technology user, and other staff), the organization, and the wider institutional and social context.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
What the sources say
- Not checked yetcites [20] Greenhalgh T, Wherton J, Papoutsi C et al (2017) Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 19(11):e367. https://doi.org/10.2196/jmir.8775 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Using a maximum variability logic, interviewees occupying different positions within the participating hospitals were selected (see Table 1 and Appendix A).”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The concepts identified in the interviews were compared with the original NASSS framework, which was afterwards refined.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While technical performance of AI applications is expected to increase continuously, its implementation in clinical radiology practice is rather complex and has so far been slow [6, 12, 13].”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Introduction
What the sources say
- Not checked yetcites [6] He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K (2019) The practical implementation of artificial intelligence technologies in medicine. Nat Med 25(1):30–36. https://doi.org/10.1038/s41591-018-0307-0 source
- Not checked yetcites [12] Dreyer KJ, Geis JR (2017) When machines think: Radiology’s next frontier. Radiology. 285(3):713–718. https://doi.org/10.1148/radiol.2017171183 source
- Not checked yetcites [13] Yu KH, Kohane IS (2019) Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf 28(3):238–241. https://doi.org/10.1136/bmjqs-2018-008551 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
AI applications will encounter barriers to implementation.
“However, in view of the unsuccessful widespread diffusion of earlier CAD systems, it can be assumed that AI applications will encounter barriers to implementation.”
The paper relies on this without support.
A large number of false positives create additional work for the radiologist.
“In clinical terms, a large number of false positives create additional work for the radiologist, which was the case of earlier unsuccessful CAD applications.”
The paper relies on this without support.
Currently, CE marks are granted without requiring proof of the performance and added benefit for clinical practice.
“Currently, CE marks are granted without requiring proof of the performance and added benefit for clinical practice.”
The paper relies on this without support.
The new MDR requires CE certification through a notified body and necessitates a large increase in requirements on quality, safety, and post-market surveillance.
“The new MDR requires CE certification through a notified body and necessitates a large increase in requirements on quality, safety, and post-market surveillance.”
The paper relies on this without support.
Measuring clinical and organizational benefits of AI at a micro-level is difficult, and there is no standard methodology to measure increases in the quality of diagnosis.
“One reason is that measuring clinical and organizational benefits of AI on a micro-level is difficult. There is, for example, no standard methodology to measure increases in the quality of diagnosis.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The lack of accuracy of AI output was also considered to pose a potential risk of harm by both clinicians and patients, as, in some cases where the algorithm may output false negative results, it may provide an incorrect sense of reassurance and cause a delay in treatment.”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, in cases where the algorithm is too sensitive, thereby providing false positive results, it may add work and costs to the treatment process [39,42,57,61,68].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinicians expressed fear around having to reframe their professional identity and responsibilities [57].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The presence of AI experts, perhaps a multidisciplinary team, particularly with clinical scientists, data science and subject matter experts with AI skills, an innovation manager, AI experts to provide training, and local champions within departments involved in the end-to-end process, was considered an important element for adoption [32,57,59,62,71].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A lack of regulation and policies from the government, including uncertainty around legal direction or law, was presented as a barrier to the application of AI technologies [37,45,57,60,77,78].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
Experimental — accuracy not yet measured
The cited source says what the claim attributes to it.
In the citing paper
“Ease of use; complex interfaces; and inconsistent performance, for example, due to false positives and negatives add burden to the workflow, creating more work for clinicians, thereby impeding adoption [42,52,57-59].”
From Barriers to and facilitators of artificial intelligence adoption in health care: scoping review (2024) · cites this paper as [57]
The cited source says what the claim attributes to it.
In the citing paper
“Similar to others, they also regarded the technology with skepticism [35,41,42].”
From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [41]
The cited source says what the claim attributes to it.
In the citing paper
“Such distrust has been shown to challenge the adoption of machine learning–based CDSSs [7-10]”
From Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Discussion
“radiology residents [31]”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > References
“31. Pinto dos Santos D Giese D Brodehl S Medical students’ attitude towards artificial intelligence: a multicentre survey”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Abstract > Materials and methods
“Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals.”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
“| Key informants | | | | Professional organization | 1 | Member of management | | Professional organization | 1 | Implementation advisor | | Professional organization | 1 | Member of management | | Imaging technology provider | 1 | Innovation manager | | Total number of interviews | 24 | |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
“on seven domains”
§ Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors > Materials and methods
“the condition, technology, value proposition, the adopter system (patient, lay caregivers, individual technology user, and other staff), the organization, and the wider institutional and social context.”
Citations
Cites
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 3×
“Technology implementation in hospital settings involves a large variety of stakeholders and organizational procedures, with strong routines and professional identities, as well as strict legal and regulatory standards [18 – 20].”
BackgroundMethodInfluentialThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
Implementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857
Current Applications and Future Impact of Machine Learning in Radiology
Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P. V., Brink, J. A., & Dreyer, K. J. (2018) · Radiology · doi:10.1148/radiol.2018171820 · Cited by 800
Medical students' attitude towards artificial intelligence: a multicentre survey
Pinto dos Santos, D., Giese, D., Brodehl, S., Chon, S. H., Staab, W., Kleinert, R., Maintz, D., & Baeßler, B. (2019) · European Radiology · doi:10.1007/s00330-018-5601-1 · Cited by 754
Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists
Jha, S., & Topol, E. J. (2016) · JAMA · doi:10.1001/jama.2016.17438 · Cited by 622
Fully Automated Deep Learning System for Bone Age Assessment
Lee, H., Tajmir, S., Lee, J., Zissen, M., Yeshiwas, B. A., Alkasab, T. K., Choy, G., & Do, S. (2017) · Journal of Digital Imaging · doi:10.1007/s10278-017-9955-8 · Cited by 452
Research works printed but not linked: 1
- Tsang L, Kracov DA, Mulryne J et al (2017) The impact of artificial intelligence on medical innovation in the European Union and United States. Intellect Prop Technol Law J 29(8):3–12
Cited by
Sorted by citations
Deep learning for chest X-ray analysis: A survey
Çallı, E., Sogancioglu, E., van Ginneken, B., van Leeuwen, K. G., & Murphy, K. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102125 · Cited by 565
van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., van Ginneken, B., & de Rooij, M. (2021) · European Radiology · doi:10.1007/s00330-021-07892-z · Cited by 461
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443
Lambert, S. I., Madi, M., Sopka, S., Lenes, A., Stange, H., Buszello, C.-P., & Stephan, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00852-5 · Cited by 404
A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare
Ahmed, M. I., Spooner, B., Isherwood, J., Lane, M., Orrock, E., & Dennison, A. (2023) · Cureus · doi:10.7759/cureus.46454 · Cited by 364
Metadata
- Authors
- Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M.
- Affiliations
- Utrecht University (NL)
- University Medical Center Utrecht (NL)
- Elisabeth-TweeSteden Ziekenhuis (NL)
- Corresponding author
- Hehakaya, Charisma
- Venue
- European Radiology
- Published
- 2020-05-26
- Volume
- 30
- Issue
- 10
- Pages
- 5525-5532
- ISSN
- 0938-7994 (ISSN-L)
- 1432-1084
- PMC
- PMC7476917
- OpenAlex
- W3032075591
- Cited by
- 329
- References
- 40
- Citations by year
- 2026: 61 · 2025: 78 · 2024: 80 · 2023: 58 · 2022: 36 · 2021: 15 · 2020: 1
- FWCI
- 8.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- European Radiology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- University Medical Center Utrecht
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- artificial intelligence in radiology, NASSS framework, radiologists, referring clinicians, local champions, trust in AI, cost containment, Dutch healthcare system
- MeSH terms
- Radiologists, Artificial Intelligence, Data Collection, Humans, Netherlands, Radiography, Radiology, Program Evaluation, Program Development, Qualitative Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
