CoChat
Sign Up
Cited by 919Artificial Intelligence ApplicationsInformation Systems Theories and ImplementationInformation Technology Governance and Strategy

Artificial intelligence in information systems research: A systematic literature review and research agenda

Open access
Collins, Christopher, Dennehy, Denis, Conboy, Kieran, Mikalef, Patrick·2021·International Journal of Information Management
DOI 10.1016/j.ijinfomgt.2021.102383
Open this paper in CoChat — free accountOpen PDF

Abstract

AI has received increased attention from the information systems (IS) research community in recent years. There is, however, a growing concern that research on AI could experience a lack of cumulative building of knowledge, which has overshadowed IS research previously. This study addresses this concern, by conducting a systematic literature review of AI research in IS between 2005 and 2020. The search strategy resulted in 1877 studies, of which 98 were identified as primary studies and a synthesise of key themes that are pertinent to this study is presented. In doing so, this study makes important contributions, namely (i) an identification of the current reported business value and contributions of AI, (ii) research and practical implications on the use of AI and (iii) opportunities for future AI research in the form of a research agenda.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic literature review analyzes 98 primary studies on AI in information systems research published between 2005 and 2020, drawn from 1,877 retrieved records. It finds fragmented AI definitions, a recent publication surge, dominance of machine learning and process automation, and gaps in tools/models and cumulative knowledge.

Key findings

  • Of the 98 primary studies, 54 gave no clear AI definition; among the 44 that did, 7 cited no source, and as many as 28 definitions were used. Russel & Norvig was the most frequently cited source, though editions varied; LeCun et al. (2015) and Rai et al. (2019) each appeared twice.
  • Publication and methods: 11 studies appeared between 2005 and 2015, while over half were published in 2019 and 2020. ICIS published 41 studies, ECIS 24, and IJIM 14. Of the 98 studies, 44 used quantitative methods, 31 qualitative, 13 were literature reviews, and 8 used mixed methods; data collection included 22 data mining, 21 experiments, 20 interviews, and 22 literary analysis.
  • Contributions were largely “lessons learned” (39 studies; Fig. 5 reports 40), followed by methods (26), advice/implication (15), guidelines (6), tools (5), and models (6). Machine learning was the most common AI type (69 studies; Table 12 lists n=68), then expert systems (11), NLP (6), machine vision (5), robotics (3), and other (14).
  • Reported business value was process automation (47 studies), cognitive insight (32), and cognitive engagement (17). The research agenda calls for clearer AI definitions, more research on societal/personal impacts, more work on non-ML AI types, and greater cumulative knowledge building.

Methods

  • The review followed Okoli’s (2015) eight-step systematic literature review guide. Searches covered AIS eLibrary, Scopus, and ISI Web of Science, targeting nine IS journals and the top two IS conferences (ICIS and ECIS), using terms including AI, artificial intelligence, machine learning, neural networks, cognitive, automation, robot, and augment.
  • Screening began with 1,877 retrieved studies; two authors independently removed duplicates, non-English studies, non-IS studies, and non-peer-reviewed publications. Inclusion/exclusion criteria were applied, yielding 98 primary studies. Analysis synthesized definitions of AI, publication year, publication channel, research methods and data collection, contribution types, AI types, and business value.

Limitations

  • The paper states publication bias as a limitation because it focused on a select number of IS journals, excluding other IS conferences and non-IS outlets. It also reports addressing construct, external, internal, and conclusion validity through a structured protocol, external reviewers, an audit trail, and multiple authors, while noting conclusion bias cannot be eliminated.
  • The text makes evident that 54 of 98 studies did not define AI and 7 defined it without citation, limiting comparability and cumulative knowledge. It also states that contribution types can be context-specific, especially methods, which may slow cumulative knowledge building.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.ijinfomgt.2021.102383

Cites

204 of 204 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 53 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The journal with the most primary studies was IJIM with 14 studies.

    “The journal with the most studies was IJIM with 14 studies, especially notable as the scope for IJIM was five years in comparison to the 15 years of the other journals.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Due to the inclusion and exclusion criteria, there were no studies on AI in Information Systems in 2007, 2008, 2010, and 2012.

    “Due to the inclusion and exclusion criteria of this study, there were no studies on AI in Information Systems in the years 2007, 2008, 2010, and 2012.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The highest number of primary studies were published in ICIS, a total of 41 studies over the 15-year period.

    “The highest number of studies were published in ICIS, a total of 41 studies over the 15-year period.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The search strategy retrieved 1877 studies.

    “The search strategy resulted in 1877 studies, of which 98 were identified as primary studies and a synthesise of key themes that are pertinent to this study is presented.”
    • Also stated in: § 3.1. A systematic guide to literature review development · plain
      “The search string used across the three databases retrieved 1877 studies (shown in Table 4).”
    Quote in the paperNot a substantial claim
  • Findingown data

    2019 and 2020 show an immense surge in AI related studies in IS.

    “2019 and 2020 show an immense surge in AI related studies in IS, signifying a much greater interest in the field.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 37
  • Classified as probably not a substantial claim. 89
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data

    Studies on AI in IS remained relatively low for most of 2005-2020, with a total of 11 studies between 2005 and 2015.

    “Fig. 4. reveals that studies on AI remained relatively low for most of this period, with a total of 11 studies between the years 2005 and 2015.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    32 of the primary studies were published in journals and 65 were published in the top two IS conferences.

    “Table 7 shows that 32 of the primary studies were published in journals and 65 were published in the top two IS conferences.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Three studies used workshops.

    “Four studies adopted focus groups to collect data, with three using workshops.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The most prominent contribution type among the 98 studies was 'lessons learned' with 40 studies, closely followed by 'methods' at 26 and to a lesser extent 'advice or implication' at 15.

    “The most prominent contribution type among the 98 studies was categorised as "lessons learned" with 40 studies, closely followed by "methods" at 26 and to a lesser extent "advice or implication" at 15.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The SLR identified, classified, and analysed 1877 studies on AI and ML in IS published between 2005 and 2020.

    “By using a systematic literature review, we identified, classified, and analysed 1877 studies on AI and ML in IS that were published between 2005 and 2020.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    The definitions of artificial intelligence in the literature were largely varied and ambiguous.

    “Prior to commencing this task however, we had to consider the problem that the definitions of artificial intelligence were largely varied and ambiguous.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Literary analysis was used by 22 of the studies.

    “Literary analysis was used by 22 of the studies, and questionnaires incorporated by two.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    While there was a lack of studies in the first decade of the relevant time, there has been a resurgence in AI research in recent years.

    “Our findings show that while there was a lack of studies in the first decade of the relevant time, there has been a resurgence in recent years.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Based on title, abstract and keyword, the 1786 studies were narrowed down to 187.

    “Based on title, abstract and keyword, the 1786 studies were further narrowed down to 187.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    NLP had 6 studies.

    “There was less range between the other types, expert systems having 11, machine vision with five and NLP at 6.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    ICIS published 41 studies on AI in Information Systems, compared to 24 studies published in ECIS.

    “ICIS published 41 studies to AI in Information Systems in comparison to the 24 studies published in ECIS.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Sample analysis was used by two studies.

    “Sample analysis was used by two studies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Based on removal of duplicates, non-scientific and non-English studies, 91 studies were removed, leaving 1786.

    “Based on the removal of duplicates, non-scientific and non-English studies, 91 were removed, leaving 1786.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    IJIM published 14 primary studies, making it the most published journal regarding AI in IS journals used in the systematic review.

    “IJIM published 14 primary studies, making them the most published regarding AI in IS journals used in the systematic review.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Theory-as-discourse, Machine Learning subset selection, causal mapping and prediction markets were each used by a single study.

    “Theory-as-discourse, Machine Learning subset selection, causal mapping and prediction markets were all used by a single study each.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    There was a resurgence in interest in AI in 2017, before it expanded in 2019 and 2020.

    “As previously shown in Fig. 4., there was a resurgence in interest in AI in 2017, before it seemed to really expand in 2019 and 2020.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    This study identified a lack of cohesion when defining AI, with as many as 28 definitions being used in the respective studies.

    “This study identified a lack of cohesion when defining AI, with as many as 28 definitions being used is the respective studies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There is a trend where AI is defined more in what its capabilities are rather than strictly defining what it is.

    “Looking at the definitions gathered from the primary studies, there seems to be a trend where AI is defined more in what its capabilities are rather than strictly defining what it is.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    An in-depth examination of the 187 remaining studies resulted in a further 90 studies being excluded.

    “An in-depth examination of the 187 remaining studies was undertook by the reviewers, which resulted in a further 90 studies being excluded.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The most common definition for AI seen was derived from Russel & Norvig, followed by LeCun et al. (2015) and Rai et al. (2019) with two occurrences each.

    “The most common definition for AI seen was derived from Russel & Norvig, though the specific edition varied, followed by LeCun et al. (2015) and Rai et al. (2019) with two occurrences each.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Four studies adopted focus groups to collect data.

    “Four studies adopted focus groups to collect data, with three using workshops.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Many primary studies did not provide a definition of AI or used definitions that were not cited.

    “However, despite AI and Machine Learning being a large part of the primary studies, many did not provide a definition, or used definitions that were not cited (see Fig. 3).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Questionnaires were incorporated by two studies.

    “Literary analysis was used by 22 of the studies, and questionnaires incorporated by two.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Four studies collected data through documentation.

    “10 studies adopted observation techniques and four collected data through documentation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Machine vision had five studies.

    “There was less range between the other types, expert systems having 11, machine vision with five and NLP at 6.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingown data

    AI is seeing a small consolidation in how it is defined; as something that exhibits human intelligence, which can be seen in the definitions used by the papers in section 4.1.

    “AI is seeing a small consolidation in how it is defined; as something that exhibits human intelligence, which can be seen in the definitions used by the papers in section 4.1.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    For expert systems, there seems to be a shift away from the 'traditional' style of ES with just a few experts managing a system to a more hybrid knowledge base that uses a variety of AI systems.

    “For expert systems, there seems to be a shift away from the "traditional" style of ES with just a few experts managing a system to a more hybrid knowledge base that uses a variety of AI systems (Wagner, 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wagner, 2017

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    This point is mentioned also in Davenport and Ronanki (2018) but is largely overlooked in empirical studies.

    “This is a point that is mentioned also in the article of Davenport and Ronanki (2018) but is largely overlook in empirical studies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Davenport and Ronanki (2018)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Findingnone given

    Much current research is focused on industry-specific uses of AI, such as AI in healthcare and manufacturing.

    “The future agenda of AI seems set on the further advancement of”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Research into extensive interactions between advanced chatbots and humans is still immature.

    “Research into extensive interactions between advanced chatbots and humans is still immature, so researchers should take advantage of the increase in NLP capability and the usage of its technologies to do further research.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingnone given

    Despite interest in AI in recent years, there remain gaps in knowledge.

    “Despite the interest in AI in recent years, there remains gaps in knowledge.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    NLP papers seem focused on front facing interactions between these agents and customers, and less on actual employees working beside these agents.

    “However, they seem focused on front facing interactions between these agents and customers, and less on actual employees working beside these agents.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Researchers are increasingly using machine learning as a major component of their methodology when researching various topics.

    “Researchers are increasingly using machine learning as a major component of their methodology when researching various topics, but currently there is a gap where much of this research could further expand on how and why they are using machine learning the way they are.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Machine vision is not taking advantage of recent technological advances in machine learning, leaving a gap to see if and how machine learning could be improved.

    “Finally, machine vision doesn't seem to be taking advantage of some of the recent technological advances seen in the new studies in machine learning, so there's a gap there for researchers to see if and how machine learning could be improved.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingreasoning

    From the findings and analysis of this SLR, there is a need for a more coherent working definition of AI among academia.

    “From the findings and analysis of this SLR, it highlights a need for a more coherent working definition of AI among academia.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Findingnone given

    There is a gap where much of the research using machine learning could further expand on how and why they are using machine learning the way they are.

    “Researchers are increasingly using machine learning as a major component of their methodology when researching various topics, but currently there is a gap where much of this research could further expand on how and why they are using machine learning the way they are.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    There is a gap for researchers to consider the use of AI in other domains, such as agriculture sustainability and the public sector.

    “This leaves a gap for researchers to consider the use of AI in other domains, such as agriculture sustainability (Nishant, Kennedy, & Corbertt, 2020) and the public sector (Dennehy, 2020;Abubakar, Behravesh, Rezapouraghda, & Yildiz, 2019).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Nishant, Kennedy, & Corbertt, 2020

      Not checked in the free run.

    • cites Dennehy, 2020

      Not checked in the free run.

    • cites Abubakar, Behravesh, Rezapouraghda, & Yildiz, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Many forms of automation, machine learning and intelligent agents are labelled as AI.

    “Many forms of automation, machine learning and intelligent agents are thus labelled as AI.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingnone given

    Much of the work that has been published to date remains untheoretical.

    “Specifically, on the use of theories in studying business value, much of the work that has been published to date remains untheoretical.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Research on technophobia in robotics is lacking.

    “A specific domain that is currently lacking in research for robotics is technophobia; previous research has examined the fear of computers and have not accounted for new and evolving technologies such as robots (Sinha, Singh, Gupta, & Singh, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sinha, Singh, Gupta, & Singh, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Research into the full aspects of how service robots could potentially affect business and people is still lacking.

    “Research into the full aspects of how service robots could potentially affect business and people is still lacking.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    The future agenda of AI is set on the further advancement of weak AI where specific tasks and decisions are attributed to machines.

    “The future agenda of AI seems set on the further advancement of”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Findingnone given

    The effects of advancements in chatbots and intelligent agents still need to be studied.

    “Chatbots and intelligent agents have made great advancements in recent years, while the effects of these advancements still need to be studied.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    A clear finding from the SLR is the need to build on cumulative knowledge.

    “A clear finding emerging from this systematic literature review is the need to (i) increase the number of rigorous academic studies on AI, especially regarding tools and models, (ii) be more detailed on the definition of AI used in studies, even when it is not the focus, and (iii) build on cumulative knowledge.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Machine vision seems to be lagging in advances compared to strides made in other AI functions.

    “Machine vision seems to be lagging in advances compared to strides made in other AI functions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Chatbots and intelligent agents have made great advancements in recent years.

    “Chatbots and intelligent agents have made great advancements in recent years, while the effects of these advancements still need to be studied.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    A clear finding from the SLR is the need to increase the number of rigorous academic studies on AI, especially regarding tools and models.

    “A clear finding emerging from this systematic literature review is the need to (i) increase the number of rigorous academic studies on AI, especially regarding tools and models, (ii) be more detailed on the definition of AI used in studies, even when it is not the focus, and (iii) build on cumulative knowledge.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    There is an increase in use of machine learning as a methodology among researchers.

    “Increase in use of machine learning as a methodology among researchers.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    A clear finding from the SLR is the need to be more detailed on the definition of AI used in studies, even when it is not the focus.

    “A clear finding emerging from this systematic literature review is the need to (i) increase the number of rigorous academic studies on AI, especially regarding tools and models, (ii) be more detailed on the definition of AI used in studies, even when it is not the focus, and (iii) build on cumulative knowledge.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Effects of extended use of advanced service robots on people is still relatively undeveloped.

    “Effects of extended use of advanced service robots on people is still relatively undeveloped.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    Borges et al. (2021) focused on specific AI interactions with organisational strategy and misses some context on how AI is defined and how it creates value.

    “The review conducted by Borges et al. (2021) focused more on specific AI interactions with organisational strategy and so misses some of the context in how it is being defined and how it creates value.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Borges et al. (2021)

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given

    Authors three and four were experienced in reviewing studies and acted as external reviewers to validate the research protocol.

    “To further mitigate this threat, authors three and four were experienced in reviewing studies and acted as external reviewers to validate the research protocol.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Once literature searches in leading journals have commenced, the leading conferences of a field should follow.

    “Once the literature searches in leading journals have commenced, the leading conferences of a field should follow (Webster & Watson, 2002).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Webster & Watson, 2002

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    There is no consensus on what defines AI or what distinguishes it from other digital technologies.

    “Yet, there is no consensus on what defines AI or what distinguishes it from other digital technologies (Bhatnagar et al., 2018;Monett & Lewis, 2018;Nilsson, 2009).”
    • Also stated in: § 2.1. Evolution of AI definition · plain
      “Despite the length of time the field has existed, there is still no commonly accepted definition (Allen, 1998;Bhatnagar et al., 2018;Brachman, 2006;Hearst & Hirsh, 2000;Nilsson, 2009).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Bhatnagar et al., 2018

      Not checked in the free run.

    • cites Monett & Lewis, 2018

      Not checked in the free run.

    • cites Nilsson, 2009

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Screening of the retrieved studies was achieved by following best practices proposed by Kitchenham (2004) and Dingsøyr and Dybå (2008).

    “Screening of the retrieved studies was achieved by following the best practices proposed by Kitchenham (2004) and Dingsøyr and Dybå (2008).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Kitchenham (2004)

      Not checked in the free run.

    • cites Dingsøyr and Dybå (2008)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Three authors were involved in data extraction of the primary studies.

    “three authors were involved in data extraction of the primary studies”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    Karger (2020) focused narrowly on the relations between AI and blockchain and excludes everything else.

    “The literature review conducted by Karger (2020) is focused narrowly on the relations between AI and blockchain and excludes everything else.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Karger (2020)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Scopus claims to be the largest database for abstracts and citations.

    “Scopus was used as it claims to be largest database for abstracts and citations (Ballew, 2009;Kitchenham & Charters, 2007).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Ballew, 2009

      Not checked in the free run.

    • cites Kitchenham & Charters, 2007

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Two authors independently analysed the 1877 studies to remove duplicates, non-English studies, non-IS studies, and non-peer reviewed scientific studies.

    “Two authors independently analysed the 1877 studies to remove (i) duplicate studies, (ii) non-English studies, (iii) non-IS studies, and (iv) non-peer reviewed scientific studies (books, book chapters, experience reports).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ISI Web of Science is the largest citation database, storing over 800 million references.

    “ISI Web of Science was used as it is the largest citation database which stores over 800 million references (Manikandan & Amsaveni, 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Manikandan & Amsaveni, 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Conclusions drawn from analysis of the 98 primary studies involved all authors.

    “conclusions drawn from analysis of the 98 primary studies involved all authors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    A full audit trail from the initial 1877 studies to the identification of 98 primary studies was provided.

    “a full 'audit trail' from the initial 1877 studies to the identification of 98 primary studies was provided”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    This study includes all relevant studies in a fifteen-year period in high quality journals and conferences, and includes studies that use AI in more oblique ways than previous SLRs.

    “This study includes all the relevant studies in a fifteen year period in a number of high quality journals and conferences, and includes studies that use AI in more oblique ways than these SLRs, such as studies that use machine learning approaches when researching their focus.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The nine selected IS journals focus on the socio-technological aspects of AI, which is the scope of this SLR.

  • May combine two claims.

    Premisecitation

    Hofmann et al. (2019) focused on the effects of AI and ML in the radiology value chain, with a time span of 2012 to 2018.

    “The literature review conducted by Hofmann et al. (2019) was primarily concerned with the effects of AI and ML in the context of the radiology value chain, and so had a time span reflecting that, focusing on 2012 to 2018.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hofmann et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The two reviewers had to agree for a study to stay a primary study.

    “The two reviewers had to agree for a study to stay a primary study.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    A literature review's quality is dependent on the rigor of the search process.

    “A literature review's quality is dependent on the rigor of the search process (Vom Brocke et al., 2009).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Vom Brocke et al., 2009

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    There is less repetition of contributions and lessons learned, and cumulative building of knowledge in this context may take time to accumulate.

    “This means there is less repetition of contributions and lessons learned, and that cumulative building of knowledge in this context may take time to accumulate.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: AI encompasses a broad array of uses and designs.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The aims of the study were not to establish a statistical causal relationship on AI in IS.

    “As the aims of this study were not to establish a statistical causal relationship on AI in IS, other mitigations were used to combat it”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Within IS, it may be more useful to look at AI through the lens of its business capabilities rather than its technologies.

    “As noted by Davenport and Ronanki (2018), within IS, it may be more useful to look at AI through the lens of its business capabilities rather than its technologies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Davenport and Ronanki (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    'Artificial Intelligence' should not be taken to mean 'Artificial Human Intelligence', since 'Intelligence' should be a more general notion than 'Human Intelligence'.

    “After all, "Artificial Intelligence" should not be taken to mean "Artificial Human Intelligence", since "Intelligence" should be a more general notion than "Human Intelligence".”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Economist declared in 2016 that 'After many false starts, artificial intelligence has taken off.'

    “In 2016, the Economist declared in its June Special Report (The Economist, 2016) that "After many false starts, artificial intelligence has taken off."”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites The Economist, 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    It is possible to combine a variety of AI types together into a single AI system.

    “It is possible to combine a variety of these types together into a single AI system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisereasoning

    The search strategy is best developed in concert with the research question.

    “Therefore, the search strategy is best developed in concert with the research question.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The surge of interest in AI among businesses and policy makers naturally resulted in a matching surge in interest among researchers.

    “This surge of interest in the use of AI among businesses and policy makers naturally resulted in a matching surge in interest among researchers, who now have much more to study due to AIs ever increasing use.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisereasoning

    'Intelligence' by its dictionary definition today was formed long before AI, and therefore strictly about human intelligence, where the various levels (conscious and subconscious) are unified, which is not the case for computer intelligence.

    “Yet "intelligence" by its dictionary definition today was formed long before AI, and therefore strictly about human intelligence, where the various levels (conscious and subconscious) are unified, which is not the case for computer intelligence.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Performing experiments can provide rich data on AI usage in a variety of complex, contextual environments.

    “As AI is still a broad and relatively nebulous field in IS, performing experiments can provide rich data on AI usage in a variety of complex, contextual environments.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: AI is a broad and relatively nebulous field in IS.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisereasoning

    For human intelligence, structure, behaviour, capability, and function are all relatively unified, while for AI these aspects are commonly pursued in different ways towards different goals.

    “For human intelligence, structure, behaviour, capability, and function are all relatively unified, while for AI these aspects are commonly pursued in different ways towards different goals.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The surge in interest in AI seems to correlate with our own findings, as researchers now have much more access to empirical evidence and AI in use.

    “This surge in interest in AI seems to correlate with our own findings, as researchers now have much more access to empirical evidence and AI in use with the advent of this "summer".”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Regular meetings with all authors were used to explore any potential bias.

    “such as regular meetings with all authors to explore any potential of bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The definitions of AI often end up being quite similar, even if they are taken from a separate source.

    “This means that the definitions of AI often end up being quite similar, even if they are taken from a separate source.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Okoli (2015) proposes a systematic review process consisting of 8 steps across 4 phases.

    “Okoli (2015) propose a systematic review process that consists of 8 steps, namely planning (2 steps), selection (2 steps), extraction (2 steps) and execution (2 steps) that are completed across 4 phases (see Fig. 1.).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Okoli (2015) source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The systematic approach was chosen for its ability to offer high quality, transparent and replicable reviews.

    “This systematic approach was chosen for its ability to offer reviews of high quality (Dingsøyr & Dybå, 2008) and transparent and replicable review (Leidner & Kayworth, 2006).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Dingsøyr & Dybå, 2008

      Not checked in the free run.

    • cites Leidner & Kayworth, 2006

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    IBM's Watson combines NLP, ML and machine vision techniques.

    “For example, IBM's Watson combines NLP, ML and machine vision techniques (Jarrahi, 2018a, 2018b).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jarrahi, 2018a, 2018b

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    To know to what degree the results of a study can be generalised, it is essential that the research process is described.

    “To know to what degree the results of a study can be generalised it is essential that the research process is described (Petersen & Wohlin, 2009).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Petersen & Wohlin, 2009

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A generally accepted approach to search string strategy is to base the search string on the research questions and include a list of synonyms, abbreviations, and alternative spellings.

    “A generally accepted approach to search string strategy is to base the search string on the research questions and include a list of synonyms, abbreviations, and alternative spellings (Kitchenham & Charters, 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kitchenham & Charters, 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    In the IS community, the increased feasibility of building empirical models from experimental data and using these models to make predictions is likely another reason for the resurgence of interest.

    “In the IS community specifically, the increased feasibility of building empirical models from experimental data and using these models to make predictions (the goal of predictive analytics using big data) is likely another reason for the resurgence of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    A second reviewer was invited to analyse the studies to pre-emptively combat potential bias.

    “A second reviewer was invited to analyse these studies as well, to pre-emptively combat potential bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Human intelligence is developed under certain evolutionary and biological restrictions, which are essential for human, but not really for intelligence in general.

    “Additionally, human intelligence is developed under certain evolutionary and biological restrictions, which are essential for human, but not really for intelligence in general.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    There may be issues with how IS research is approaching the defining of AI as there seems to be a belief that there is a true meaning of 'intelligence' that AI research projects should abide by, at least among those that consider defining AI.

    “However, there may be issues with how IS research is approaching the defining of AI as there seems to be a believe that there is a true (i.e., real, natural) meaning of "intelligence" that AI research projects should abide by, at least among those that consider defining AI.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    The lack of sufficient detail seen in many of the primary studies showcases a possible lack in the cumulative building of knowledge.

    “While it is not feasible to ask every researcher, who uses a machine learning technique in their research to dedicate an entire section to AI, the lack of sufficient detail seen in many of the primary studies showcases a possible lack in the cumulative building of knowledge.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data

    The fact that machine learning applications are the most researched technology within the AI domain provides some indication about where future investments should be directed, as well as the expected type of business value that they can deliver.

    “The fact that machine learning applications are the most researched technology within the AI domain provides some indication about where future investments should be directed, as well as the expected type of business value that they can deliver.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The specific applications and technologies of AI that are mostly researched provide practitioners some indication about the future deployments and common technologies in organisations.

    “The specific applications and technologies of AI that are mostly researched provide practitioners some indication about the future deployments and common technologies in organisations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The machine learning dominance seems due to, at least in part, the wide variety of contexts in which machine learning is useful in comparison to the other types.

    “This machine learning dominance seems to be due to, at least in part, the wide variety of contexts in which machine learning is useful in comparison to the other types.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Cognitive engagement was likely least common partly because it deals more with customers, and businesses are more conservative with customer facing technology.

    “This is likely partly due to the cognitive engagement dealing more with customers, and businesses being more conservative with customer facing technology (Davenport & Ronanki, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Davenport & Ronanki, 2018

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The advancement of technology making supervised learning from datasets and machine learning much more feasible means researchers are now focused on this new foray, and 'tools' and 'methods' are left to languish because the resurgence doesn't lift up all aspects of AI equally.

    “That is, the advancement of technology making supervised learning from datasets and machine learning much more feasible means that researchers are now focused on this new foray, and "tools" and "methods" are left to languish because it seems the resurgence doesn't lift up all aspects of AI equally.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    These activities involve data analysis at such speeds and volumes that no human would be possible to process.

    “These activities involve data analysis at such speeds and at volumes that no human would be possible to process.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Process automation is likely the most reported business value because it is both the least expensive and the easiest to implement of the three value types.

    “This is likely due to process automation being both the least expensive and the easiest to implement of the three value types discussed here (Davenport & Ronanki, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Davenport & Ronanki, 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    IS alone is ineffective in generating value.

    “Additionally, much of IS literature recognises that IS alone is ineffective in generating value, so complementary assets are key to realising value from IS (Shea, Dow, & Chong, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Shea, Dow, & Chong, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Wagner (2017) theorised that the reason for the lack of impact is that earlier developers were able to capitalise on the 'low hanging fruit' that had bigger impact for organisations.

    “This was noted by (Wagner, 2017), who also theorised that the reason for this lack of impact is that the earlier developers were able to capitalise on the 'low hanging fruit' that had bigger impact for organisations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wagner, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Davenport and Ronanki (2018) described cognitive insight as 'analytics on steroids'.

    “Cognitive insight was the second most common type of derived business value among the primary studies and was described by Davenport and Ronanki (2018) as "analytics on steroids".”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Davenport and Ronanki (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    It has been noted by other studies that the shift to hybrid knowledge bases hasn't resulted in a greater impact than earlier systems, despite advances in technology.

    “However, it has been noted by other studies that this shift hasn't resulted in a greater impact than earlier systems, despite the advances in technology.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wagner, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    The idea of having frustrating conversions with a chat-bot that just can't seem to understand what the user is saying is still quite strong in the public consciousness, never mind high-profile failures such as Taybot.

    “The idea of having frustrating conversions with a chat-bot that just can't seem to understand what the user is saying still quite strong in the public consciousness, never mind the high-profile failures such as Taybot (Badjatiya, Gupta, Gupta, & Varma, 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Badjatiya, Gupta, Gupta, & Varma, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Previous research examined the fear of computers and did not account for new and evolving technologies such as robots.

    “A specific domain that is currently lacking in research for robotics is technophobia; previous research has examined the fear of computers and have not accounted for new and evolving technologies such as robots (Sinha, Singh, Gupta, & Singh, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sinha, Singh, Gupta, & Singh, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Complementary assets are key to realising value from IS.

    “Additionally, much of IS literature recognises that IS alone is ineffective in generating value, so complementary assets are key to realising value from IS (Shea, Dow, & Chong, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Shea, Dow, & Chong, 2019

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Some of the other types of AI have received less research focus, though all of them can offer much both practically and academically.

    “This means that some of the other types of AI have received less research focus, though all of them can offer much both practically and academically.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Major themes that have been under-reached remain difficult to identify with an absence of theoretically grounded work.

    “Adding to the above, major themes that have been under-reached remain difficult to identify with an absence of theoretically grounded work.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    This increased complexity may also mean it will take more time and effort than usual to see a return on the time and resources invested.

    “This increased complexity may also mean it will take more time and effort than usual to see a return on the time and resources invested.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Google Trends show that popularity of the search term for machine learning has surpassed the popularity of AI by almost twice as much.

    “Google Trends show that popularity of the search term for machine learning has surpassed the popularity of AI by almost twice as much (Google Trends, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Google Trends, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Strong AI does not seem likely to be made a reality within the foreseeable future.

    “However, despite the great strides in AI noted in the, so called”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kurzweil, 2005

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Adopters of RPA have noted the automation can radically transform back offices, delivering much lower costs while improving service quality, and decreasing delivery times, as well as freeing up employees from tedious tasks.

    “Adopters of RPA have noted the automation can radically transform back offices, delivering much lower costs while improving service quality, and decreasing delivery times, as well as freeing up employees from tedious tasks so they can focus on more important, challenging, and varied work (Lacity, Willcocks, & Andrew, 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lacity, Willcocks, & Andrew, 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    While the prominence of machine learning may be understandable, it is still a concern.

    “Therefore, while the prominence of machine learning may be understandable, it is still a concern.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The reason for the lack of focus on tools or methods is likely related to the reasons for the AI resurgence noted in section 5.2.

    “It seems likely that the reason for this lack of focus on tools or methods is related in some ways to the reasons for the AI resurgence itself noted in section 5.2.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    It also makes the comparison between findings and the identification of complementarities much more difficult.

    “It also makes the comparison between findings and the identification of complementarities much more difficult.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The frequent appearance of machine learning as the AI application used in this paper can be attributed to the fact that it concerns a very broad spectrum of potential applications.

    “In addition, the frequent appearance of machine learning as the AI application used in this paper can be attributed to the fact that it concerns a very broad spectrum of potential applications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    It has been noted that AI technology has tended to 'become a somewhat broad church where many forms of automation and limited intelligent machines are labelled as AI'.

    “It has been noted that AI technology has tended to "become a somewhat broad church where many forms of automation and limited intelligent machines are labelled as AI" (Dwivedi et al., 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dwivedi et al., 2021 source

      Not checked in the free run.

115 of 115

  • The authors state this as an assumption.

    The nine selected IS journals focus on the socio-technological aspects of AI, which is the scope of this SLR.

    “These databases were used to retrieve studies from relevant IS journals, which were chosen as we belief these nine journals focus on the social-technological aspects of AI, which is the scope of this SLR.”

  • The paper relies on this without support.

    There is a growing concern that research on AI could experience a lack of cumulative building of knowledge, which has overshadowed IS research previously.

    “There is, however, a growing concern that research on AI could experience a lack of cumulative building of knowledge, which has overshadowed IS research previously.”

  • The paper relies on this without support.

    AIS eLibrary is unique among the selected databases in being able to provide studies from the leading IS conferences.

    “AIS eLibrary was used as of the databases used it was unique in being able to provide studies from the leading IS conferences.”

  • The paper relies on this without support.

    AI is a broad and relatively nebulous field in IS.

    “As AI is still a broad and relatively nebulous field in IS, performing experiments can provide rich data on AI usage in a variety of complex, contextual environments.”

  • The paper relies on this without support.

    AI encompasses a broad array of uses and designs.

    “This is compounded upon by AI encompassing such a broad array of uses and designs.”

  • The paper relies on this without support.

    AI research tends to swing between its 'winters' and 'summers'.

    “It's already been noted that AI research tends to swing between its "winters" and "summers", and it seems clear that we have headed into one of its summer periods in recent years.”

  • The paper relies on this without support.

    AI applications cover a breadth of different techniques, technologies, and set a different set of requirements on data, infrastructure and leveraging them in the organisational setting.

    “Since AI applications cover a breadth of different techniques, technologies, and set a different set of requirements on data, infrastructure and leveraging them in the organisational setting it is important that future research accurately frames the definition of AI that is used.”

  • The paper relies on this without support.

    The subject and definition of human intelligence is still heavily debated even now.

    “But the subject and definition of human intelligence is something that is still heavily debated even now.”

  • The paper relies on this without support.

    AI has potential in all its forms.

    “There is potential for AI in all its forms.”

  • The paper relies on this without support.

    AI encompasses a broad array of ideas and practices, ranging from the specifics of the technology used to how it is defined.

    “However, AI does tend to encompass a broad array of ideas and practises, ranging from the specifics of the technology used to even how it is defined.”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Future research called a fourth aim after three aims are listed

    Minor

    The introduction lists three aims, with future research as the third; the research-agenda section calls it the fourth and final aim.

  • Dartmouth conference dated both 1951 and 1956

    Minor

    The paper gives two different years for the Dartmouth conference.

  • Business-value counts cover only 96 studies

    Minor

    The three business-value counts sum to 96. The paper says all 98 primary studies were mapped to one most relevant value category.

    Reported 98 · recomputed 96

  • Machine-learning study count differs between text and Table 12

    Minor

    The text reports 69 studies categorised as machine learning, while Table 12 labels the category as containing 68.

  • Experiments called most popular despite a higher data-mining count

    Minor

    The paper reports 21 studies using experiments and 22 using data mining, then calls experiments the most popular data-collection choice.

  • Research-method counts cover only 96 studies

    Minor

    The four reported research-method counts total 96, although this analysis is said to cover 98 primary studies.

    Reported 98 · recomputed 96

  • Lessons-learned contribution count changes from 39 to 40

    Minor

    The findings section gives 39 studies for this contribution type, but its description of Fig. 5 gives 40.

  • AI-type counts exceed the number of primary studies

    Minor

    The reported counts for machine learning, expert systems, machine vision, NLP, robotics and other sum to 108, despite the stated rule that each of the 98 studies is categorised solely by its primary AI type.

    Reported 98 · recomputed 108

8 of 8

Cites

Flagged first, then by citations

7 of the 211 references couldn’t be matched to a known paper.

Research works printed but not linked: 27

  1. Berente, N., Gu, B., Recker, J., & Santhanam, R. (2019). Managing AI. MIS Quarterly.
  2. Brachman, 2006, AA)AI—more than the sum of its parts, 2005 AAAI Presidential Address, 27, 19
  3. Buettner, R., Frick, J., & Rieg, T. (2019). High performance detection of epilepsy in seizure-free EEG recordings: A novel machine-learning approach using very specific epilectic EEG sub-bands. ICIS 2019 Proceedings.
  4. Dennehy, 2020, Ireland post-pandemic: Utilizing AI to kick-start economic recovery, Cutter Business Technology Journal, 33, 22
  5. Jarrahi, 2018, Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making, Business Horizons, 64, 577, 10.1016/j.bushor.2018.03.007 · DOI · Paper page
  6. Langley, P. (2011). Artificial Intelligence. AISB Quarterly.
  7. Li, Y. (2017). Deep Reinforcement Learning: An Overview. arXiv preprint arXiv.
  8. Merkert, J., Mueller, M., & Hubl, M. (2015). A survery of the application of machine learning in decisiion support syystems. ECIS 2015 Proceedings.
  9. Petersen, 2015, Guidelines for conducting systematic mapping studies in software engineering: an update, Information Software Technology, 64, 1, 10.1016/j.infsof.2015.03.007 · DOI · Paper page
  10. Rai, 2019, Next-generation digital platforms: Towards human-AI hybrids, MIS Quarterly, 43, 3
  11. Schwartz, 2019, Green AI, Commun. ACM, 63, 54
  12. Wilson, J., & Daugherty, P. R. (2018, July-August). Collaborative Intelligence: Humans and AI Are Joining Forces. Harvard Business Review.
  13. Meyer, 2014, A machine learning approach to improving dynamic decision making, Information Systems Research, 25, 239, 10.1287/isre.2014.0513 · DOI
  14. Mettler, 2017, Service robots in hospitals: New perspectives on niche evolution and technology affordances, European Journal of Information Systems, 26, 451, 10.1057/s41303-017-0046-1 · DOI · Paper page
  15. Nissen, 2006, Incorporating software agents into supply chains: Experimental investigation with a procurement task, MIS Quarterly: Management Information Systems, 30, 145, 10.2307/25148721 · DOI
  16. Yin, 2019, Regulating cryptocurrencies: A supervised machine learning approach to de-anonymizing the bitcoin blockchain, Journal of Management Information Systems, 36, 37, 10.1080/07421222.2018.1550550 · DOI
  17. P20. Chatterjee, S., Saeedfar, P., Tofangchi, S. & Kolbe, L. (2018). Intelligent road maintenance: a machine learning approach for surface defect detection. Research Papers. ECIS 2018 Proceedings.
  18. P25. Hofmann, P., Oesterle, S., Rust, P. & Urbach, N. (2019). Machine learning approaches along the radiology value chain – rethinking value propositions. ECIS 2019 Proceedings.
  19. P44. Rzepka, C. & Berger, B. (2018). User Interaction with AI-enabled Systems: A Systematic Review of IS Research. ICIS 2018 Proceedings.
  20. Abubakar, 2019, Applying artificial intelligence technique to predict knowledge hiding behavior, International Journal of Information Management, 49, 45, 10.1016/j.ijinfomgt.2019.02.006 · DOI · Paper page
  21. Dwivedi, 2019, Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy, International Journal of Information Management
  22. Nishant, 2020, Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda, International Journal of Information Management, 53, 10.1016/j.ijinfomgt.2020.102104 · DOI · Paper page
  23. Sipior, 2020, Considerations for development and use of AI in response to COVID-19, International Journal of Information Management, 55, 10.1016/j.ijinfomgt.2020.102170 · DOI
  24. Sinha, 2020, Robotics at workplace: An integrated Twitter analytics – SEM based approach for behavioral intention to accept, International Journal of Information Management, 55, 10.1016/j.ijinfomgt.2020.102210 · DOI
  25. Borges, 2020, The strategic use of artificial intelligence in the digital era: Systematic literature review and future research directions, International Journal of Information Management
  26. Coombs, 2020, Will COVID-19 be the tipping point for the Intelligent Automation of work? A review of the debate and implications for research, International Journal of Information Management, 55, 10.1016/j.ijinfomgt.2020.102182 · DOI · Paper page
  27. P83. Karger, E. (2020). Combining Blockchain and Artificial Intelligence – Literature Review and State of the Art. ICIS 2020 Research Papers.

Cited by

Sorted by citations

Related

Authors
Collins, C., Dennehy, D., Conboy, K., & Mikalef, P.
Affiliations
  • Ollscoil na Gaillimhe – University of Galway (IE)
  • Norwegian University of Science and Technology (NO)
Corresponding author
Collins, Christopher
ORCID
Venue
International Journal of Information Management
Publisher
Elsevier BV
Published
2021-10
Volume
60
Pages
102383
ISSN
0268-4012 (ISSN-L)
DOI
10.1016/j.ijinfomgt.2021.102383
OpenAlex
W3178738300
Semantic Scholar
52cf81382eebec7d73c0de2f19af81c7a7028ab3
Cited by
783
References
211
Citations by year
2026: 127 · 2025: 232 · 2024: 193 · 2023: 133 · 2022: 79 · 2021: 18 · 2020: 1
FWCI
7.58
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Information Management (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
Funders
National University of Ireland, Galway
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Information Technology Governance and Strategy
Keywords
artificial intelligence, information systems research, research agenda, business value of AI, research trends

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review