CoChat
Sign Up
Cited by 54Electronic Health Records SystemsMobile Health and mHealth ApplicationsInnovation in Digital Healthcare Systems

Interoperability opportunities and challenges in linking mhealth applications and eRecord systems: Botswana as an exemplar

Open access
Ndlovu, Kagiso, Scott, Richard E., Mars, Maurice·2021·BMC Medical Informatics and Decision Making
DOI 10.1186/s12911-021-01606-7PMC PMC8379582

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Significant investments have been made towards the implementation of mHealth applications and eRecord systems globally. However, fragmentation of these technologies remains a big challenge, often unresolved in developing countries. In particular, evidence shows little consideration for linking mHealth applications and eRecord systems. Botswana is a typical developing country in sub-Saharan Africa that has explored mHealth applications, but the solutions are not interoperable with existing eRecord systems. This paper describes Botswana’s eRecord systems interoperability landscape and provides guidance for linking mHealth applications to eRecord systems, both for Botswana and for developing countries using Botswana as an exemplar. Methods: A survey and interviews of health ICT workers and a review of the Botswana National eHealth Strategy were completed. Perceived interoperability benefits, opportunities and challenges were charted and analysed, and future guidance derived. Results: Survey and interview responses showed the need for interoperable mHealth applications and eRecord systems within the health sector of Botswana and within the context of the National eHealth Strategy. However, the current Strategy does not address linking mHealth applications to eRecord systems. Across Botswana’s health sectors, global interoperability standards and Application Programming Interfaces are widely used, with some level of interoperability within, but not between, public and private facilities. Further, a mix of open source and commercial eRecord systems utilising relational database systems and similar data formats are supported. Challenges for linking mHealth applications and eRecord systems in Botswana were identified and categorised into themes which led to development of guidance to enhance the National eHealth Strategy. Conclusion: Interoperability between mHealth applications and eRecord systems is needed and is feasible. Opportunities and challenges for linking mHealth applications to eRecord systems were identified, and future guidance stemming from this insight presented. Findings will aid Botswana, and other developing countries, in resolving the pervasive disconnect between mHealth applications and eRecord systems.

AI snapshot

Written from the paper’s full text.

TL;DR

This study assessed Botswana’s eHealth interoperability landscape and found that mHealth applications and eRecord systems are largely fragmented, with some interoperability within but not between public and private facilities, and the National eHealth Strategy does not address linking mHealth applications to eRecord systems. It identifies opportunities and challenges and offers guidance, including use of HL7/FHIR, governance, standards, unique patient identifiers, and capacity building.

Key findings

  • Of 18 HICT personnel surveyed, 15 responded (83%). All surveyed facilities had at least one eRecord system, but only one public hospital supported an mHealth application (Kgonafalo), which was not linked to any EMR. Public hospitals used IPMS, DHIS2, and OpenMRS; public clinics used IPMS and PIMS; private facilities used varied commercial and open-source systems.
  • Interoperability was reported within, but not between, public and private facilities. Four HICT personnel reported technical interoperability (two private hospitals, one public clinic, one private clinic), three reported non-interoperability, two public hospitals reported partial interoperability, and three did not know.
  • FAIR guideline responses showed limited understanding of system features: only “Findable” had a positive mean score of 53.6%, while “unsure” responses ranged from 21.4% to 48.2% across the four FAIR criteria.
  • Challenges were categorized into four themes: eHealth legislation and governance; eHealth software and infrastructure; data standards, security and unique patient identifier; and capacity building. The Botswana eHealth Strategy, adopted March 2020, has a Standards and Interoperability pillar but does not use the terms mHealth, personal health record, or eRecord, and does not explicitly require mHealth–eRecord interoperability.

Methods

  • Mixed-methods design: an online survey and key informant interviews of health ICT workers, plus a review of the Botswana National eHealth Strategy.
  • Eighteen HICT personnel were purposively sampled; 15 responded (83%) from MOHW (3), public hospitals (4), private hospitals (2), public clinics (3), and private clinics (3). Eight key informants were selected from survey respondents. The survey had 23 closed-ended questions (2 dichotomous, 7 multiple choice, 14 Likert scale) and 19 open-ended questions; FAIR items used a 3-point Likert scale. It was administered via REDCap from 24 September 2019 to 23 March 2020.
  • Analysis used descriptive statistics (mean, standard deviation, median); interviews were transcribed, entered into NVivo, and thematically analyzed with final themes agreed by author consensus; strategy content was summarized and charted.

Limitations

  • Stated in the paper: there are few private facilities in Botswana, so fewer private facilities were included; the eRecord landscape differed across public and private facilities, limiting comparison; and only one mHealth application (Kgonafalo) was reported and it was not linked to any EMR, so there was no national experience of linking mHealth applications to eRecord systems.
  • Evident from the text: the small purposive sample (15 survey respondents and 8 interviewees) and single-country focus limit generalizability; findings rely on self-reported survey and interview data plus document review, without technical validation or testing of interoperability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12911-021-01606-7

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 65 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Survey and interview responses showed the need for interoperable mHealth applications and eRecord systems within the health sector of Botswana and within the context of the National eHealth Strategy.

    “Survey and interview responses showed the need for interoperable mHealth applications and eRecord systems within the health sector of Botswana and within the context of the National eHealth Strategy.”
    Quote in the paperSubstantial claim
  • Findingown data

    Botswana's National eHealth Strategy does not address linking mHealth applications to eRecord systems.

    “However, the current Strategy does not address linking mHealth applications to eRecord systems.”
    • Also stated in: § Background · plain
      “Whilst Botswana’s recent eHealth Strategy addresses the development of an interoperability platform, it provides no guidance regarding interoperability of mHealth applications and eRecord systems.”
    • Also stated in: § Botswana’s national eHealth strategy review · plain
      “Importantly, the Strategy does not explicitly note the need to make mHealth applications interoperable with eRecord systems.”
    Quote in the paperSubstantial claim
  • Findingnone given

    Diverse eRecord systems from different vendors and donors exist across public and private health sectors in Botswana.

    “Diverse eRecord systems from different vendors and donors exist across public and private health sectors, and are not interoperable resulting in fragmented care delivery, duplication of effort and unnecessary healthcare expenditure.”
    Quote in the paperSubstantial claim
  • Findingown data

    Across Botswana's health sectors, global interoperability standards and Application Programming Interfaces are widely used.

    “Across Botswana’s health sectors, global interoperability standards and Application Programming Interfaces are widely used, with some level of interoperability within, but not between, public and private facilities.”
    Quote in the paperNot a substantial claim
  • Findingown data

    HICTs at two public hospitals and one private clinic did not know if their eRecord systems were interoperable.

    “HICTs at two public hospitals and one private clinic did not know if their eRecord systems were interoperable.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 42
  • Classified as probably not a substantial claim. 29

  • May combine two claims.

    Findingown data

    Four HICT personnel reported eRecord systems at their facilities to be technically interoperable; two at private hospitals, and one each at a public and private clinic.

    “Four HICT personnel reported eRecord systems at their facilities to be technically interoperable; two at private hospitals, and one each at a public and private clinic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Private hospitals hosted different eRecord systems and used them to support both clinical services and medical billing processes.

    “Private hospitals hosted different eRecord systems and used them to support both clinical services and medical billing processes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The Trimed and Arcus Air eRecord systems as well as the Sukraa and the Services Applications and Products (SAP) system, used at two private hospitals respectively, were reported to be HL7 compliant and envisaged to be interoperable.

    “The Trimed and Arcus Air eRecord systems as well as the Sukraa and the Services Applications and Products (SAP) system, used at two private hospitals respectively, were reported to be HL7 compliant and envisaged to be interoperable.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All surveyed health facilities had either one or more eRecord systems in place, and only one public hospital supported an mHealth application.

    “All surveyed health facilities had either one or more eRecord systems in place, and only one public hospital supported an mHealth application.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    All public hospitals accessed the Integrated Patient Management System (IPMS), District Health Information System (DHIS) and the Open-Medical Record System (OpenMRS), and public clinics accessed the IPMS and a stand-alone Patient Information Management System (PIMS).

    “All public hospitals accessed the Integrated Patient Management System (IPMS), District Health Information System (DHIS) and the Open-Medical Record System (OpenMRS), and public clinics accessed the IPMS and a stand-alone Patient Information Management System (PIMS).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The mHealth application (Kgonafalo), although not linked to any EMR, was HL7 compliant, and the SAP system supported an HL7 Integration Adapter Healthcare module, which enables translation of HL7 XML to native HL7 v2 pipe format structure.

    “Of note was that the mHealth application (Kgonafalo), although not linked to any EMR, was also HL7 compliant and the SAP system supported an HL7 Integration Adapter Healthcare module, which enables translation of HL7 XML to native HL7 v2 pipe format structure.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Challenges for linking mHealth applications and eRecord systems in Botswana were identified and categorised into themes which led to development of guidance to enhance the National eHealth Strategy.

    “Challenges for linking mHealth applications and eRecord systems in Botswana were identified and categorised into themes which led to development of guidance to enhance the National eHealth Strategy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Botswana, there is some level of interoperability within, but not between, public and private health facilities.

    “Across Botswana’s health sectors, global interoperability standards and Application Programming Interfaces are widely used, with some level of interoperability within, but not between, public and private facilities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Lack of maintenance budgets to repair damaged hardware, lack of electricity in some facilities, frequent Internet downtimes affecting data capture and contributing to slow performance of eRecord systems were reported.

    “The lack of maintenance budgets to repair damaged hardware, lack of electricity in some facilities, frequent Internet downtimes affecting data capture and contributing to slow performance of eRecord systems were also reported, similar to previous studies in developing countries [34, 39, 41].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [39] Luna D, Almerares A, Mayan JC, de Quirós FGB, Otero C. Health informatics in developing countries: Going beyond pilot practices to sustainable implementations: a review of the current challenges. Healthc Inform Res. 2014;20(1):3–10. https://doi.org/10.4258/hir.2014.20.1.3. source

      Not checked in the free run.

    • cites [41] Mohammadzadeh N, Safdari R. Patient monitoring in mobile health: opportunities and challenges. Med Arch. 2014;68(1):57–60. https://doi.org/10.5455/medarh.2014.68.57-60. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The infrastructure of eRecord systems varied between the public and private health sectors, which do not share data and use different patient identifiers.

    “It also emerged that the infrastructure of eRecord systems varied between the public and private health sectors, which do not share data and use different patient identifiers.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Interviewees noted that different software versions exist across facilities, and their upgrade is time-consuming.

    “Interviewees noted that different software versions exist across facilities, and their upgrade is time-consuming.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Data gathered from earlier mHealth telemedicine initiatives was not captured in an eRecord system and is therefore no longer available.

    “Indeed data gathered from earlier mHealth telemedicine initiatives was not captured in an eRecord system, and is therefore no longer available; another demonstration of the impact of and need for interoperability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Respondents in the public sector were more aware of the National eHealth Strategy and the proposed interoperability platform than their counterparts in the private sector, whilst the opposite was true for understanding of interoperability concepts.

    “Respondents in the public sector were more aware of the National eHealth Strategy and the proposed interoperability platform than their counterparts in the private sector, whilst the opposite was true for understanding of interoperability concepts.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Public facilities commonly use eRecord systems for patient record management while the private sector also used them to support hospital billing and store routine clinical data.

    “Moreover, public facilities commonly use eRecord systems for patient record management while the private sector also used them to support hospital billing and store routine clinical data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Two public hospitals also responded positively to FAIR guidelines and both supported the IPMS, DHIS2 and OpenMRS, all of which are interoperable via the HL7 standard.

    “Two public hospitals also responded positively to FAIR guidelines and both supported the IPMS, DHIS2 and OpenMRS, all of which are interoperable via the HL7 standard.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The IPMS was reported to support twelve interoperable modules of which only the Admissions, Laboratory and Microbiology modules had been implemented.

    “The IPMS was reported to support twelve interoperable modules of which only the Admissions, Laboratory and Microbiology modules had been implemented.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The Optima eRecord system adopted at private clinics was reported to be a cloud-based solution and interoperable with other eRecord systems such as MatrixCare, PointClickCare and HealthMEDX.

    “The Optima eRecord system adopted at private clinics was reported to be a cloud-based solution and interoperable with other eRecord systems such as MatrixCare, PointClickCare and HealthMEDX.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The only overall positive response was for 'Findable' which had a mean score of just 53.6% with three of the four questions scoring 50% or more.

    “The only overall positive response was for ‘Findable’ which had a mean score of just 53.6% with three of the four questions scoring 50% or more.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Even after a marked reduction in the number of eRecord systems in Botswana, different databases are still used and most are still not interoperable within or between facilities or sectors, despite using widely accepted interoperability standards, data formats and APIs.

    “The study shows that even after a marked reduction in the number of eRecord systems in Botswana different databases are still used and most are still not interoperable within or between facilities or sectors, despite using widely accepted interoperability standards, data formats and APIs (Table 2), supporting the need for interoperability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The majority (82%) of eRecord systems and the mHealth application support relational database structures (Table 1), open interoperability standards and APIs (Table 2).

    “The majority (82%) of eRecord systems and the mHealth application support relational database structures (Table 1), open interoperability standards and APIs (Table 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Overall, the private sector responses to the FAIR guideline questions were positive, and came from the health informatician and information technology officers at 2 private hospitals which supported HL7 compliant eRecord systems (R-Care, Sukraa Hospital Information Management System, TriMed systems and Arcus Air (web based)).

    “Overall, the private sector responses to the FAIR guideline questions were positive, and came from the health informatician and information technology officers at 2 private hospitals which supported HL7 compliant eRecord systems (R-Care, Sukraa Hospital Information Management System, TriMed systems and Arcus Air (web based)).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Interviewees perceived the need for legislation to ensure stakeholder coordination, collaboration, and compliance with standards for eRecord systems and mHealth applications.

    “Interviewees perceived the need for legislation to ensure stakeholder coordination, collaboration, and compliance with standards for eRecord systems and mHealth applications.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    There were two examples of NoSQL (non-relational databases).

    “There were two examples of NoSQL (non-relational databases).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Only one mHealth application (Kgonafalo system) was reported and is not linked to any EMR hence there is no national experience of linking mHealth applications to eRecord systems in Botswana.

    “Only one mHealth application (Kgonafalo system) was reported and is not linked to any EMR hence there is no national experience of linking mHealth applications to eRecord systems in Botswana.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    Although the National eHealth Strategy was silent on 'mHealth', it identified some aspects related to interoperability of eRecord systems.

    “Although the National eHealth Strategy was silent on ‘mHealth’, it identified some aspects related to interoperability of eRecord systems.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    This study has identified challenges and opportunities for Botswana's public and private eRecord systems to interoperate between themselves, and with mHealth applications.

    “This study has identified challenges and opportunities for Botswana’s public and private eRecord systems to interoperate between themselves, and with mHealth applications.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Past localised efforts to use mHealth in Botswana covered four clinical areas, but none were linked to any eRecord system.

    “Past localised efforts have been made to use mHealth in four clinical areas [24], but none were linked to any eRecord system(s).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Ndlovu K, Littman-Quinn R, Park E, Dikai Z, Kovarik CL. Scaling up a mobile telemedicine solution in Botswana: keys to sustainability. Front Pub Health. 2014;2:275. https://doi.org/10.3389/fpubh.2014.00275. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    A recent systematic review of mHealth interventions in developing countries concluded that most mobile health interventions are not ready for interoperability and to be integrated into existing health information systems.

    “This is consistent with a recent systematic review of mHealth interventions in developing countries which concluded that “most of the mobile health interventions are not ready for interoperability and to be integrated into the existing health information systems” [25].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] Hoque MR, Rahman MS, Nipa NJ, Hasan MR. Mobile health interventions in developing countries: a systematic review. Health Inform. 2020;26(4):2792–810. https://doi.org/10.1177/1460458220937102. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Interoperability between mHealth applications and eRecord systems is needed and is feasible.

    “Interoperability between mHealth applications and eRecord systems is needed and is feasible.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The lack of interoperability between eRecord systems in Botswana results in fragmented care delivery, duplication of effort and unnecessary healthcare expenditure.

    “Diverse eRecord systems from different vendors and donors exist across public and private health sectors, and are not interoperable resulting in fragmented care delivery, duplication of effort and unnecessary healthcare expenditure.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Linking mHealth applications and eRecord systems should result in seamless data exchange within a healthcare community with subsequent benefit.

    “Linking of mHealth applications and eRecord systems should result in seamless data exchange within a healthcare community with subsequent benefit [15, 16].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15] Heart T, Ben-Assuli O, Shabtai I. A review of PHR, EMR and EHR integration: a more personalized healthcare and public health policy. Health Policy Technol. 2017;6(1):20–5. https://doi.org/10.1016/j.hlpt.2016.08.002. source

      Not checked in the free run.

    • cites [16] Hook JM, Pan E, Adler-Milstein J, Bu D, Walker J. The value of healthcare information exchange and interoperability in New York state. AMIA 2006 Symposium Proceedings. 2006; 953. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839305/pdf/AMIA2006_0953.pdf. Accessed 12 April 2021.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Botswana has identified mHealth as a means of improving healthcare provision and delivery.

    “Botswana has identified mHealth as a means of improving healthcare provision and delivery [9].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] Littman-Quinn R, Chandra A, Schwartz A, Fadlelmola FM, Ghose S, Luberti AA, et al. mHealth applications for telemedicine and public health intervention in Botswana. In: Cunningham P, Cunningham M, editors. 2011 IST-Africa conference proceedings, IST 2011. 2011;1–11.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Implementation and adoption of eRecord systems and mHealth applications in sub-Saharan Africa has been slow, and systems in place have usually been developed by donors for a single problem, resulting in silos of eRecord systems and donor-driven pilotitis.

    “The implementation and adoption of eRecord systems and mHealth applications in SSA has been slow and the systems that are in place have usually been developed by donors for a single problem, such as HIV/AIDS, tuberculosis, or malaria. This has resulted in silos of eRecord systems often containing similar data on the same patients, and extensive ‘donor-driven pilotitis’ [22].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] McCann D. A. Ugandan mHealth Moratorium Is a Good Thing. ICT Works [Internet]. 2012. https://www.ictworks.org/ugandan-mhealth-moratorium-good-thing/#. Accessed 8 Feb 2021.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Literature points to mHealth applications as vulnerable to bandwidth limitations, short battery life-spans, and unreliable access to electricity supply.

    “Although a viable option in Botswana, literature points to mHealth applications as vulnerable to bandwidth limitations, short battery life-spans, and unreliable access to electricity supply [34, 41].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [41] Mohammadzadeh N, Safdari R. Patient monitoring in mobile health: opportunities and challenges. Med Arch. 2014;68(1):57–60. https://doi.org/10.5455/medarh.2014.68.57-60. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Reported options to address these challenges include utilisation of broadband Internet, high-speed WiFi and battery packs (power-banks).

    “Reported options to address these challenges include utilisation of broadband Internet, high-speed WiFi and battery packs (power-banks) [34, 40].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [40] Juma K, Nahason M, Apollo W, Gregory W, Patrick O. Current Status of E-Health in Kenya and emerging global research trends. Int J Inf Commun. 2012;2(1):50–4.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Corruption and unpredictable changes in policy have prevented successful implementation of eHealth.

    “Corruption and unpredictable changes in policy have prevented successful implementation of eHealth [34].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    This requires reliable and stable Internet connectivity or a hybrid model that stores data locally and synchronises whenever there is stable connectivity.

    “This requires reliable and stable Internet connectivity or a hybrid model that stores data locally and synchronises whenever there is stable connectivity [44, 45].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [44] Adamko A, Garai A, Pentek I. Review and requirement specification of telemedicine systems interoperability with common data exchange hub. In: Proceedings of IEEE 14th international symposium on intelligent systems and informatics, 2016, pp. 73–8. source

      Not checked in the free run.

    • cites [45] El-Sappagh S, Ali F, Hendawi A, Jang JH, Kwak KS. A mobile health monitoring-and-treatment system based on integration of the SSN sensor ontology and the HL7 FHIR standard. BMC Med Inform Decis Mak. 2019;19:97. https://doi.org/10.1186/s12911-019-0806-z. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Cloud-based infrastructure has been suggested to support specific user needs of facilities that cannot afford to install and maintain sophisticated hardware.

    “To support specific user needs of facilities that cannot afford to install and maintain sophisticated hardware, cloud-based infrastructure has been suggested [44].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Adamko A, Garai A, Pentek I. Review and requirement specification of telemedicine systems interoperability with common data exchange hub. In: Proceedings of IEEE 14th international symposium on intelligent systems and informatics, 2016, pp. 73–8. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Software that is not designed with user preference and ease of use in mind is an issue.

    “as is software that is not designed with user preference and ease of use in mind [38, 41].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [38] Iroju O, Soriyan A, Gambo I, Olaleke J. Interoperability in healthcare: Benefits, challenges and resolutions. Int J Inno App Stud. 2013;3(1):262–70.

      Not checked in the free run.

    • cites [41] Mohammadzadeh N, Safdari R. Patient monitoring in mobile health: opportunities and challenges. Med Arch. 2014;68(1):57–60. https://doi.org/10.5455/medarh.2014.68.57-60. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Open source eRecord systems have presented opportunities in other settings.

    “Similarly, open source eRecord systems have presented opportunities in other settings [34, 35, 40]”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [35] Bakar A, Sheikh Y, Sultan B. Opportunities and challenges of open source software integration in developing countries: case of Zanzibar Health Sector. J Health Infor Dev Ctries. 2012;6(2):443–53.

      Not checked in the free run.

    • cites [40] Juma K, Nahason M, Apollo W, Gregory W, Patrick O. Current Status of E-Health in Kenya and emerging global research trends. Int J Inf Commun. 2012;2(1):50–4.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The Botswana eHealth Strategy promotes use of universally available software, services and content (global goods applications), endorsed by the WHO and designed to be interoperable.

    “The Botswana eHealth Strategy promotes use of universally available software, services and content (global goods applications), endorsed by the WHO and designed to be interoperable [43].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] Measure Evaluation. Global digital health resources and maturity models: a summary. [internet]. 2018. https://www.measureevaluation.org/resources/publications/fs-18-305, Accessed 26 Mar, 2021.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Strong political and administrative buy-in could lead to the development of a robust health information exchange (HIE) and efficient governance structures that could influence cultural norms as well as stakeholders' prioritisation of data and health information systems.

    “This aligns with previous studies which found that strong political and administrative buy-in could lead to the development of a robust health information exchange (HIE) and efficient governance structures that could influence cultural norms as well as stakeholders’ prioritisation of data and health information systems [34–37].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [35] Bakar A, Sheikh Y, Sultan B. Opportunities and challenges of open source software integration in developing countries: case of Zanzibar Health Sector. J Health Infor Dev Ctries. 2012;6(2):443–53.

      Not checked in the free run.

    • cites [36] Barkman C, Weinehall L. Policymakers and mHealth: roles and expectations, with observations from Ethiopia, Ghana and Sweden. Glob Health Action. 2017;10(Sup3):1337356. https://doi.org/10.1080/16549716.2017.1337356. source

      Not checked in the free run.

    • cites [37] Kierkegaard P. Governance structures impact on eHealth. Health Policy Technol. 2015;4(1):39–46. https://doi.org/10.1016/j.hlpt.2014.10.016. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Combined with coordinated compliance with the Botswana-specific Data Protection Act, such engagement could improve the necessary safeguards towards the right to privacy of individuals and the collection and transfer of their personal data from mHealth applications to and between eRecord systems in Botswana and across-borders.

    “Combined with coordinated compliance with the Botswana-specific Data Protection Act [30] (“the DPA”) such engagement could improve the necessary safeguards towards the right to privacy of individuals and the collection and transfer of their personal data from mHealth applications to and between eRecord systems in Botswana and across-borders.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Government of Botswana, Data Protection Act. Law 32 of 2018. https://www.bocra.org.bw/sites/default/files/documents/DataProtectionAct.pdf. Accessed 12 April 2021.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Software not developed in native languages leads to language barriers and is an issue.

    “In addition, software not developed in native languages [34] leads to language barriers and is an issue”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    There are few private facilities within Botswana, therefore fewer private facilities were included in the study compared to public facilities.

    “There are few private facilities within Botswana, therefore fewer private facilities were included in the study compared to public facilities.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    Interoperability standards identified in this study (HL7, ISO/IEEE 11073, DICOM standards) and network security standards prescribed by the DPA are reference points.

    “Interoperability standards identified in this study (HL7, ISO/IEEE 11073, DICOM standards) and network security standards prescribed by the DPA are reference points.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The Strategy also identified the adoption of global goods (universally available software, services and content) as an appropriate approach.

    “The Strategy also identified the adoption of global goods (universally available software, services and content) as an appropriate approach (subsection 3.5.3).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The American Medical Informatics Association's 10 by 10 education program, the Informatics Training for Global Health Program of the Fogarty International Center, and the introduction of medical informatics courses at Universities have all tried to address this.

    “The problem is not unique to the developing world, and the American Medical Informatics Association’s 10 by 10 education program, the Informatics Training for Global Health Program of the Fogarty International Center, and the introduction of medical informatics courses at Universities have all tried to address this [50–52].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [50] Feldman SS, Hersh W. Evaluating the AMIA-OHSU 10x10 program to train healthcare professionals in medical informatics. In: Proceedings of AMIA annual symposium. 2008;182–6. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2656028/pdf/amia-0182-s2008.pdf. Accessed 12 Apr 2021.

      Not checked in the free run.

    • cites [51] Hasman A, Albert A, Wainwright P, Klar R, Sosa M, editors. Education and training in health informatics in Europe: State of the art, guidelines, applications. Amsterdam, The Netherlands. IOS Press. 2005. p. 286.

      Not checked in the free run.

    • cites [52] Mantas J, Ammenwerth E, Demiris G, Hasman A, Haux R, Hersh W, et al. Recommendations of the International Medical Informatics Association (IMIA) on education in biomedical and health informatics-first revision. Euro J Biomed Inform. 2011;7(2). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Given the wide adoption of the HL7 standard, an applicable API interoperability standard is the Fast Health Interoperability Resources (FHIR).

    “Given the wide adoption of the HL7 standard (Table 2), an applicable API interoperability standard is the Fast Health Interoperability Resources (FHIR, pronounced “fire”).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    MySQL databases and Java applications also enhanced theoretical and technical homogeneity.

    “where MySQL databases and Java applications also enhanced theoretical and technical homogeneity [35].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] Bakar A, Sheikh Y, Sultan B. Opportunities and challenges of open source software integration in developing countries: case of Zanzibar Health Sector. J Health Infor Dev Ctries. 2012;6(2):443–53.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Integrating the Healthcare Enterprise (IHE) workflows were reported to promote coordinated use of established standards such as DICOM and HL7 to address specific clinical needs.

    “Integrating the Healthcare Enterprise (IHE) workflows were reported to promote coordinated use of established standards such as DICOM and HL7 to address specific clinical needs [55].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [55] Integrating the Healthcare Enterprise. Making Healthcare Interoperable. [Internet]. 2021. http://www.ihe.net. Accessed 19 Mar 2021.

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given

    The eRecord landscape was not the same across the private and public facilities, so comparing the two was limited.

    “Similarly, the eRecord landscape was not the same across the private and public facilities, so comparing the two was limited.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    While this allows data retrieval within each system, the absence of nationally unique identifiers remains a barrier to system wide interoperability.

    “While this allows data retrieval within each system the absence of nationally unique identifiers remains a barrier to system wide interoperability.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Cloud services could alleviate mobile device limitations of internal storage, processing capacity and short battery lifespan.

    “Consequently, cloud services could alleviate mobile device limitations of internal storage, processing capacity and short battery lifespan [56].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56] Sulistio A, Reich C, Doelitzscher F. Cloud infrastructure and applications—CloudIA. In: Proceedings of IEEE international conference on cloud computing 2009. Springer, Berlin, Heidelberg. 2009; 583–588. https://doi.org/10.1007/978-3-642-10665-1_56. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Previously reported advantages of RESTful architectures include light-weight interfaces that allow for faster transmission and processing of data structures, more suitable for mobile phones and tablet devices.

    “Previously reported advantages of RESTful architectures include, “light-weight interfaces that allow for faster transmission and processing of data structures, more suitable for mobile phones and tablet devices. RESTful interfaces also facilitate faster development cycles through their simpler structures” [54].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] Bender D, Sartipi K. HL7 FHIR: An Agile and RESTful approach to healthcare information exchange. In: Proceedings of the 26th IEEE International symposium on computer-based medical systems. Porto, Portugal, 2013, p. 326–331. doi: https://doi.org/10.1109/CBMS.2013.6627810. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Despite the lack of a standard query language, limited technical support and lack of maturity, large Internet-based companies (Facebook, LinkedIn, Amazon and Google) have adopted NoSQL database systems to address Big Data challenges including optimised information use and scalability.

    “Despite the lack of a standard query language, limited technical support and lack of maturity [46], large Internet-based companies (Facebook, LinkedIn, Amazon and Google) have adopted NoSQL database systems to address Big Data challenges including optimised information use and scalability [46].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] Ali W, Shafique MU, Majeed MA, Raza A. Comparison between SQL and NoSQL databases and their relationship with big data analytics. Asian J Res Comp Sci. 2019;21:1.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The need for an accreditation system to ensure proper use of these standards has been noted earlier.

    “The need for an accreditation system to ensure proper use of these standards has been noted earlier [44].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Adamko A, Garai A, Pentek I. Review and requirement specification of telemedicine systems interoperability with common data exchange hub. In: Proceedings of IEEE 14th international symposium on intelligent systems and informatics, 2016, pp. 73–8. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    NoSQL provide a schema-less database design and offer a more scalable option than traditional relational database management systems (RDBMS).

    “NoSQL provide a schema-less database design and offer a more scalable option than traditional relational database management systems (RDBMS) [46, 47].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [46] Ali W, Shafique MU, Majeed MA, Raza A. Comparison between SQL and NoSQL databases and their relationship with big data analytics. Asian J Res Comp Sci. 2019;21:1.

      Not checked in the free run.

    • cites [47] Mathew PS, Pillai AS. Big data solutions in healthcare: problems and perspectives. In: 2015 International conference on innovations in information, embedded and communication systems (ICIIECS) 2015:1–6. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    There is a shortage of trained staff to support eRecord systems and existing staff are overburdened.

    “Consistent with other studies in the developing world, there is a shortage of trained staff to support eRecord systems and existing staff are overburdened [34, 35, 38–40].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [34] Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health information exchange in low- and middle-income country settings: a systematic review. Health Policy Plan. 2016;31(9):1310–25. https://doi.org/10.1093/heapol/czw056. source

      Not checked in the free run.

    • cites [35] Bakar A, Sheikh Y, Sultan B. Opportunities and challenges of open source software integration in developing countries: case of Zanzibar Health Sector. J Health Infor Dev Ctries. 2012;6(2):443–53.

      Not checked in the free run.

    • cites [38] Iroju O, Soriyan A, Gambo I, Olaleke J. Interoperability in healthcare: Benefits, challenges and resolutions. Int J Inno App Stud. 2013;3(1):262–70.

      Not checked in the free run.

    • cites [39] Luna D, Almerares A, Mayan JC, de Quirós FGB, Otero C. Health informatics in developing countries: Going beyond pilot practices to sustainable implementations: a review of the current challenges. Healthc Inform Res. 2014;20(1):3–10. https://doi.org/10.4258/hir.2014.20.1.3. source

      Not checked in the free run.

    • cites [40] Juma K, Nahason M, Apollo W, Gregory W, Patrick O. Current Status of E-Health in Kenya and emerging global research trends. Int J Inf Commun. 2012;2(1):50–4.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The health-sector interoperability architecture (based on open standards e.g. OpenHIE or OpenHIM) supporting common data registries, terminology services, authentication, and security services, could simplify complexities of interfaces that will be built between mHealth applications and eRecord systems through common APIs.

    “For example, the health-sector interoperability architecture (based on open standards e.g. OpenHIE or OpenHIM) supporting common data registries, terminology services, authentication, and security services, could simplify complexities of interfaces that will be built between mHealth applications and eRecord systems through common APIs (subsection 3.5.3).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Given the exponential growth of data volumes across multiple eRecord systems and mHealth applications, NoSQL databases are becoming indispensible.

    “Given the exponential growth of data volumes (structured and unstructured Big Data) across multiple eRecord systems and mHealth applications, NoSQL databases should be the preferred option and are becoming indispensible.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    FHIR offers a contemporary, resource-oriented HTTP interface to search for, create, read, update, and delete FHIR resources using a RESTful based approach to access clinical, administrative, and infrastructure data.

    “It offers a contemporary, resource-oriented HTTP interface to search for, create, read, update, and delete FHIR resources using a Representational State Transfer (REST, often referred to as RESTful architectures) based approach to access clinical, administrative, and infrastructure data [53].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016;23(5):899–908. https://doi.org/10.1093/jamia/ocv189. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Synchronisation of patient data collected electronically and in paper systems with different identifiers leads to missing patient data, fragmented care and the development of silos of information that cannot be used for patient care in different settings.

    “This leads to missing patient data, fragmented care and the development of silos of information that cannot be used for patient care in different settings.”
    Quote in the paperSubstantial claim
63 of 63

  • The paper relies on this without support.

    Botswana is a typical developing country in sub-Saharan Africa with an interoperability setting similar to other developing countries, particularly in SSA.

    “Botswana is a typical developing country in sub-Saharan Africa that has explored mHealth applications, but the solutions are not interoperable with existing eRecord systems.”
    “In this paper, Botswana is used as an exemplar for developing world countries with a similar interoperability setting (particularly in SSA), and an approach to identifying and resolving mHealth and eRecord systems interoperability issues is described.”

  • The paper relies on this without support.

    The HICT personnel's survey and interview reports accurately reflect the interoperability characteristics and challenges of their eRecord systems and mHealth applications.

    “Survey and interview responses showed the need for interoperable mHealth applications and eRecord systems within the health sector of Botswana and within the context of the National eHealth Strategy.”

  • The paper relies on this without support.

    Botswana's health ICT landscape and interoperability challenges are representative of developing countries in general.

    “This is an important and recommended step for developing countries in general, where uncertainty of linking mHealth applications to eRecord systems exists.”

  • The paper relies on this without support.

    The FAIR survey questions validly measure respondents' understanding of eRecord system features and interoperability needs.

    “Thus, for six of the 14 questions in the FAIR survey, half of the respondents answered that they were ‘unsure’ (Table 3).”

  • The paper relies on this without support.

    Cloud-based infrastructure is a viable option in Botswana.

    “Although a viable option in Botswana, literature points to mHealth applications as vulnerable to bandwidth limitations, short battery life-spans, and unreliable access to electricity supply [34, 41].”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Interoperability solutions in healthcare are key to overcoming some of the industry's most difficult challenges and promise to drastically cut healthcare costs.

    “Interoperability solutions in healthcare are the key to overcoming some of the industry's most difficult challenges, and they promise to drastically cut healthcare costs [14].”

    From Interoperability of heterogeneous health information systems: a systematic literature review (2023) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Public-hospital response count does not account for the sampled hospitals

    Minor

    Seven public hospitals were sampled and nonrespondents are identified at only two of them, leaving at least five sampled public hospitals with responses. The results list respondents from only four public hospitals.

  • Accessible "Yes" mean is understated

    Minor

    The four Accessible questions report "Yes" percentages of 50.0, 14.3, 64.3 and 21.4. Their mean is 37.5%, not the reported 35.7%.

    Reported 35.7 · recomputed 37.5

  • MOHW respondents are both additional to and included in the 18-person sample

    Minor

    The methods say the three MOHW members participated in addition to the 18 personnel sampled from facilities. The results instead count three MOHW respondents within a total of 18 personnel.

3 of 3

The citation lists haven’t been fetched yet

References 38 · Cited by 54

Research works printed but not linked: 1

  1. Abubaker TE. A performance evaluation of SQL and NOSQL Database on HealthCare Data (Doctoral dissertation, University of Science and Technology).

Authors
Ndlovu, K., Scott, R. E., & Mars, M.
Affiliations
  • University of Botswana (BW)
  • University of Calgary (CA)
  • Flinders University (AU)
Corresponding author
Ndlovu, Kagiso
ORCID
Venue
BMC Medical Informatics and Decision Making
Publisher
Springer Science and Business Media LLC
Published
2021-08-21
Volume
21
Issue
1
Pages
246-246
ISSN
1472-6947 (ISSN-L)
DOI
10.1186/s12911-021-01606-7
PMC
PMC8379582
OpenAlex
W3169723321
Semantic Scholar
cb9265222a08c583d0aa401731369cde44ff4680
Cited by
54
References
38
Citations by year
2026: 9 · 2025: 13 · 2024: 11 · 2023: 12 · 2022: 5 · 2021: 4
FWCI
1.42
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Libraries and Cultural Resources (University of Calgary) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Fogarty International Center (D43TW007004)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Health Professions › Health Information Management › Innovation in Digital Healthcare Systems
Keywords
mobile health apps, electronic health records, interoperability, health information technology, Botswana, application programming interfaces, developing countries, health information exchange
MeSH terms
Botswana, Humans, Telemedicine, Africa South of the Sahara

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review