CoChat
Sign Up
Cited by 226Electronic Health Records SystemsTechnology and Human Factors in Education and HealthInformation Systems Theories and Implementation

Evaluating information technology in health care: barriers and challenges

Open access
Heathfield, Heather, Pitty, David, Hanka, Rudolph·1998·BMJ
DOI 10.1136/bmj.316.7149.1959PMC PMC1113407
Open this paper in CoChat — free account

Abstract

There is strong push for clinical leadership in the development and procurement of information technology in health care.1 The lack of clinical input to date has been cited as a major factor in the failure of information technology in health services2 and has prompted many clinicians to become involved in such endeavours. Furthermore, there are various clinical decision support systems available, the merits of which clinicians are expected to judge (such as Prodigy3 and Capsule4). It is essential that clinicians have a knowledge of evaluation issues in order that they can assess the strengths and weaknesses of evaluation studies and thus interpret their results meaningfully, and also contribute to the design and implementation of such studies to provide them with useful information. #### Summary points Clinicians are becoming increasingly involved in the development and procurement of information technology in health care, yet evaluation studies have provided little useful information to assist them Evaluations by means of randomised controlled trials have not yet provided any major indication of improved patient outcomes or cost effectiveness, are difficult to generalise, and do not provide the scope or detail necessary to inform decision making Clinical information systems are a different kind of intervention from drugs, and techniques used to evaluate drugs (particularly randomised controlled trials) are not always appropriate The challenge for clinical informatics is to develop multi-perspective evaluations that integrate quantitative and qualitative methods Evaluation is not just for accountability but to improve our understanding of the role of information technology in health care and our ability to deliver systems that offer a wide range of clinical and economic benefits Decision makers may be …

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.316.7149.1959

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The infrequent use of RCTs in EHR evaluations has been due to multiple concerns including ethics, difficulty to control for effect mediators, costs and time-span, impracticalities of randomizing parts of a hospital system, and difficulties in measuring effects across diverse outcomes.

    “This has been due to multiple concerns including: the ethics of such approaches; difficulty to ‘control for’ potentially important effect mediators; associated costs and time-span [17,74]; impracticalities of randomizing parts of a hospital system; and difficulties in measuring the effects across a diverse array of outcomes [13,75,76].”

    From Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems (2012) · cites this paper as [17]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. [12]Van der Loo JA. Overview of published assessment and evaluation studies. In: Van Gennip EMSJ, Talmon JL, editors. Assessment and evaluation of information technologies. Amsterdam: IOS Press; 1995. pp. 64–78. [Google Scholar]
  2. [13]Brennan S, Dodds B. The electronic patient record programme: a voyage of discovery. Br J Healthcare Comput Inf Manage. 1997;14:16–18. [Google Scholar]
  3. [15]Heathfield HA. Decision support systems. In: van Bemmel JH, McCray AT, editors. Yearbook of Medical Informatics 1995. Stuttgart: Schattauer Verlagsgesellschaft mbH; 1995. pp. 455–457. [Google Scholar]
  4. [25]Andersen JG, Aydin CE, Jay SJ. Evaluating healthcare information systems: methods and applications. Thousand Oaks, CA: Sage; 1994. [Google Scholar]
  5. [26]Kaplan B. A model of a comprehensive evaluation plan for complex information systems: clinical imaging systems as an example. In: Brown A, Remeny D, editors. Proceedings of second European conference on information technology investment evaluation. Birmingham: Operational Research Society; 1995. pp. 174–181. [Google Scholar]
  6. [27]Heathfield HA, Hudson P, Kay S, Nicholson L, Peel V, Williams J, et al. Issues in multidisciplinary assessment of healthcare information systems. J IT People (in press).

Cited by

Sorted by citations

Related

Authors
Heathfield, H., Pitty, D., & Hanka, R.
Affiliations
  • Manchester Metropolitan University (GB)
  • Royal Brompton Hospital (GB)
  • University of Cambridge (GB)
Corresponding author
Heathfield, Heather
Venue
BMJ
Publisher
BMJ
Published
1998-06-27
Volume
316
Issue
7149
Pages
1959
ISSN
  • 0007-1447 (ISSN-L)
  • 0959-8138
  • 1468-5833
DOI
10.1136/bmj.316.7149.1959
PMC
PMC1113407
OpenAlex
W2006849636
Cited by
226
References
26
Citations by year
2024: 3 · 2022: 3 · 2021: 7 · 2020: 4 · 2019: 3 · 2018: 4 · 2017: 2 · 2016: 5 · 2015: 9 · 2014: 9 · 2013: 7 · 2012: 13
FWCI
24.46
Type
journal article
Language
en
Open access
Licence
www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
Locations
  • BMJ (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
  • DigitalGeorgetown (Georgetown University Library) (submittedVersion)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
Keywords
health information technology evaluation, clinical decision support systems, cost-effectiveness, multi-perspective evaluation, clinical informatics
MeSH terms
Decision Making, Evaluation Studies as Topic, United Kingdom, Medical Informatics Applications, State Medicine, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review