Measuring the impact of the computer on the consultation: an open source application to combine multiple observational outputs
Abstract
A diverse range of tools and techniques can be used to observe the clinical consultation and the use of information technology. These technologies range from transcripts; to video observation with one or more cameras; to voice and pattern recognition applications. Currently, these have to be observed separately and there is limited capacity to combine them. Consequently, when multiple methods are used to analyse the consultation a significant proportion of time is spent linking events in one log file (e.g. mouse movements and keyboard use when prescribing alerts appear) with what was happening in the consultation at that time. The objective of this study was to develop an application capable of combining and comparing activity log-files and with facilities to view simultaneously all data relating to any time point or activity. Interviews, observations and design prototypes were used to develop a specification. Class diagram of the application design was used to make further development decisions. The application development used object-orientated design principles. We used open source tools; Java as the programming language and JDeveloper as the development environment. The final output is log file aggregation (LFA) tool which forms part of the wider aggregation of log files for analysis (ALFA) open source toolkit ( www.biomedicalinformatics.info/alfa/ ). Testing was done using sample log files and reviewed the application's utility for analysis of the consultation activities. Separation of the presentation and functionality in the design stage enabled us to develop a modular and extensible application. The application is capable of converting and aggregating several log files of different formats and displays them in different presentation layouts. We used the Java Media Framework to aggregate video channels. Java extensible mark-up language (XML) package facilitated the conversion of aggregated output into XML format. Analysts can now move easily between observation tools and find all the data related to an activity. The LFA application makes new analysis tasks feasible and established tasks much more efficient. Researchers can now store multiple log file data as a single file isolate and investigate different doctor-computer-patient interaction.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Roter Interaction Analysis System (RIAS) is a validated and widely used tool that codes verbal utterances into mutually exclusive categories.
“The validated Roter Interaction Analysis System (RIAS) 45 is a widely used tool which codes the verbal 'utterances' of both the patient and clinician into 41 mutually exclusive categories”
From ‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing (2013) · cites this paper as [45]
Cites
The Rational Unified Process: An Introduction
Kruchten, P. B. (1998) · Cited by 2,597 · see citing works
Cognitive and usability engineering methods for the evaluation of clinical information systems.
Kushniruk, A. W., & Patel, V. L. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.01.003 · Cited by 760 · see citing works
(2002) · Computers in health care · doi:10.1007/978-1-4613-0069-4 · Cited by 425 · see citing works
Factors and Forces Affecting EHR System Adoption: Report of a 2004 ACMI Discussion
Ash, J. S. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1684 · Cited by 406 · see citing works
Ram, Grol, Rethans, Schouten, Vleuten, & Kester (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00348.x · Cited by 170 · see citing works
e-Prescribing, Efficiency, Quality: Lessons from the Computerization of UK Family Practice
Schade, C. P., Sullivan, F., de Lusignan, S., & Madeley, J. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2041 · Cited by 113 · see citing works
Toward a theory of organized multimodal integration patterns during human-computer interaction
Oviatt, S. L., Coulston, R., Tomko, S., Xiao, B., Lunsford, R., Wesson, M., & Carmichael, L. M. (2003) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/958432.958443 · Cited by 105 · see citing works
Doctor, patient and computer—A framework for the new consultation
PEARCE, C., Dwan, K., Arnold, M., Phillips, C. B., & Trumble, S. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.07.002 · Cited by 81 · see citing works
Video capture of clinical care to enhance patient safety
Weinger, M. B. (2004) · BMJ Quality & Safety · doi:10.1136/qhc.13.2.136 · Cited by 69 · see citing works
The Development of Primary Care Information Technology in the United Kingdom
de Lusignan, S., & Chan, T. (2008) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000324664.88131.d2 · Cited by 54 · see citing works
McKinstry, B. (2004) · Family Practice · doi:10.1093/fampra/cmh116 · Cited by 31 · see citing works
de Lusignan, S., Kumarapeli, P., Chan, T., Pflug, B., van Vlymen, J., Jones, B., & Freeman, G. K. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.1080 · Cited by 24 · see citing works
A gaze and speech multimodal interface
Zhang, Q., Imamiya, A., Mao, X., & Go, K. (2004) · IEEE International Conference on Distributed Computing Systems Workshops (ICDCSW) · doi:10.1109/icdcsw.2004.1284033 · Cited by 24 · see citing works
Analysing the doctor_patient_computer relationship: the use of video data
Pearce, C. M., Dwan, K., Arnold, M., et al. (2006) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v14i4.633 · Cited by 21 · see citing works
People and Computers XV - Interaction without Frontiers : Joint Proceedings of HCI 2001 and IHM 2001
BLANDFORD, A. E., Vanderdonckt, J. M., & Gray, P. (2001) · Springer eBooks · Cited by 20 · see citing works
Leong, A., Koczan, P., de Lusignan, S., & Sheeler, I. (2006) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230600991668 · Cited by 17 · see citing works
de Lusignan, S., Wells, S., Russell, C., BEVINGTON, W. P., & Arrowsmith, P. (2002) · Medical Informatics and the Internet in Medicine · doi:10.1080/1463923031000081603 · Cited by 10 · see citing works
Refsum, C., Kumarapeli, P., Gunaratne, A. V. C., Dodds, R., Hasan, A., & de Lusignan, S. (2008) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v16i2.683 · Cited by 10 · see citing works
Self-citationLow-cost three-channel video for assessment of the clinical consultation.
Sheeler, I., Koczan, P., Wallage, W., & de Lusignan, S. (2007) · PubMed · Cited by 8 · see citing works
Kumarapeli, P., de Lusignan, S., Koczan, P., Jones, B., & Sheeler, I. (2008) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v15i4.665 · Cited by 8 · see citing works
Self-citationde Lusignan, S., Wilson, E., Dyble, A., Grant, T., Theadom, A., & Chan, T. (2003) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-3-12 · Cited by 8 · see citing works
Moulene, M. V., de Lusignan, S., Freeman, G. K., van Vlymen, J. N., Sheeler, I., Singleton, A., & Kumarapeli, P. (2007) · PubMed · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001484 · Cited by 48 · see citing works
- Authors
- Pflug, B., Kumarapeli, P., van Vlymen, J., Ammenwerth, E., & de Lusignan, S.
- Affiliations
- St George's, University of London (GB)
- Venue
- Informatics for Health and Social Care
- Publisher
- Informa UK Limited
- Published
- 2010-01
- Volume
- 35
- Issue
- 1
- Pages
- 10-24
- ISSN
- 1753-8157 (ISSN-L)
- 1753-8165
- OpenAlex
- W2008171975
- Cited by
- 8
- References
- 22
- Citations by year
- 2016: 1 · 2013: 3 · 2012: 2
- FWCI
- 4.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Informatics for Health and Social Care (publishedVersion)
- PubMed (publishedVersion)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- Research Repository (Kingston University London) (submittedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- computerized physician order entry, multimodal data integration, video observation, open source software, Java Media Framework
- MeSH terms
- Computer Systems, Humans, Software Design, User-Computer Interface, Remote Consultation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
