Patient-specific computer-based decision support in primary healthcare—a randomized trial
Open accessThe DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Computer-based decision support systems are a promising method for incorporating research evidence into clinical practice. However, evidence is still scant on how such information technology solutions work in primary healthcare when support is provided across many health problems. In Finland, we designed a trial where a set of evidence-based, patient-specific reminders was introduced into the local Electronic Patient Record (EPR) system. The aim was to measure the effects of such reminders on patient care. The hypothesis was that the total number of triggered reminders would decrease in the intervention group compared with the control group, indicating an improvement in patient care. METHODS: From July 2009 to October 2010 all the patients of one health center were randomized to an intervention or a control group. The intervention consisted of patient-specific reminders concerning 59 different health conditions triggered when the healthcare professional (HCP) opened and used the EPR. In the intervention group, the triggered reminders were shown to the HCP; in the control group, the triggered reminders were not shown. The primary outcome measure was the change in the number of reminders triggered over 12 months. We developed a unique data gathering method, the Repeated Study Virtual Health Check (RSVHC), and used Generalized Estimation Equations (GEE) for analysing the incidence rate ratio, which is a measure of the relative difference in percentage change in the numbers of reminders triggered in the intervention group and the control group. RESULTS: In total, 13,588 participants were randomized and included. Contrary to our expectation, the total number of reminders triggered increased in both the intervention and the control groups. The primary outcome measure did not show a significant difference between the groups. However, with the inclusion of patients followed up over only six months, the total number of reminders increased significantly less in the intervention group than in the control group when the confounding factors (age, gender, number of diagnoses and medications) were controlled for. CONCLUSIONS: Computerized, tailored reminders in primary care did not decrease during the 12 months of follow-up time after the introduction of a patient-specific decision support system. TRIAL REGISTRATION: ClinicalTrial.gov NCT00915304.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, after 6 months, the incidence of reminders was significantly lower in the intervention group compared with the control group, especially when the data were adjusted for confounding factors such as age, gender, comorbidities, and medications.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [108]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“After 12 months, the incidence of trigger reminders increased in both intervention and control groups, but overall no differences were observed between the two groups.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [108]
The cited source says what the claim attributes to it.
In the citing paper
“especially when the data were adjusted for confounding factors such as age, gender, comorbidities, and medications.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [108]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · cited 1×
“Liang and Zeger proposed the GEE approach in 1986 to deal with impractical probability distribution in handling correlated responses [30].”
BackgroundMcCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
(2005) · Cited by 3,748
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · cited 2×
“The EBMeDS service, developed using the best available evidence [9,10], aimed to offer recommendations to HCPs ’ workflow across several conditions in primary care practice.”
MethodKawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
Akaike's Information Criterion in Generalized Estimating Equations
Pan, W. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00120.x · Cited by 2,493 · cited 1×
“All models were evaluated in terms of how well they fitted the data using the quasi-likelihood under the independence model information criterion (QIC) for model selection [31].”
MethodThe Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442 · cited 1×
“Our results reaffirm previous evidence [39-41] that implementation of computer-based decision support is problematic.”
ResultGeneration of allocation sequences in randomised trials: chance, not choice
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07683-3 · Cited by 774
Research designs for studies evaluating the effectiveness of change and improvement strategies
Eccles, M. (2003) · Quality and Safety in Health Care · doi:10.1136/qhc.12.1.47 · Cited by 586 · cited 1×
“We used the robust RCT method [32,33], and the most likely explanation for the results is that the recording of diagnostic codes improved markedly during the trial (Table 3).”
MethodClinical expertise in the era of evidence-based medicine and patient choice
Haynes, R. B. (2002) · Evidence-Based Medicine · doi:10.1136/ebm.7.2.36 · Cited by 586 · cited 1×
“Background The treatment of patients is based on clinical expertise, whose key elements are research evidence, clinical situations and circumstances, and patients’ preferences and actions [1].”
BackgroundImproving Patient Care: The Implementation of Change in Clinical Practice
Cosby, J. L. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2005.016824 · Cited by 557
Improving Patient Care: The Implementation of Change in Clinical Practice
Grol, R. P. T. M., Wensing, M., Eccles, M., Davis, D., & Wensign, M. (2005) · Cited by 553
Effects of evidence-based clinical practice guidelines on quality of care: a systematic review
Lugtenberg, M., Burgers, J. S., & Westert, G. P. (2009) · Quality and Safety in Health Care · doi:10.1136/qshc.2008.028043 · Cited by 478 · cited 2×
“Tailoring the guidance to HCPs ’ needs has been indicated as a key issue for successful implementation [5,7].”
BackgroundMethodThe effects of on-screen, point of care computer reminders on processes and outcomes of care
Shojania, K. G., Jennings, A., Ramsay, C. R., Grimshaw, J. M., Kwan, J. L., & Lo, L. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001096.pub2 · Cited by 466 · cited 1×
“According to a Cochrane Review of 28 studies, computer reminders achieved a median improvement in process adherence of 4.2% [12].”
MethodInfluentialRoshanov, P. S., Fernandes, N., Wilczynski, J. M., Hemens, B. J., You, J. J., Handler, S. M., Nieuwlaat, R., Souza, N. M., Beyene, J., Spall, H. G. C. V., et al. (2013) · BMJ · doi:10.1136/bmj.f657 · Cited by 456 · cited 1×
“According to a meta-regression of 162 randomized trials [35], the odds of success of computer-based decision support are greater for systems that require HCPs to provide reasons when overriding advice than for systems that do not.”
MethodImproving Preventive Care by Prompting Physicians
Balas, E. A. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.3.301 · Cited by 385 · cited 1×
“Focused computer-generated reminders and alerts work well in a variety of single conditions [13-16] and in preventive care [17].”
BackgroundComputer Programs to Support Clinical Decision Making
Shortliffe, E. H. (1987) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1987.03400010065029 · Cited by 331 · cited 1×
“Such automatic systems combine medical evidence with patient-specific data from the Electronic Patient Record (EPR), which supports clinical decision making [9-11].”
Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs
BURGERS, J. S. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/15.1.31 · Cited by 331 · cited 1×
“The evidence is translated into practical form, for example, in clinical practice guidelines [2], whereas active incorporation of these into everyday practice has only recently become a recognized target for research [3,4].”
BackgroundSelf-citationMedical Statistics-A Commonsense Approach.
Rees, D. G., Campbell, M. J., & Machin, D. (1991) · The Statistician · doi:10.2307/2348298 · Cited by 291
Clinical Decision Support: The Road Ahead
Greenes, R. A. (2006) · Cited by 262
Sequist, T. D., Gandhi, T. K., Karson, A. S., Fiskio, J. M., Bugbee, D., Sperling, M., Cook, E. F., Orav, E. J., Fairchild, D. G., & Bates, D. W. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1788 · Cited by 251
Clinical expertise in the era of evidence-based medicine and patient choice
(2002) · ACP Journal Club · doi:10.7326/acpjc-2002-136-2-a11 · Cited by 249
Bryan, C., & Boren, S. (2008) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v16i2.679 · Cited by 170 · cited 1×
“There is still, however, scant evidence on how such information technology solutions work across many diseases or conditions in primary healthcare where multi-professional teams [20,21] care for patients with multiple health problems, both acute and chronic [22,23].”
BackgroundElectronic Health Record–Based Decision Support to Improve Asthma Care: A Cluster-Randomized Trial
Bell, L. M., Grundmeier, R., Localio, R., Zorc, J., Fiks, A. G., Zhang, X., Stephens, T. B., Swietlik, M., & Guevara, J. P. (2010) · Pediatrics · doi:10.1542/peds.2009-1385 · Cited by 146 · cited 1×
“Focused computer-generated reminders and alerts work well in a variety of single conditions [13-16] and in preventive care [17].”
BackgroundVaronen, H., Kortteisto, T., Kaila, M., & for the EBMeDS Study Group (2008) · Family Practice · doi:10.1093/fampra/cmn020 · Cited by 139 · cited 1×
“Local HCPs, not only the Chief Medical Officer, may have to be involved for an adequate understanding of their needs as the starting point for developing and implementing reminders, as had been indicated previously [34].”
BackgroundSelf-citationWright, A., Pang, J., Feblowitz, J. C., Maloney, F. L., Wilcox, A. R., McLoughlin, K. S., Ramelson, H., Schneider, L., & Bates, D. W. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000521 · Cited by 88 · cited 1×
“This confirms previous findings [43,44].”
Information technology for patient safety
Huckvale, C., Car, J., Akiyama, M., Jaafar, S., Khoja, T. A. M., Khalid, A. B., Sheikh, A., & Majeed, A. (2010) · BMJ Quality & Safety · doi:10.1136/qshc.2009.038497 · Cited by 85 · cited 1×
“Decision support can in many settings improve the quality of care and help to avoid mistakes in clinical work, thereby improving patient safety [18,19].”
BackgroundNurses’ use of online clinical evidence
Gosling, A. S., Westbrook, J. I., & Spencer, R. (2004) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2004.03079.x · Cited by 85 · cited 1×
“There is still, however, scant evidence on how such information technology solutions work across many diseases or conditions in primary healthcare where multi-professional teams [20,21] care for patients with multiple health problems, both acute and chronic [22,23].”
BackgroundHeselmans, A., Aertgeerts, B., Donceel, P., Geens, S., de Velde, S. V., & Ramaekers, D. (2012) · Journal of Medical Systems · doi:10.1007/s10916-012-9841-3 · Cited by 82 · cited 1×
“HCPs seem to accept the service in principle [27,42], but in practice they may neglect using the reminders for many practical reasons.”
BackgroundHeselmans, A., Van de Velde, S., Donceel, P., Aertgeerts, B., & Ramaekers, D. (2009) · Implementation Science · doi:10.1186/1748-5908-4-82 · Cited by 71
Fransen, G. A., van Marrewijk, C. J., Mujakovic, S., Muris, J. W., Laheij, R. J., Numans, M. E., de Wit, N. J., Samsom, M., Jansen, J. B., & Knottnerus, J. A. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-16 · Cited by 67
Kortteisto, T., Komulainen, J., Mäkelä, M., Kunnamo, I., & Kaila, M. (2012) · BMC Health Services Research · doi:10.1186/1472-6963-12-349 · Cited by 66 · cited 4×
“HCPs seem to accept the service in principle [27,42], but in practice they may neglect using the reminders for many practical reasons.”
BackgroundInfluentialSelf-citationSmoking As a Vital Sign: Prompts to Ask and Assess Increase Cessation Counseling
McCullough, A., Fisher, M., Goldstein, A. O., Kramer, K. D., & Ripley-Moffitt, C. (2009) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2009.06.080211 · Cited by 48 · cited 1×
“Focused computer-generated reminders and alerts work well in a variety of single conditions [13-16] and in preventive care [17].”
BackgroundPagliari, C., Sheikh, A., Mukherjee, M., Cresswell, K. M., McLean, S., Nurmatov, U. B., McKinstry, B., Hemmi, A., Procter, R., Car, J., et al. (2011) · Cited by 46
Effect of a computerized body mass index prompt on diagnosis and treatment of adult obesity.
Schriefer, S. P., Landis, S. E., Turbow, D., & Patch, S. C. (2009) · PubMed · Cited by 38
Impact of Decision Support in Electronic Medical Records on Lipid Management in Primary Care
Gill, J. M., Chen, Y. X., Glutting, J. J., Diamond, J. J., & Lieberman, M. I. (2009) · Population Health Management · doi:10.1089/pop.2009.0003 · Cited by 33 · cited 1×
“There is still, however, scant evidence on how such information technology solutions work across many diseases or conditions in primary healthcare where multi-professional teams [20,21] care for patients with multiple health problems, both acute and chronic [22,23].”
BackgroundMartens, J. D., van der Weijden, T., Winkens, R. A. G., Kester, A. D. M., Geerts, P. J. H., Evers, S., & Severens, J. L. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.05.013 · Cited by 30 · cited 1×
“Our results reaffirm previous evidence [39-41] that implementation of computer-based decision support is problematic.”
ResultHerzberg, S., Rahbar, K., Stegger, L., Schäfers, M. A., & Dugas, M. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.02.004 · Cited by 23 · cited 1×
“This confirms previous findings [43,44].”
Screening for abdominal aortic aneurysm with electronic clinical reminders
Padberg, F. T., Hauck, K., Mercer, R. G., Lal, B. K., & Pappas, P. J. (2009) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2009.07.021 · Cited by 14 · cited 1×
“Focused computer-generated reminders and alerts work well in a variety of single conditions [13-16] and in preventive care [17].”
BackgroundMEDICAL STATISTICS: A COMMONSENSE APPROACH 3rd edition
Jones, P. W. (2001) · The Obstetrician & Gynaecologist · doi:10.1576/toag.2001.3.2.103 · Cited by 10 · cited 1×
“We used the robust RCT method [32,33], and the most likely explanation for the results is that the recording of diagnostic codes improved markedly during the trial (Table 3).”
MethodImplementing clinical decision support for primary care professionals - the process
Kortteisto, T., Komulainen, J., Kunnamo, I., Mākelä, M., & Kaila, M. (2012) · Finnish Journal of eHealth and eWelfare · Cited by 4
Self-citation(1995) · PubMed
Research works printed but not linked: 2
- Hojgaard L Forward look - implementation of medical research in clinical practice 2011 Strasbourg: European science foundation
- Car J Black A Anandan C Cresswell K Pagliari C McKinstry B Procter R Majeed A Sheikh A The impact of eHealth on the quality & safety of healthcare. A systematic overview & synthesis of the literature. Report for the NHS Connecting for Health Evaluation Programme. 2008 London: Imperial College London
Cited by
The Empirical Foundations of Telemedicine Interventions in Primary Care
Bashshur, R. L., Howell, J. D., Krupinski, E. A., Harms, K. M., Bashshur, N., & Doarn, C. R. (2016) · Telemedicine and e-Health · doi:10.1089/tmj.2016.0045 · Cited by 370
“Kortteisto et al.(108) 2014 Finland RCT 13,568 PHR Text reminders 12 months Quality [ Volume of reminders lower in intervention group”
Keyworth, C., Hart, J., Armitage, C. J., & Tully, M. P. (2018) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-018-0661-3 · Cited by 149
“Kortteisto [72] 2014 Primary Care Physicians nurses physiotherapists ward nurses a psychologist Reminders for best practice guidelines/recommendations Adherence to clinical patient management guidelines Healthcare professional decision supports Y Y N na”
BackgroundKwag, K. H., González‐Lorenzo, M., Banzi, R., Bonovas, S., & Moja, L. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5234 · Cited by 113
Sönnichsen, A. C., Trampisch, U. S., Rieckert, A., Piccoliori, G., Vögele, A., Flamm, M., Johansson, T., Esmail, A., Reeves, D., Löffler, C., et al. (2016) · Trials · doi:10.1186/s13063-016-1177-8 · Cited by 115
“An early version of EBMeDS containing a variety of decision support rules was evaluated in a randomized controlled trial [36].”
MethodSelf-citationMoja, L., Polo Friz, H., Capobussi, M., Kwag, K., Banzi, R., Ruggiero, F., González-Lorenzo, M., Liberati, E. G., Mangia, M., Nyberg, P., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.17094 · Cited by 68
Self-citation
Metadata
- Authors
- Kortteisto, T., Raitanen, J., Komulainen, J., Kunnamo, I., Mäkelä, M., Rissanen, P., & Kaila, M.
- Affiliations
- University of Tampere (FI)
- Urho Kaleva Kekkonen Institute (FI)
- Finnish Medical Society Duodecim (FI)
- University of Helsinki (FI)
- Corresponding author
- Kortteisto, Tiina
- Venue
- Implementation Science
- Published
- 2014-01-20
- Volume
- 9
- Issue
- 1
- Pages
- 15-15
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3901002
- OpenAlex
- W2170819724
- Semantic Scholar
- 397440f43bb358ce9ae20c21fd77818e43fd9adb
- Cited by
- 27
- References
- 45
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 1 · 2021: 3 · 2019: 3 · 2018: 3 · 2017: 4 · 2016: 6 · 2015: 4
- FWCI
- 8.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Tampere University Institutional Repository (Tampere University) (publishedVersion)
- Työväentutkimus Vuosikirja (submittedVersion)
- Tampere University Institutional Repository (Tampere University) (submittedVersion)
- STM:n Hallinnonalan avoin julkaisuarkisto (Julkari) (submittedVersion)
- Funders
- Tekes
- Suomalainen Lääkäriseura Duodecim
- Terveyden ja hyvinvoinnin laitos
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- computer-based decision support, electronic health records, primary care, evidence-based practice, clinical reminders, generalized estimating equations
- MeSH terms
- Adolescent, Adult, Child, Female, Finland, Humans, Information Systems, Male, Middle Aged, Primary Health Care, Quality of Health Care, Reminder Systems, Decision Support Systems, Clinical, Young Adult, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.