Patient use and compliance with medical advice delivered by a web-based triage system in primary care
Abstract
We studied a web-based triage system which was accessible to the general public in the Netherlands. In a retrospective analysis we investigated the type of complaints that were submitted and the kind of advice provided. Over a period of 15 months, 13,133 different people began using the web-based triage system and 3812 patients went right through the triage process to the end. The most frequent complaints were common cold symptoms, such as cough and a sore throat (22%), itch problems (13%), urinary complaints (12%), diarrhoea (10%), headache (8%) and lower back pain (8%). Most commonly, the system generated the advice to contact a doctor (85%) and in 15% of the cases the system provided fully automated, problem-tailored, self-care advice. A total of 192 patients participated in a prospective study and completed an online survey immediately after the delivery of advice. A follow-up questionnaire on actual compliance was completed by 35 patients. Among these, 20 (57%) had actually complied with the advice provided by the system. A regression analysis revealed that intention to comply was strongly related to actual compliance. In turn, intention to comply was strongly related to attitude towards the advice ( P < 0.001). Web-based triage can contribute to a more efficient primary care system, because it facilitates the gatekeeper function.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A lack of human centeredness explains the incidence of usability problems and high attrition rates.
“This lack of human centeredness explains the incidence of usability problems [7-9], or high attrition rates [10-18].”
From A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies (2011) · cites this paper as [8]
Six working principles for a holistic development approach were derived from the review of current frameworks and empirical research.
“We introduce six working principles derived from the review of current frameworks, as well as from empirical research [7-9,14,21,56].”
From A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies (2011) · cites this paper as [8]
eHealth technologies often do not fit with the intended users' needs.
“Also, eHealth technologies often do not fit with the intended users’ needs [7-9,14,68-70].”
From A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies (2011) · cites this paper as [8]
Cites
E-health: transforming the physician/patient relationship
Ball, M. J., & Lillis, J. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(00)00130-1 · Cited by 406 · see citing works
Nijland, N., van Gemert-Pijnen, J., Boer, H., Steehouder, M. F., & Seydel, E. R. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.957 · Cited by 150 · see citing works
Self-citationE-Health Care Information Systems: An Introduction for Students and Professionals
Tan, J. (2005) · Cited by 114 · see citing works
Nijland, N., van Gemert-Pijnen, J. E. W. C., Boer, H., Steehouder, M. F., & Seydel, E. R. (2009) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2009.06.002 · Cited by 84 · see citing works
Self-citationChronic Disease Prevention and Management: Implications for Health Human Resources in 2020
Orchard, M. C., Green, E., Sullivan, T. J., Greenberg, A., & Mai, V. (2013) · Healthcare Quarterly · doi:10.12927/hcq.2013.19496 · Cited by 17 · see citing works
Web-Based Triage in a College Health Setting
Sole, M. L., Stuart, P. L., & Deichen, M. (2006) · Journal of American College Health · doi:10.3200/jach.54.5.289-294 · Cited by 15 · see citing works
"For your ears only" : quality of telephone triage at out-of-hours centres in the Netherlands
Derkx, H. (2008) · doi:10.26481/dis.20080618hd · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies
van Gemert-Pijnen, J. E., Nijland, N., van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1672 · Cited by 1,126 · see citing works
Self-citationAsynchronous and synchronous teleconsultation for diabetes care: A systematic literature review
Verhoeven, F., Tanja-Dijkstra, K., Nijland, N., Eysenbach, G., & van Gemert-Pijnen, L. (2010) · Journal of Diabetes Science and Technology · doi:10.1177/193229681000400323 · Cited by 247 · see citing works
Self-citationde Lusignan, S., Mold, F., Sheikh, A., Majeed, A., Wyatt, J. C., Quinn, T., Cavill, M., Gronlund, T. A., Franco, C., Chauhan, U., et al. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-006021 · Cited by 212 · see citing works
Chambers, D., Cantrell, A. J., Johnson, M., Preston, L., Baxter, S. K., Booth, A., & Turner, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027743 · Cited by 218 · see citing works
“A similar level of agreement between algorithm and clinician (39%) was reported by Poote et al,(18) while the system evaluated by Nijland et al advised patients to visit a doctor in 85% of cases, even when the symptoms were appropriate for self-care.(20) The only studies to report clearly equal or superior accuracy of…”
BackgroundMethodInfluentialGottliebsen, K., & Petersson, G. (2020) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100114 · Cited by 80 · see citing works
Methods for Human-Centered eHealth Development: Narrative Scoping Review
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2022) · Journal of Medical Internet Research · doi:10.2196/31858 · Cited by 67 · see citing works
Methods for Human-Centered eHealth Development: Narrative Scoping Review (Preprint)
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2021) · doi:10.2196/preprints.31858 · Cited by 67 · see citing works
Pairon, A., Philips, H., & Verhoeven, V. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2022.1040926 · Cited by 48 · see citing works
Nijland, N. (2010) · doi:10.3990/1.9789036531337 · Cited by 45 · see citing works
Self-citationWachtler, C., Coe, A., Davidson, S. K., Fletcher, S. L., Mendoza, A., Sterling, L., & Gunn, J. M. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9502 · Cited by 41 · see citing works
Yardley, L., Joseph, J., Michie, S., Weal, M., Wills, G., & Little, P. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1599 · Cited by 38 · see citing works
Chambers, D., Cantrell, A. J., Johnson, M., Preston, L. R., Baxter, S. K., Booth, A., & Turner, J. (2019) · Health Services and Delivery Research · doi:10.3310/hsdr07290 · Cited by 35 · see citing works
Mold, F., & de Lusignan, S. (2015) · Journal of Personalized Medicine · doi:10.3390/jpm5040452 · Cited by 30 · see citing works
de Graaf, M., Totté, J. E. E., Breugem, C. C., van Os‐Medendorp, H., & Pasmans, S. G. M. A. (2013) · JMIR Research Protocols · doi:10.2196/resprot.2897 · Cited by 29 · see citing works
Ethical, legal, and social aspects of symptom checker applications: a scoping review
Müller, R., Klemmt, M., Ehni, H., Henking, T., Kuhnmünch, A., Preiser, C., Koch, R., & Ranisch, R. (2022) · Medicine Health Care and Philosophy · doi:10.1007/s11019-022-10114-y · Cited by 29 · see citing works
A study of automated self-assessment in a primary care student health centre setting
Poote, A., French, D., Dale, J. R., & powell, J. A. (2014) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x14529246 · Cited by 27 · see citing works
Jiang, Y., Sereika, S. M., Dabbs, A. D., Handler, S. M., & Schlenk, E. A. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.07.012 · Cited by 26 · see citing works
Spinou, A. (2020) · Physiological Research · doi:10.33549/physiolres.934407 · Cited by 26 · see citing works
Performance of a new symptom checker in patient triage: Canadian cohort study
Chan, F., Lai, S., Pieterman, M., Richardson, L., Singh, A., Peters, J., Toy, A., Piccininni, C. R., Rouault, T., Wong, K., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0260696 · Cited by 23 · see citing works
Santé digitale : promesses, défis et craintes. Une revue de la littérature
Carral, M. D. R., Roux, P., Bruchez, C., & Santiago-Delefosse, M. J. (2016) · Pratiques Psychologiques · doi:10.1016/j.prps.2016.06.004 · Cited by 21 · see citing works
Triage Errors in Primary and Pre–Primary Care
Nguyen, H., Meczner, A., Burslam-Dawe, K., & Hayhoe, B. W. J. (2022) · Journal of Medical Internet Research · doi:10.2196/37209 · Cited by 21 · see citing works
Gustafsson, S. R., Vikman, I., Axelsson, K., & Sävenstedt, S. (2014) · Primary Health Care Research & Development · doi:10.1017/s1463423613000522 · Cited by 21 · see citing works
North, F., Jensen, T. B., Stroebel, R. J., Nelson, E. M., Johnson, B. J., Thompson, M. C., Pecina, J. L., & Crum, B. A. (2023) · Health Services Research and Managerial Epidemiology · doi:10.1177/23333928231168121 · Cited by 11 · see citing works
User Experience of Symptom Checkers: A Systematic Review
You, Y., Ma, R., & Gui, X. (2022) · PubMed Central · doi:10.48550/arxiv.2208.09100 · Cited by 8 · see citing works
E-Health-Angebote in der Onkologie
Lange, L., Schulz, H., & Bleich, C. (2018) · Der Onkologe · doi:10.1007/s00761-018-0348-5 · Cited by 6 · see citing works
A multimethod study of NHS 111 online
Turnbull, J. C., MacLellan, J., Churruca, K., Ellis, L. A., Prichard, J. S., Browne, D., Braithwaite, J., Petter, E., Chisambi, M., & Pope, C. J. (2023) · Health and Social Care Delivery Research · doi:10.3310/ytrr9821 · Cited by 5 · see citing works
Implementation costs of telephone nurse triage service
Rebolho, R. C., & Raupp, F. M. (2023) · Cadernos de Saúde Pública · doi:10.1590/0102-311xen095522 · Cited by 4 · see citing works
Lowe, C., Atherton, L., Lloyd, P. A., Waters, A., & Morrissey, D. (2025) · Journal of Medical Internet Research · doi:10.2196/67269 · Cited by 4 · see citing works
eHealth Wikiplatform to Increase the Uptake and Impact of eHealth Technologies
van Gemert‐Pijnen, J. E. W. C., Nijland, N., van Limburg, M., Kelders, S. M., van Velsen, L., Ossebaard, H. C., & Brandenburg, B. J. (2011) · University of Twente Research Information · Cited by 3 · see citing works
Self-citationSampson, F. C., Knowles, E., Long, J., Turner, J., & Coster, J. E. (2023) · Emergency Medicine Journal · doi:10.1136/emermed-2022-213009 · Cited by 1 · see citing works
Sharma, R., Latimer‐Cheung, A. E., Cairney, J., & Arbour‐Nicitopoulos, K. P. (2017) · TSpace (University of Toronto)
Painter, A., Hayhoe, B. W. J., & El-Osta, A. A. R. (2022) · doi:10.2196/preprints.37408
Sampson, F. C., Knowles, E., Long, J., Turner, J., & Coster, J. E. (2022) · medRxiv · doi:10.1101/2022.11.29.22282892
Austin, L., Kfrerer, M., Faulds, C., Austin, R. D., Ziebart, C., & Pepe, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4917165
Nijland, N., C., V., Kelders, S. M., Brandenburg, B. J., & Seydel, E. R. (2010) · Studies in health technology and informatics · doi:10.3233/978-1-60750-588-4-821
Self-citationAustin, L., Kfrerer, M., Faulds, C., Austin, R. D., Ziebart, C., & Pepe, D. (2025) · medRxiv · doi:10.64898/2025.12.09.25341895
Curry, E. N., Miller, N. E., North, F., Thompson, M. C., & Pecina, J. L. (2026) · Telemedicine Journal and e-Health · doi:10.1177/15305627261440766
Symptom checkers: Gaps in their evidence base and suggestions for accuracy evaluation
Donner‐Banzhoff, N., Schaefer, H., Schlicker, N., Becker, A., Haasenritter, J., & van der Wardt, V. (2026) · Digital Health · doi:10.1177/20552076261487359
- Authors
- Nijland, N., Cranen, K., Boer, H., van Gemert-Pijnen, J. E. W. C., & Seydel, E. R.
- Affiliations
- University of Twente (NL)
- University of Twente, Enschede, The Netherlands
- Corresponding author
- Nijland, Nicol
- ORCID
- Cranen, Karlijn
- Venue
- Journal of Telemedicine and Telecare
- Publisher
- SAGE Publications
- Published
- 2010-01
- Volume
- 16
- Issue
- 1
- Pages
- 8-11
- ISSN
- 1357-633X (ISSN-L)
- 1758-1109
- OpenAlex
- W2096336416
- Cited by
- 37
- References
- 7
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 1 · 2023: 5 · 2022: 5 · 2021: 2 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 1 · 2016: 2 · 2015: 1 · 2014: 3 · 2013: 1
- FWCI
- 2.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Telemedicine and Telecare (publishedVersion)
- PubMed (publishedVersion)
- University of Twente Research Information (publishedVersion)
- University of Twente Research Information (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- medication adherence, medical advice, self-care advice, intention to comply, primary care, gatekeeper function, online health information
- MeSH terms
- Adolescent, Adult, Decision Making, Computer-Assisted, Female, Humans, Male, Netherlands, Patient Compliance, Patient Dropouts, Primary Health Care, Retrospective Studies, Self Care, Severity of Illness Index, Triage, Remote Consultation, Internet, Gatekeeping, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
