Interobserver Reliability of Store-and-Forward Teledermatology in a Clinical Practice Setting
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Although many studies have evaluated the diagnostic reliability of store-and-forward (SF) teledermatology, the reliability of the technique for the diagnosis of general skin conditions in a clinical practice setting has never been demonstrated. We evaluated the reliability of SF teledermatology in clinical practice by analyzing the diagnostic agreement achieved in a subgroup of patients from the DERMATEL-2 study. Patients referred from primary care settings were randomized to 3 groups: SF, a combination of videoconferencing and SF technology (VC-SF), and a control group. This article focuses on the SF group. Clinical data were recorded and photographs taken by primary care physicians, who forwarded the data digitally. Each SF consultation package was assessed by 3 dermatologists (D1,D2,D3). Subsequently all the patients were assessed by a single dermatologist (D1) in a face-to-face (FF) consultation. Finally, 2 other dermatologists (D4,D5) assessed the agreement between the diagnoses obtained by SF and FF. In total, 457 patients (200 males and 257 females) aged between 2 months and 86 years were randomized (192 to SF, 176 to VC-SF, and 89 to the control group). The diagnostic categories were as follows: tumors (49.4%), inflammatory (25.7%), adnexal (11%), infectious (9.4%) and other processes (4.4%). Since 170 patients had SF consultations deemed valid for analysis, the study included a total of 510 SF assessments. Most of the images and clinical records were of high quality (71.2% and 91.2% respectively), and diagnostic confidence was high in 81.4% of the cases studied. In 58.4% of cases the condition was managed exclusively by teledermatology. Levels of complete and aggregate interobserver agreement between SF and FF evaluators were 0.72 and 0.90, respectively, for diagnosis and 0.61 and 0.80 for treatment. Diagnostic agreement correlated with the image quality (P < .001), diagnostic confidence (P < .001), felt need for conventional consultation (P < .001), and the quality of the clinical record (P = .013). The interobserver reliability of SF diagnosis in clinical practice is good. Dermatologists are able to predict errors in diagnosis by analyzing their own diagnostic confidence and evaluating the quality of the images. Aunque existen múltiples estudios de fiabilidad diagnóstica en teledermatología de almacenamiento (TDA), aún no se ha demostrado una fiabilidad elevada para enfermedad general cutánea en un escenario real. DERMATEL-2 fue un estudio aleatorizado de concordancia diagnóstica en TDA en condiciones de práctica clínica. Pacientes remitidos desde atención primaria fueron aleatorizados en 3 grupos: TDA; híbrida videoconferencia-almacenamiento (VC-TDA) y grupo control (GC). Este artículo se centra en el grupo de almacenamiento. Médicos de atención primaria tomaron datos y fotografías clínicas remitiéndolas a distancia. Cada consulta de TDA fue evaluada por 3 dermatólogos diferentes (D1,D2,D3). Todos los pacientes fueron finalmente vistos por el mismo dermatólogo (D1) en la consulta presencial (CP). Dos dermatólogos adicionales (D4, D5) evaluaron las concordancias TDA-CP. Se aleatorizaron un total de 457 pacientes 4-4-2: 192 TDA, 176 VC-TDA y 89 GC; 200 varones y 257 mujeres, 0-86 años. Se incuyó enfermedad tumoral (49,4%), inflamatoria (25,7%), anexial (11%), infecciosa (9,4%) y otros (4,4%). Hubo 170 pacientes de TDA válidos para el análisis, rindiendo 510 teleconsultas-TDA. La imagen (71,2%), la historia clínica (91,2%) y la confianza diagnóstica (81,4%) fueron de calidad alta. En el 58,4% fue posible el manejo exclusivo on-line. Los acuerdos interobservador (completo/agregado) TDA-CP fueron 0,72/0,90 para el diagnóstico y 0,61/0,80 en el tratamiento. El acuerdo diagnóstico se correlacionó con la calidad de la imagen (p < 0,001), la confianza diagnóstica (p < 0,001), la necesidad de consulta presencial (p < 0,001) y la calidad de la historia clínica (p = 0,013). La fiabilidad diagnóstica de la TDA en condiciones de práctica clínica es elevada. Los dermatólogos pueden predecir errores diagnósticos analizando su confianza diagnóstica y la calidad de las fotografías.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Teledermatology for diagnosis and management of skin conditions: A systematic review
Warshaw, E. M., Hillman, Y. J., Greer, N. L., Hagel, E. M., MacDonald, R., Rutks, I. R., & Wilt, T. J. (2011) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2010.08.026 · Cited by 310 · cited 4×
“VC requires greater coordination and consumes more time and resources.(4,5) It can also be more costly than conventional face-to-face (FF) consultation.”
BackgroundResultInfluentialAssessment of the accuracy of low-cost store-and-forward teledermatology consultation
High, W. A., Houston, M. S., Calobrisi, S. D., Drage, L. A., & McEvoy, M. T. (2000) · Journal of the American Academy of Dermatology · doi:10.1067/mjd.2000.104519 · Cited by 183
Store-and-Forward Teledermatology in Skin Cancer Triage
Moreno-Ramirez, D., Ferrandiz, L., Nieto-Garcia, A., Carrasco, R., Moreno-Alvarez, P., Galdeano, R., Bidegain, E., Rios-Martin, J. J., & Camacho, F. M. (2007) · Archives of Dermatology · doi:10.1001/archderm.143.4.479 · Cited by 163
van der Heijden, J. P., de Keizer, N. F., Bos, J. D., Spuls, P. I., & Witkamp, L. (2011) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2011.10509.x · Cited by 160
Reliability and accuracy of dermatologists’ clinic-based and digital image consultations
Whited, J. D., Hall, R. P., Simel, D. L., Foy, M. E., Stechuchak, K. M., Drugge, R. J., Grichnik, J. M., Myers, S. A., & Horner, R. D. (1999) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(99)70003-4 · Cited by 155
Diagnostic Accuracy and Image Quality Using a Digital Camera for Teledermatology
Krupinski, E. A., LeSueur, B., Ellsworth, L., Levine, N., Hansen, R., Silvis, N., Sarantopoulos, P., Hite, P., Wurzel, J., Weinstein, R. S., et al. (1999) · Telemedicine Journal · doi:10.1089/107830299312005 · Cited by 154 · cited 3×
“In the study by Krupinski et al.(14) the photographs were taken by a medical student under the direction of a dermatologist, and in the study by Lyon et al.”
MethodResultPak, H., Triplett, C. A., Lindquist, J. H., Grambow, S. C., & Whited, J. D. (2007) · Journal of Telemedicine and Telecare · doi:10.1258/135763307779701185 · Cited by 125
Telemedicine in dermatology: a randomised controlled trial
Bowns, I., Collins, K., Walters, S., & McDonagh, A. (2006) · Health Technology Assessment · doi:10.3310/hta10430 · Cited by 109
Telemedicine evaluation of cutaneous diseases: A blinded comparative study
Lesher, Jr., Davis, L. S., Gourdin, F. W., English, D., & Thompson, W. O. (1998) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(98)70534-1 · Cited by 105
Heffner, V. A., Lyon, V. B., Brousseau, D. C., Holland, K. E., & Yen, K. (2009) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2008.11.026 · Cited by 97 · cited 1×
“Dermatology appears to be an ideal specialty for telemedicine.(3) Indeed, it is the clinical specialty that currently accounts for the largest number of telemedicine studies.”
BackgroundLim, A. C., Egerton, I. B., See, A., & Shumack, S. P. (2001) · Australasian Journal of Dermatology · doi:10.1046/j.1440-0960.2001.00529.x · Cited by 93 · cited 1×
“Intraobserver diagnostic agreement was very high in our study, with values comparable to those reported by the study in the literature reporting the best results for reliability.(25) In addition, our findings relating to reliability (SF vs SF intramodality agreement) are similar to those published previously.”
ResultEdison, K. E., Ward, D. S., Dyer, J. A., Lane, W., Chance, L., & Hicks, L. L. (2008) · Telemedicine and e-Health · doi:10.1089/tmj.2008.0001 · Cited by 85 · cited 1×
“While VC allows the specialist to interact directly with the patient, the reliability of the method (as measured by diagnostic agreement) can be similar to that of SF teleconsultation.(4,7,8) Thus, medical professionals and healthcare providers increasingly consider SF to be the most cost effective and convenient syst…”
MethodTeledermatology in the U.K.: lessons in service innovation
Finch, T. L., Mair, F. S., & May, C. R. (2007) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2006.07608.x · Cited by 84
Mahendran, R., Goodfield, M. J. D., & Sheehan-Dare, R. A. (2005) · Clinical and Experimental Dermatology · doi:10.1111/j.1365-2230.2005.01735.x · Cited by 80
The reliability of diagnosis using store-and-forward teledermatology
Du Moulin, M. F. M. T., Bullens-Goessens, Y. I. J. M., Henquet, C. J. M., Brunenberg, D. E. M., de Bruyn-Geraerds, D. P., Winkens, R. A. G., Dirksen, C. D., Vierhout, W. P. M., & Neumann, H. A. M. (2003) · Journal of Telemedicine and Telecare · doi:10.1258/135763303769211247 · Cited by 72 · cited 5×
“conditions rather than the ideal conditions created in the other studies.(12) Some studies focusing exclusively on skin tumors have achieved good reliability in a clinical setting, an outcome that supports our findings.”
BackgroundResultInfluentialBaba, M., Seçkin, D., & Kapdağli, S. (2005) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x0501100706 · Cited by 61 · cited 2×
“In addition, our findings relating to reliability (SF vs SF intramodality agreement) are similar to those published previously.(7,14,16,17) In DERMATEL, diagnostic agreement correlated with the teledermatologists’ confidence in their own diagnosis and was associated with the recommendation that an in-person consultati…”
MethodResultRomero, G., Sánchez, P., García, M., Cortina, P., Vera, E., & Garrido, J. A. (2010) · Clinical and Experimental Dermatology · doi:10.1111/j.1365-2230.2009.03503.x · Cited by 55 · cited 4×
“Although the correlation between diagnostic agreement and diagnostic confidence has been studied extensively in the literature,(7,14,18) the present study was the first to analyze the correlation between diagnostic agreement and the decision to recommend an in-person consultation.(4) Our statistical analysis showed wi…”
BackgroundMethodInfluentialTelemedicina y teledermatología (II)
Romero, G., Cortina, P., & Vera, E. (2008) · Actas Dermo-Sifiliográficas · doi:10.1016/s0001-7310(08)74755-0 · Cited by 22
Research works printed but not linked: 14
- Wootton, 2000, Multicentre randomised control trial comparing real time teledermatology with conventional outpatient dermatological care: Societal cost-benefit analysis, BMJ, 320, 1252, 10.1136/bmj.320.7244.1252 · DOI · Paper page
- Eminovic, 2007, Maturity of teledermatology evaluation research: A systematic literature review, Br J Dermatol, 156, 412, 10.1111/j.1365-2133.2006.07627.x · DOI · Paper page
- Lyon, 1997, Digital imaging and teledermatology: Educational and diagnostic applications of a portable digital imaging system for the trainee dermatologist, Clin Exp Dermatol, 22, 163, 10.1111/j.1365-2230.1997.tb01052.x · DOI · Paper page
- Kvedar, 1999, Teledermatology in a capitated delivery system using distributed information architecture: Design and development, Telemed J, 5, 357, 10.1089/107830299311925 · DOI · Paper page
- Pak, 2003, Teledermatology: An intraobserver diagnostic correlation study, part ii, Cutis, 71, 476
- Taylor, 2000, An assessment of the potential effect of a teledermatology system, J Telemed Telecare, 6, S74, 10.1258/1357633001934221 · DOI · Paper page
- Zelickson, 1997, Teledermatology in the nursing home, Arch Dermatol, 133, 171, 10.1001/archderm.1997.03890380041006 · DOI · Paper page
- Romero, 2008, Telemedicine and teledermatology (i): Concepts and applications, Actas Dermosifiliogr, 99, 506, 10.1016/S0001-7310(08)74735-5 · DOI
- Wootton, 2011, Estimating travel reduction associated with the use of telemedicine by patients and healthcare professionals: Proposal for quantitative synthesis in a systematic review, BMC Health Serv Res, 11, 185, 10.1186/1472-6963-11-185 · DOI · Paper page
- Ferrandiz, 2008, An economic analysis of presurgical teledermatology in patients with nonmelanoma skin cancer, Actas Dermosifiliogr, 99, 795
- Pak, 2009, Cost minimization analysis of a store-and-forward teledermatology consult system, Telemed J E Health, 15, 160, 10.1089/tmj.2008.0083 · DOI · Paper page
- Ferrandiz, 2012, Effect of teledermatology on the prognosis of patients with cutaneous melanoma, Arch Dermatol, 148, 1025, 10.1001/archdermatol.2012.778 · DOI · Paper page
- Warshaw, 2009, Accuracy of teledermatology for pigmented neoplasms, J Am Acad Dermatol, 61, 753, 10.1016/j.jaad.2009.04.032 · DOI · Paper page
- Nordal, 2001, A comparative study of teleconsultations versus face-to-face consultations, J Telemed Telecare, 7, 257, 10.1258/1357633011936507 · DOI · Paper page
Cited by
Teledermatology for diagnosing skin cancer in adults.
Chuchu, N., Dinnes, J., Takwoingi, Y., Matin, R. N., Bayliss, S. E., Davenport, C., Moreau, J. F., Bassett, O., Godfrey, K., O'Sullivan, C., et al. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013193 · Cited by 175
Practice Guidelines for Teledermatology
McKoy, K., Antoniotti, N. M., Armstrong, A., Bashshur, R., Bernard, J., Bernstein, D., Burdick, A., Edison, K., Goldyne, M., Kovarik, C., et al. (2016) · Telemedicine and e-Health · doi:10.1089/tmj.2016.0137 · Cited by 99
The Empirical Foundations of Teledermatology: A Review of the Research Evidence
Bashshur, R. L., Shannon, G. W., Tejasvi, T., Kvedar, J. C., & Gates, M. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2015.0146 · Cited by 90
Ferrándiz, L., Ojeda-Vila, T., Corrales, A., Martín-Gutiérrez, F. J., Ruíz-de-Casas, A., Galdeano, R., Álvarez-Torralba, I., Sánchez-Ibáñez, F., Domínguez-Toro, J. M., Encina, F., et al. (2017) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2016.10.041 · Cited by 75
Diagnostic reliability in teledermatology: a systematic review and a meta-analysis
Bourkas, A. N., Barone, N., Bourkas, M. E. C., Mannarino, M., Fraser, R. D. J., Lorincz, A., Wang, S. C., & Ramirez-GarciaLuna, J. L. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-068207 · Cited by 51
Metadata
- Authors
- Romero Aguilera, G., Cortina de la Calle, P., Vera Iglesias, E., Sánchez Caminero, P., García Arpa, M., & Garrido Martín, J. A.
- Venue
- Actas Dermo-Sifiliográficas (English Edition)
- Publisher
- Elsevier BV
- Published
- 2014-07
- Volume
- 105
- Issue
- 6
- Pages
- 605-613
- ISSN
- 0001-7310 (ISSN-L)
- 1578-2190
- OpenAlex
- W2126526970
- Cited by
- 19
- References
- 18
- FWCI
- 1.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor, 2256 d)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.