The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Rule-based and machine learning algorithms identify patients with systemic sclerosis accurately in the electronic health record
Open accessAbstract
BACKGROUND: Systemic sclerosis (SSc) is a rare disease with studies limited by small sample sizes. Electronic health records (EHRs) represent a powerful tool to study patients with rare diseases such as SSc, but validated methods are needed. We developed and validated EHR-based algorithms that incorporate billing codes and clinical data to identify SSc patients in the EHR. METHODS: We used a de-identified EHR with over 3 million subjects and identified 1899 potential SSc subjects with at least 1 count of the SSc ICD-9 (710.1) or ICD-10-CM (M34*) codes. We randomly selected 200 as a training set for chart review. A subject was a case if diagnosed with SSc by a rheumatologist, dermatologist, or pulmonologist. We selected the following algorithm components based on clinical knowledge and available data: SSc ICD-9 and ICD-10-CM codes, positive antinuclear antibody (ANA) (titer ≥ 1:80), and a keyword of Raynaud's phenomenon (RP). We performed both rule-based and machine learning techniques for algorithm development. Positive predictive values (PPVs), sensitivities, and F-scores (which account for PPVs and sensitivities) were calculated for the algorithms. RESULTS: PPVs were low for algorithms using only 1 count of the SSc ICD-9 code. As code counts increased, the PPVs increased. PPVs were higher for algorithms using ICD-10-CM codes versus the ICD-9 code. Adding a positive ANA and RP keyword increased the PPVs of algorithms only using ICD billing codes. Algorithms using ≥ 3 or ≥ 4 counts of the SSc ICD-9 or ICD-10-CM codes and ANA positivity had the highest PPV at 100% but a low sensitivity at 50%. The algorithm with the highest F-score of 91% was ≥ 4 counts of the ICD-9 or ICD-10-CM codes with an internally validated PPV of 90%. A machine learning method using random forests yielded an algorithm with a PPV of 84%, sensitivity of 92%, and F-score of 88%. The most important feature was RP keyword. CONCLUSIONS: Algorithms using only ICD-9 codes did not perform well to identify SSc patients. The highest performing algorithms incorporated clinical data with billing codes. EHR-based algorithms can identify SSc patients across a healthcare system, enabling researchers to examine important outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Jamian et al (2019) utilized EHRs to identify SSc, finding that the best algorithms combined clinical and billing codes, with top features including positive antinuclear antibody titers and presence of Raynaud’s phenomenon keyword.”
From Artificial intelligence–enabled precision medicine for inflammatory skin diseases (2026) · cites this paper as Jamian et al, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
Aletaha, D., Neogi, T., Silman, A. J., Funovits, J., Felson, D. T., Bingham, C. O., Birnbaum, N. S., Burmester, G. R., Bykerk, V. P., Cohen, M. D., et al. (2010) · Arthritis & Rheumatism · doi:10.1002/art.27584 · Cited by 9,631
van den Hoogen, F. H. J., Khanna, D., Fransen, J., Johnson, S. R., Baron, M. G., Tyndall, A., Matucci‐Cerinic, M., Naden, R. P., Medsger, T. A., Carreira, P. E., et al. (2013) · Arthritis & Rheumatism · doi:10.1002/art.38098 · Cited by 3,399
Power and sample size calculations
Dupont, W. D., & Plummer, W. D. (1990) · Controlled Clinical Trials · doi:10.1016/0197-2456(90)90005-m · Cited by 2,112
Development of a Large-Scale De-Identified DNA Biobank to Enable Personalized Medicine
Roden, D. M., Pulley, J. M., Basford, M., Bernard, G. R., Clayton, E. W., Balser, J., & Masys, D. R. (2008) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2008.89 · Cited by 966
Walker, U. A., Tyndall, A., Czirják, L., Denton, C. P., Farge, D., Kowal‐Bielecka, O., Müller‐Ladner, U., Bocelli‐Tyndall, C., & Matucci‐Cerinic, M. (2007) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2006.062901 · Cited by 927
Portability of an algorithm to identify rheumatoid arthritis in electronic health records
Carroll, R. J., Thompson, W. K., Eyler, A. E., Mandelin, A. M., Cai, T., Zink, R., Pacheco, J. A., Boomershine, C. S., Lasko, T. A., Xu, H., et al. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000583 · Cited by 235
Dumitrescu, L. C., Ritchie, M. D., Brown-Gentry, K. D., Pulley, J. M., Basford, M., Denny, J. C., Oksenberg, J. R., Roden, D. M., Haines, J. L., & Crawford, D. C. (2010) · Genetics in Medicine · doi:10.1097/gim.0b013e3181efe2df · Cited by 106
Moores, K. G., & Sathe, N. A. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.06.104 · Cited by 98
Cited by
Sorted by citations
Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations
Chan, S. Y., Reddy, V. D., Myers, B. A., Thibodeaux, Q. G., Brownstone, N. D., & Liao, W. J. (2020) · Dermatology and Therapy · doi:10.1007/s13555-020-00372-0 · Cited by 285
Maarseveen, T. D., Meinderink, T., Reinders, M. J. T., Knitza, J., Huizinga, T. W. J., Kleyer, A., Simón, D., van den Akker, E. B., & Knevel, R. (2020) · JMIR Medical Informatics · doi:10.2196/23930 · Cited by 53
Sammani, A., Bagheri, A., van der Heijden, P. G. M., Riele, A. S. J. M. T., Baas, A. F., Oosters, C. A. J., Oberski, D. L., & Asselbergs, F. W. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00404-9 · Cited by 41
Machine Learning in Rheumatic Diseases
Jiang, M., Li, Y., Jiang, C., Zhao, L., Zhang, X., & Lipsky, P. E. (2020) · Clinical Reviews in Allergy & Immunology · doi:10.1007/s12016-020-08805-6 · Cited by 40
Machine Learning Analysis of RNA-seq Data for Diagnostic and Prognostic Prediction of Colon Cancer
Bostancı, E., Koçak, E., Ünal, M., Güzel, M. S., Açıcı, K., & Aşuroğlu, T. (2023) · Sensors · doi:10.3390/s23063080 · Cited by 39
Yen, E. Y., Singh, D. R., & Singh, R. R. (2020) · Arthritis Care & Research · doi:10.1002/acr.24411 · Cited by 34
Enhancing medical coding efficiency through domain-specific fine-tuned large language models
Hou, Z., Liu, H., Bian, J., He, X., & Zhuang, Y. (2025) · npj Health Systems · doi:10.1038/s44401-025-00018-3 · Cited by 38
Yuan, K. Z., Yoon, C. H., Gu, Q., Munby, H., Walker, A. S., Zhu, T., & Eyre, D. W. (2025) · Communications Medicine · doi:10.1038/s43856-025-00790-1 · Cited by 36
Madrid-García, A., Barbancho, B. M., Rodríguez‐González, A., Fernández‐Gutiérrez, B., Rodríguez‐Rodríguez, L., & Menasalvas, E. (2023) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2023.152213 · Cited by 29
Electronic Medical Record–Based Case Phenotyping for the Charlson Conditions: Scoping Review
Lee, S. “., Doktorchik, C. T. A., Martin, E. A., D’Souza, A. G., Eastwood, C. A., Shaheen, A. A., Naugler, C. T., Lee, J., & Quan, H. (2021) · JMIR Medical Informatics · doi:10.2196/23934 · Cited by 24
Hammam, N., Li, J., Evans, M., Kay, J. L., Izadi, Z., Anastasiou, C., Gianfrancesco, M. A., Yazdany, J., & Schmajuk, G. (2021) · Arthritis Research & Therapy · doi:10.1186/s13075-021-02615-7 · Cited by 26
Maarseveen, T. D., Maurits, M. P., Niemantsverdriet, E., Mil, A. H. M. V. D. H., Huizinga, T. W. J., & Knevel, R. (2021) · Arthritis Research & Therapy · doi:10.1186/s13075-021-02553-4 · Cited by 25
Murdaca, G., Caprioli, S., Tonacci, A., Billeci, L., Greco, M., Negrini, S., Cittadini, G., Zentilin, P., Spagnolo, E. V., & Gangemi, S. (2021) · Diagnostics · doi:10.3390/diagnostics11101880 · Cited by 24
Hammam, N., Evans, M., Morgan, E. M., Reimold, A. M., Anastasiou, C., Kay, J. L., Yazdany, J., & Schmajuk, G. (2020) · Arthritis Care & Research · doi:10.1002/acr.24496 · Cited by 20
Alotaibi, A. S. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-05921-z · Cited by 20
van Leeuwen, J. R., Penne, E. L., Rabelink, T. J., Knevel, R., & Teng, Y. K. O. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107757 · Cited by 20
Sammani, A., Jansen, M., de Vries, N. M., de Jonge, N., Baas, A. F., Riele, A. S. J. M. T., Asselbergs, F. W., & Oerlemans, M. I. F. J. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.768847 · Cited by 19
Driving maneuver classification from time series data: a rule based machine learning approach
Haque, M. M., Sarker, S., & Dewan, M. A. A. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03328-3 · Cited by 19
Natural Language Processing and Text Mining (Turning Unstructured Data into Structured)
Bagheri, A., Giachanou, A., Mosteiro, P. J., & Verberne, S. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_5 · Cited by 13
Mills, G. A., Dey, D. I., Kassim, M., Yiwere, A., & Broni, K. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020065 · Cited by 11
Benavent, D., Muñoz‐Fernández, S., de la Morena, I., Fernández‐Nebro, A., Marín‐Corral, J., Rosa, E. C., Taberna, M., Sanabra, C., & Sastré, C. (2023) · Therapeutic Advances in Musculoskeletal Disease · doi:10.1177/1759720x231220818 · Cited by 10
Tukpah, A.-M. C., Rose, J. A., Seger, D. L., Dellaripa, P. F., Hunninghake, G. M., & Bates, D. W. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283775 · Cited by 8
Cheng, D., Bensellam, N., Sanchez, K., Zhang, A. D., Biba, U., Ershadi, S., Gregoire, S., Zangenah, N., Acevedo-Fontanez, L. A., Fladger, A., et al. (2026) · JAMA Dermatology · doi:10.1001/jamadermatol.2025.5268 · Cited by 8
Kose, A. H. P., & Mangaonkar, K. V. (2023) · Combinatorial Chemistry & High Throughput Screening · doi:10.2174/1386207326666230306114626 · Cited by 8
Ghasemi, P., & Lee, J. (2024) · JMIR Medical Informatics · doi:10.2196/52896 · Cited by 10
Fayaz, S., Karuppasamy, S., & Chinnaiyan, P. (2025) · Informatics · doi:10.3390/informatics12030064 · Cited by 8
Artificial intelligence–enabled precision medicine for inflammatory skin diseases
Tang, A. S., Wei, M. L., Haemel, A., La, C., Sirota, M., & Lee, E. Y. (2026) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2025.10.596 · Cited by 10
Comparison of Diagnosis Codes to Clinical Notes in Classifying Patients with Diabetic Retinopathy
Yonamine, S., Ma, C. J., Alabi, R. O., Kaidonis, G., Chan, L., Borkar, D. S., Stein, J. D., Arnold, B. F., & Sun, C. Q. (2024) · Ophthalmology Science · doi:10.1016/j.xops.2024.100564 · Cited by 6
Tukpah, A.-M. C., Rose, J. A., Seger, D. L., Dellaripa, P. F., Hunninghake, G. M., & Bates, D. W. (2024) · Lara D. Veeken · doi:10.1093/rheumatology/keae430 · Cited by 6
Distinct Cancer Risk Profiles in Patients With Systemic Sclerosis With Autoantibody Stratification
Mahajan, A., Vazquez-Machado, M. C., Zangenah, N., Sparks, J. A., & LaChance, A. (2025) · Arthritis & Rheumatology · doi:10.1002/art.43339 · Cited by 6
Peterson, H., Vela, K., Barnado, A., & Patrick, A. E. (2023) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2023.152167 · Cited by 6
Revel‐Vilk, S., Shalev, V., Gill, A., Paltiel, O., Manor, O., Tenenbaum, A., Azani, L., & Chodick, G. (2024) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-024-03042-y · Cited by 5
Yuan, K. Z., Yoon, C. H., Gu, Q., Munby, H., Walker, A. S., Zhu, T., & Eyre, D. W. (2024) · medRxiv · doi:10.1101/2024.04.17.24305966 · Cited by 5
Identification of Systemic Sclerosis through Machine Learning Algorithms and Gene Expression
Alfonso, G., & Castillo, R. (2022) · Mathematics · doi:10.3390/math10244632 · Cited by 4
Santema, J. J. M. M., van Wijngaarden, J. D. H., Hiddink, E., Deken, F., Kleinsmann, M. S., & van Os, H. J. A. (2026) · Journal of Medical Internet Research · doi:10.2196/83084 · Cited by 3
Text Mining in Healthcare: Bringing Structure to Electronic Health Records
Bagheri, A. (2021) · Utrecht University Repository (Utrecht University) · doi:10.33540/531 · Cited by 3
Ghasemi, P., & Lee, J. (2023) · doi:10.2196/preprints.52896 · Cited by 3
Singh, D. R., & Gupta, N. (2025) · International Conference on Computing Technologies & Data Communication (ICCTDC) · doi:10.1109/icctdc64446.2025.11158136 · Cited by 2
Cohen, A. M., Kaner, J., Miller, R., Kopesky, J. W., & Hersh, W. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad244 · Cited by 2
Ozen, G., O’Rorke, M., Romitti, P. A., & Domsic, R. T. (2026) · ACR Open Rheumatology · doi:10.1002/acr2.90091 · Cited by 1
Applications of artificial intelligence in rare skin diseases and skin cancers: A scoping review
Alkhateeb, M., Nayeem, A., Alokla, M., Hajj, N. E., Alam, T., Ahmed, A., Belhouari, S. B., Sheikh, J. I., & Abd-Alrazaq, A. A. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100408 · Cited by 1
Jeong, E., Mogos, M. F., & Chen, Y. (2025) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14918 · Cited by 1
Electronic Health Record Data in Cardiovascular Research
Sammani, A. Z. (2022) · Utrecht University Repository (Utrecht University) · doi:10.33540/1169 · Cited by 1
Maarseveen, T. D., Maurits, M. P., Niemantsverdriet, E., van Mil, A. V. D. H. –., Huizinga, T. W. J., & Knevel, R. (2021) · Research Square · doi:10.21203/rs.3.rs-394821/v1
Modelling Baby’s Health Management System Using Rule-Based Algorithm
Ismail, R., Wahab, F. A., Awang, W. S. W., Azizan, A., & Rusli, I. S. B. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2319/1/012026
Data-driven approaches in rheumatology: learning from real-world data
Madrid-García, A. (2022) · doi:10.20868/upm.thesis.72300
Luo, Y., Ross, L., Zheng, J., & Bernstein, E. J. (2023) · Journal of Scleroderma and Related Disorders · doi:10.1177/23971983231197268
Alfonso, G., & Alfonso Pérez, G. (2023) · Tesis Doctorals en Xarxa (Consorci de Serveis Universitaris de Catalunya) · doi:10.6035/14122.2023.851383
Rituximab versus mycophenolate mofetil for cancer risk in systemic sclerosis
Mahajan, A., Piette, E. W., Sparks, J. A., & LaChance, A. (2025) · Lara D. Veeken · doi:10.1093/rheumatology/keaf396
Avanços no diagnóstico precoce da artrite reumatoide
de Oliveira, R. D. R. (2025) · Revista Paulista de Reumatologia · doi:10.46833/reumatologiasp.2025.24.1.12-18
Schreyer, W. M., Melson, R. A., Anderson, C., Madison, C. J., Katsoulakis, E., & Thompson, R. F. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00088
Hammam, N., Li, J., Evans, M., Kay, J. L., Izadi, Z., Anastasiou, C., Gianfrancesco, M. A., Yazdany, J., & Schmajuk, G. (2021) · Figshare · doi:10.6084/m9.figshare.16544281
Maarseveen, T. D., Maurits, M. P., Niemantsverdriet, E., van der Helm-van Mil, A. H. M., Huizinga, T. W. J., & Knevel, R. (2021) · Figshare · doi:10.6084/m9.figshare.14826230
Maarseveen, T. D., Maurits, M. P., Niemantsverdriet, E., van der Helm-van Mil, A. H. M., Huizinga, T. W. J., & Knevel, R. (2021) · Figshare · doi:10.6084/m9.figshare.14826221
Saka‐Herrán, C., Kumar, T. M. P., Keane, D., Albert, P., Clyne, B., McCarthy, C., Tynan, G., Dunne, N., Flood, M., McCarthy, E. M., et al. (2026) · medRxiv · doi:10.64898/2026.09.23.26363795
Metadata
- Authors
- Jamian, L., Wheless, L., Crofford, L. J., & Barnado, A.
- Affiliations
- Vanderbilt University Medical Center (US)
- Corresponding author
- Barnado, April
- Venue
- Arthritis Research & Therapy
- Published
- 2019-12
- Volume
- 21
- Issue
- 1
- Pages
- 305-305
- ISSN
- 1478-6354 (ISSN-L)
- 1478-6362
- PMC
- PMC6937803
- OpenAlex
- W2997101676
- Cited by
- 57
- References
- 14
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 7 · 2023: 13 · 2022: 6 · 2021: 11 · 2020: 5
- FWCI
- 4.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Arthritis Research & Therapy (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute of Child Health and Human Development (NICHD 5K12HD043483-12)
- National Institute of Arthritis and Musculoskeletal and Skin Diseases (1 K08AR072757-01)
- National Center for Research Resources (UL1 RR024975)
- National Center for Advancing Translational Sciences (ULTR000445)
- Topics
- Health Sciences › Medicine › Rheumatology › Systemic Sclerosis and Related Diseases
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- systemic sclerosis, electronic health records, patient identification algorithm, ICD-10-CM codes, ICD-9 codes, billing codes, antinuclear antibodies, Raynaud's phenomenon, random forest, rule-based algorithm, rare diseases, positive predictive value, sensitivity and specificity, chart review validation, clinical data integration, health information exchange
- MeSH terms
- Machine Learning, Adult, Aged, Aged, 80 and over, Algorithms, Female, Humans, Male, Middle Aged, Scleroderma, Systemic, Sensitivity and Specificity, Reproducibility of Results, Databases, Factual, International Classification of Diseases, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
