Development of a Natural Language Processing Model for Extracting Kidney Biopsy Pathology Diagnoses
Open accessAbstract
Rationale & Objective: Kidney biopsy reports are in a nonindexed text format, and the diagnosis requires labor-intensive manual abstraction. Natural language processing (NLP) has not been rigorously tested for kidney biopsy diagnosis extraction. Our objective was to develop an accurate model to extract the biopsy diagnosis from free-text reports. Study Design: Text classification using NLP. Setting & Participants: 2,666 patients with 3,042 native kidney biopsy reports in the Portable Document Format, from June 2016 to December 2023. Predictor: Kidney biopsy diagnosis. Outcomes: The performance of the NLP algorithm for all and the 20 most common diagnoses based on precision, recall, F1 score, and area under the receiver operating curve (AUROC). Analytical Approach: A domain expert manually abstracted the diagnosis, and a renal pathologist validated a random subset (n = 200). Structured Query Language server and Python processed reports into machine-readable free text. We used PubMed Bidirectional Encoder Representations from Transformers to develop our NLP algorithm. We randomly split the reports into training (80%; n = 2,434) and testing (20%; n = 608) sets to train the NLP system. We further divided the testing set into 20% validation and 80% fine-tuning sets. Results: The median age was 57 years, with 50% female, 29% African Americans, and 23% Hispanic participants. The 5 most frequent glomerular diagnoses were diabetic kidney disease (23.7%), focal segmental glomerulosclerosis (15.5%), lupus nephritis (9.7%), immunoglobulin A nephropathy (8.9), and membranous nephropathy (7.2%). The Cohen kappa coefficient for interrater reliability was 0.76. PubMed Bidirectional Encoder Representations from Transformers fine-tuned with a training set showed the average AUROC for NLP performance in the testing set of 0.95 across all diagnoses with an F1 score of 0.57. For the 20 most common diagnoses, the AUROC was 0.97 with an F1 score of 0.72. Limitations: Single centered; sample size and use limited to research purposes. Conclusions: We demonstrate an accurate and scalable NLP system to extract the primary diagnosis from free-text kidney biopsy reports, which can facilitate epidemiologic studies and identify patients for clinical trial recruitment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The BERT-based model demonstrated the scalability and accuracy of LLMs for pathological report structuring and clinical data mining.
“In renal function research, Bobart et al. [136] developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports, achieving an average AUC of 0.95 across reports for DN, lupus nephritis, and other diseases, reducing manual annotation workload and demonstrating the scalability and accuracy of LLMs for pathological report”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [136]
The Methods describe a PubMed BERT model that extracts diagnoses from kidney biopsy reports, and the Results report a mean AUROC of 0.95 across diagnoses. The Conclusions call the system accurate and scalable, but the Discussion describes large-scale use and testing in external cohorts as future work. The paper supports this specific model’s potential for clinical data extraction, not a demonstrated result for LLMs generally; it also does not measure a reduction in manual annotation workload. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe BERT-based model reduced manual annotation workload.
“In renal function research, Bobart et al. [136] developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports, achieving an average AUC of 0.95 across reports for DN, lupus nephritis, and other diseases, reducing manual annotation workload and demonstrating the scalability and accuracy of LLMs for pathological report”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [136]
The Rationale & Objective describes manual diagnosis abstraction as labor-intensive, and the Discussion says the model accurately extracts diagnoses from free-text reports. That supports its potential to replace some manual extraction, but the Methods > Annotation says all study reports were manually annotated; the paper does not report a measured reduction in annotation workload. Page numbers were not available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredBobart et al. developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports.
“In renal function research, Bobart et al. [136] developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports, achieving an average AUC of 0.95 across reports for DN, lupus nephritis, and other diseases, reducing manual annotation workload and demonstrating the scalability and accuracy of LLMs for pathological report”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [136]
The BERT-based model achieved an average AUC of 0.95 across reports for diabetic nephropathy, lupus nephritis, and other diseases.
“In renal function research, Bobart et al. [136] developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports, achieving an average AUC of 0.95 across reports for DN, lupus nephritis, and other diseases, reducing manual annotation workload and demonstrating the scalability and accuracy of LLMs for pathological report”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [136]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741 · see citing works
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · see citing works
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
Tuttle, K. R., Alicic, R. Z., Duru, O. K., Jones, C. R., Daratha, K. B., Nicholas, S. B., McPherson, S. M., Neumiller, J. J., Bell, D. S., Mangione, C. M., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.18169 · Cited by 275 · see citing works
Sethi, S. M., Vriese, A. S. D., & Fervenza, F. C. (2022) · The Lancet · doi:10.1016/s0140-6736(22)00461-5 · Cited by 100 · see citing works
Wetmore, J. B., Guo, H., Liu, J., Collins, A. J., & Gilbertson, D. T. (2016) · Kidney International · doi:10.1016/j.kint.2016.04.026 · Cited by 87 · see citing works
Kim, Y., Lee, J. H., Choi, S., Lee, J. M., Kim, J.-H., Seok, J., & Joo, H. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-77258-w · Cited by 51 · see citing works
Gorenstein, L., Konen, E., Green, M., & Klang, E. (2024) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2024.01.012 · Cited by 50 · see citing works
on behalf of the CURE-CKD investigators, Norris, K. C., Duru, O. K., Alicic, R. Z., Daratha, K. B., Nicholas, S. B., McPherson, S., Bell, D. S., Shen, J. I., Jones, C. R., et al. (2019) · BMC Nephrology · doi:10.1186/s12882-019-1558-9 · Cited by 29 · see citing works
Self-citationNatural Language Processing in Diagnostic Texts from Nephropathology
Legnar, M., Daumke, P., Hesser, J., Porubský, Š., Popovic, Z., Bindzus, J. N., Siemoneit, J.-H. H., & Weis, C. (2022) · Diagnostics · doi:10.3390/diagnostics12071726 · Cited by 19 · see citing works
The Cleveland Clinic Kidney Biopsy Epidemiological Project
Bobart, S. A., Portalatin, G. M., Sawaf, H., Shettigar, S. K., Carrion-Rodriguez, A., Liang, H., Herlitz, L., & Gebreselassie, S. K. (2022) · Kidney360 · doi:10.34067/kid.0005882022 · Cited by 18 · see citing works
Self-citationLam, H., Nguyen, F. T., Wang, X., Stock, A., Lenskaya, V., Kooshesh, M., Li, P., Qazi, M., Wang, S., Dehghan, M., et al. (2022) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2022.100154 · Cited by 17 · see citing works
Jupin-Delevaux, É., Djahnine, A., Talbot, F., Richard, A., Gouttard, S., Mansuy, A., Douek, P., Si-Mohamed, S. A., & Boussel, L. (2023) · Research in Diagnostic and Interventional Imaging · doi:10.1016/j.redii.2023.100027 · Cited by 13 · see citing works
Barr, B., Harasemiw, O., Gibson, I. W., Tremblay-Savard, O., & Tangri, N. (2023) · Canadian Journal of Kidney Health and Disease · doi:10.1177/20543581231178963 · Cited by 5 · see citing works
Ketogenic-Diet Shake Containing Uncaria tomentosa-Associated Acute Interstitial Nephritis
Portalatin, G. M., Shettigar, S. K., Carrion-Rodriguez, A., Medikayala, S., Herlitz, L., Sandy, D., Gebreselassie, S. K., & Bobart, S. A. (2022) · Case Reports in Nephrology and Dialysis · doi:10.1159/000526391 · Cited by 2 · see citing works
Self-citationDomain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Gu, Y., Tinn, R., Cheng, H., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.15779
Flagged first, then by citations
Research works printed but not linked: 1
- [13]Brown T.B. Mann B. Ryder N. Language models are few-shot learners Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20 2020 Curran Associates Inc. 1877 1901
Cited by
Sorted by citations
Xie, X., Wu, C., Huang, Z., Zhou, Y., Huang, J., Dao, F., Yan, D., Deng, K., Lyu, H., Ma, C., et al. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag083 · Cited by 4 · see citing works
“In renal function research, Bobart et al. [136] developed a biomedical BERT-based natural language processing model to automatically extract diagnostic information from kidney biopsy reports, achieving an average AUC of 0.95 across reports for DN, lupus nephritis, and other diseases, reducing…”
InfluentialNatural Language Processing in Healthcare: From Unstructured Data to Clinical Intelligence
Kumar, D., Choudhary, N., Gupta, V., Tanwar, R., Bahmani, K., Antal, S., Jarora, S., Gupta, S., Shaikh, M. S., Webster, T. J., et al. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200698 · Cited by 2 · see citing works
Large-Language Models for data extraction from written kidney biopsy reportsPreprint
Niggemeier, L., Hölscher, D. L., Herkens, T. C., Gilles, P., Böör, P., & Bülow, R. D. (2026) · medRxiv · doi:10.64898/2026.02.23.26346945 · Cited by 1 · see citing works
Explainable NLP Approach for Detection of Comorbidities in Chronic Kidney Disease Patients
Arif, H., Raza, A., Kumar, A., Ali, A. S., & Shah, B. K. (2025) · International Conference on Innovative Data Communication Technologies and Application (ICIDCA) · doi:10.1109/icidca66325.2025.11280449
Machine Learning as a Multifaceted Tool in Glomerular Diseases: Beyond Risk Prediction
Vargas-Brochero, M. J., Schlesinger, R. P., Cheungpasitporn, W., Thongprayoon, C., & Miao, J. (2026) · Glomerular Diseases · doi:10.1159/gdz/acyag004
- Authors
- Bobart, S. A., Hsu, E., Potter, T., Truong, L., Waterman, A., Jones, S., & Shafi, T.
- Affiliations
- Houston Methodist (US)
- Baylor Scott & White Medical Center - Temple (US)
- Corresponding author
- Bobart, Shane A.
- Venue
- Kidney Medicine
- Publisher
- Elsevier BV
- Published
- 2025-08
- Volume
- 7
- Issue
- 8
- Pages
- 101047
- ISSN
- 2590-0595 (ISSN-L)
- PMC
- PMC12311501
- OpenAlex
- W4411306376
- Cited by
- 5
- References
- 16
- Citations by year
- 2026: 4 · 2025: 1
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Kidney Medicine (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Funders
- NIH (1OT2OD032581)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- natural language processing, text classification, BERT, clinical data abstraction, diabetic nephropathy, focal segmental glomerulosclerosis, lupus nephritis, immunoglobulin A nephropathy, membranous nephropathy, glomerular disease, clinical trial recruitment, inter-rater reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
