CoChat
Sign Up
PreprintCited by 21Machine Learning in HealthcareTopic ModelingBiomedical Text Mining and Ontologies

Self-Verification Improves Few-Shot Clinical Information Extraction

Open access
Gero, Zelalem, Singh, Chandan, Cheng, Hao, Naumann, Tristan, Galley, Michel, Gao, Jianfeng, Poon, Hoifung·2023·arXiv
DOI 10.48550/arxiv.2306.00024arXiv 2306.00024
Open this paper in CoChat — free accountOpen PDF

Abstract

Extracting patient information from unstructured text is a critical task in health decision-support and clinical research. Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning, in contrast to supervised learning which requires much more costly human annotations. However, despite drastic advances in modern LLMs such as GPT-4, they still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs. This is made possible by the asymmetry between verification and generation, where the latter is often much easier than the former. Experimental results show that our method consistently improves accuracy for various LLMs in standard clinical information extraction tasks. Additionally, self-verification yields interpretations in the form of a short text span corresponding to each output, which makes it very efficient for human experts to audit the results, paving the way towards trustworthy extraction of clinical information in resource-constrained scenarios. To facilitate future research in this direction, we release our code and prompts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.00024

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 37

  1. Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., and Sontag, D. Large language models are few-shot clinical information extractors. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 1998-2022, 2022.
  2. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.
  3. Dao, T., Fu, D., Ermon, S., Rudra, A., and R´ e, C. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems , 35:1634416359, 2022.
  4. Edin, J., Junge, A., Havtorn, J. D., Borgholt, L., Maistro, M., Ruotsalo, T., and Maaløe, L. Automated medical coding on mimic-iii and mimic-iv: A critical review and replicability study. arXiv preprint arXiv:2304.10909 , 2023. · arXiv
  5. Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V. Y., Lao, N., Lee, H., Juan, D.-C., et al. Rarr: Researching and revising what language models say, using language models. arXiv preprint arXiv:2210.08726 , 2022. · arXiv
  6. Guti´ errez, B. J., McNeal, N., Washington, C., Chen, Y., Li, L., Sun, H., and Su, Y. Thinking about gpt-3 in-context learning for biomedical ie? think again. arXiv preprint arXiv:2203.08410 , 2022. · arXiv
  7. Johnson, A. E., Pollard, T. J., Shen, L., Lehman, L.-w. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., and Mark, R. G. Mimic-iii, a freely accessible critical care database. Scientific data , 3(1):1-9, 2016.
  8. Kornblith, A. E., Singh, C., Devlin, G., Addo, N., Streck, C. J., Holmes, J. F., Kuppermann, N., Grupp-Phelan, J., Fineman, J., Butte, A. J., et al. Predictability and stability testing to assess clinical decision instrument performance for children after blunt torso trauma. PLOS Digital Health , 1(8):e0000076, 2022.
  9. Laursen, M., Pedersen, J., Hansen, R., Savarimuthu, T. R., and Vinholt, P. Danish clinical named entity recognition and relation extraction. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa) , pp. 655-666, 2023.
  10. Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., and Wright, A. Assessing the value of chatgpt for clinical decision support optimization. MedRxiv , pp. 2023-02, 2023.
  11. Lundberg, S. M. and Lee, S.-I. A unified approach to interpreting model predictions. Advances in neural information processing systems , 30, 2017.
  12. Ma, Y., Cao, Y., Hong, Y., and Sun, A. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! arXiv preprint arXiv:2303.08559 , 2023. · arXiv
  13. MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., and Huang, Z. Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education ResearchVolume 2 , pp. 37-39, 2022.
  14. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 , 2023. · arXiv
  15. Nye, B., Li, J. J., Patel, R., Yang, Y ., Marshall, I. J., Nenkova, A., and Wallace, B. C. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In Proceedings of the conference. Association for Computational Linguistics. Meeting , volume 2018, pp. 197. NIH Public Access, 2018.
  16. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 , 2022. · arXiv
  17. Patil, S. G., Zhang, T., Wang, X., and Gonzalez, J. E. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334 , 2023. · arXiv
  18. Peng, B., Galley, M., He, P., Cheng, H., Xie, Y ., Hu, Y ., Huang, Q., Liden, L., Yu, Z., Chen, W., et al. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813 , 2023. · arXiv
  19. Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. Measuring and narrowing the compositionality gap in language models, 2022.
  20. Rajani, N. F., McCann, B., Xiong, C., and Socher, R. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361 , 2019. · arXiv
  21. Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., and Reitter, D. Measuring attribution in natural language generation models. arXiv preprint arXiv:2112.12870 , 2021. · arXiv
  22. Ribeiro, M. T., Singh, S., and Guestrin, C. ' why should i trust you?' explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pp. 1135-1144, 2016.
  23. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence , 1(5):206-215, 2019.
  24. Scheurer, J., Campos, J. A., Korbak, T., Chan, J. S., Chen, A., Cho, K., and Perez, E. Training language models with language feedback at scale. arXiv preprint arXiv:2303.16755 , 2023. · arXiv
  25. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 , 2020. · arXiv
  26. Singh, C., Askari, A., Caruana, R., and Gao, J. Augmenting interpretable models with llms during training. arXiv preprint arXiv:2209.11799 , 2022a. · arXiv
  27. Singh, C., Morris, J. X., Aneja, J., Rush, A. M., and Gao, J. Explaining patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01848 , 2022b. · arXiv
  28. Singh, C., Hsu, A. R., Antonello, R., Jain, S., Huth, A. G., Yu, B., and Gao, J. Explaining black box text modules in natural language with language models. arXiv preprint arXiv:2305.09863 , 2023. · arXiv
  29. Toma, A., Lawler, P. R., Ba, J., Krishnan, R. G., Rubin, B. B., and Wang, B. Clinical camel: An open-source expert-level medical language model with dialogue-based knowledge encoding. arXiv preprint arXiv:2305.12031 , 2023. · arXiv
  30. Wang, B., Deng, X., and Sun, H. Iteratively prompt pre-trained language models for chain of thought. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 2714-2730, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022. emnlp-main.174 .
  31. Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y ., Mehrabi, S., Sohn, S., et al. Clinical information extraction applications: a literature review. Journal of biomedical informatics , 77:34-49, 2018.
  32. Wiegreffe, S. and Pinter, Y. Attention is not not explanation. arXiv preprint arXiv:1908.04626 , 2019. · arXiv
  33. Wu, C., Zhang, X., Zhang, Y., Wang, Y., and Xie, W. Pmc-llama: Further finetuning llama on medical papers. arXiv preprint arXiv:2304.14454 , 2023. · arXiv
  34. Wu, T., Terry, M., and Cai, C. J. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In CHI Conference on Human Factors in Computing Systems , pp. 1-22, 2022.
  35. Xu, B., Wang, Q., Mao, Z., Lyu, Y., She, Q., and Zhang, Y. k nn prompting: Beyond-context learning with calibration-free nearest neighbor inference. arXiv preprint arXiv:2303.13824 , 2023. · arXiv
  36. Zhang, H., Liu, X., and Zhang, J. Summit: Iterative text summarization via chatgpt. arXiv preprint arXiv:2305.14835 , 2023. · arXiv
  37. Zweigenbaum, P., Demner-Fushman, D., Yu, H., and Cohen, K. B. Frontiers of biomedical text mining: current progress. Briefings in bioinformatics , 8(5):358-375, 2007.

Cited by

Sorted by citations

Authors
Gero, Z., Singh, C., Cheng, H., Naumann, T., Galley, M., Gao, J., & Poon, H.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2306.00024
arXiv
2306.00024
OpenAlex
W4379251563
Cited by
21
References
0
Citations by year
2026: 2 · 2025: 11 · 2024: 8
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
Keywords
self-verification, clinical information extraction, few-shot learning, large language models, clinical text, output verification, interpretability, provenance, trustworthy AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review