Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 2

The DOI matches its DataCite record.

Unifying Corroborative and Contributive Attributions in Large Language Models

Worledge, Theodora, Shen, Judy Hanwen, Meister, Nicole, Winston, Caleb, Guestrin, Carlos·2023·arXiv
DOI 10.48550/arxiv.2311.12233arXiv 2311.12233
Open this paper in CoChat — free account

Abstract

As businesses, products, and services spring up around large language models, the trustworthiness of these models hinges on the verifiability of their outputs. However, methods for explaining language model outputs largely fall across two distinct fields of study which both use the term "attribution" to refer to entirely separate techniques: citation generation and training data attribution. In many modern applications, such as legal document generation and medical question answering, both types of attributions are important. In this work, we argue for and present a unified framework of large language model attributions. We show how existing methods of different types of attribution fall under the unified framework. We also use the framework to discuss real-world use cases where one or both types of attributions are required. We believe that this unified framework will guide the use case driven development of systems that leverage both types of attribution, as well as the standardization of their evaluation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2311.12233

Cites

Statistics

Figures

Figure 1 Overview of our proposed unified framework for large language model attributions. We include tasks that require both contributive and corroborative attributions and properties that apply to both types of attributions.
Figure 1 Overview of our proposed unified framework for large language model attributions. We include tasks that require both contributive and corroborative attributions and properties that apply to both types of attributions.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 52

  1. R. Azamfirei, S. R. Kudchadkar, and J. Fackler, "Large language models and the perils of their hallucinations," Critical Care, vol. 27, no. 1, pp. 1-2, 2023.
  2. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., "On the opportunities and risks of foundation models," arXiv preprint arXiv:2108.07258, 2021. · arXiv
  3. X. Yue, B. Wang, K. Zhang, Z. Chen, Y. Su, and H. Sun, "Automatic evaluation of attribution by large language models," arXiv preprint arXiv:2305.06311, 2023. · arXiv
  4. K. Guu, K. Lee, Z. Tung, P. Pasupat, and M. Chang, "Retrieval augmented language model pre-training," in International conference on machine learning. PMLR, 2020, pp. 3929-3938.
  5. B. Bohnet, V. Q. Tran, P. Verga, R. Aharoni, D. Andor, L. B. Soares, J. Eisenstein, K. Ganchev, J. Herzig, K. Hui et al., "Attributed question answering: Evaluation and modeling for attributed large language models," arXiv preprint arXiv:2212.08037, 2022. · arXiv
  6. H. Rashkin, V. Nikolaev, M. Lamm, L. Aroyo, M. Collins, D. Das, S. Petrov, G. S. Tomar, I. Turc, and D. Reitter, "Measuring attribution in natural language generation models," arXiv preprint arXiv:2112.12870, 2021. · arXiv
  7. P. W. Koh and P. Liang, "Understanding black-box predictions via influence functions," in International conference on machine learning. PMLR, 2017, pp. 1885-1894.
  8. A. Ilyas, S. M. Park, L. Engstrom, G. Leclerc, and A. Madry, "Datamodels: Predicting predictions from training data," arXiv preprint arXiv:2202.00622, 2022. · arXiv
  9. G. Pruthi, F. Liu, S. Kale, and M. Sundararajan, "Estimating training data influence by tracing gradient descent," Advances in Neural Information Processing Systems, vol. 33, pp. 19 920-19 930, 2020.
  10. R. Grosse, J. Bae, C. Anil, N. Elhage, A. Tamkin, A. Tajdini, B. Steiner, D. Li, E. Durmus, E. Perez et al., "Studying large language model generalization with influence functions," arXiv preprint arXiv:2308.03296, 2023. · arXiv
  11. E. Akyürek, T. Bolukbasi, F. Liu, B. Xiong, I. Tenney, J. Andreas, and K. Guu, "Towards tracing knowledge in language models back to the training data," in Findings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 2429-2446.
  12. O. Honovich, R. Aharoni, J. Herzig, H. Taitelbaum, D. Kukliansy, V. Cohen, T. Scialom, I. Szpektor, A. Hassidim, and Y. Matias, "True: Re-evaluating factual consistency evaluation," arXiv preprint arXiv:2204.04991, 2022. · arXiv
  13. Z. Hammoudeh and D. Lowd, "Training data influence analysis and estimation: A survey," arXiv preprint arXiv:2212.04612, 2022. · arXiv
  14. A. Madsen, S. Reddy, and S. Chandar, "Post-hoc interpretability for neural nlp: A survey," ACM Computing Surveys, vol. 55, no. 8, pp. 1-42, 2022.
  15. J. Ramos et al., "Using tf-idf to determine word relevance in document queries," in Proceedings of the first instructional conference on machine learning, vol. 242, no. 1. Citeseer, 2003, pp. 29-48.
  16. A. Roberts, C. Raffel, and N. Shazeer, "How much knowledge can you pack into the parameters of a language model?" in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020.
  17. E. Cosijn and P. Ingwersen, "Dimensions of relevance," Information Processing & Management, vol. 36, no. 4, pp. 533-550, 2000.
  18. Z. Guo, M. Schlichtkrull, and A. Vlachos, "A survey on automated fact-checking," Transactions of the Association for Computational Linguistics, vol. 10, pp. 178-206, 2022.
  19. X. Yue, X. Pan, W. Yao, D. Yu, D. Yu, and J. Chen, "C-more: Pretraining to answer open-domain questions by consulting millions of references," arXiv preprint arXiv:2203.08928, 2022. · arXiv
  20. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning-i," Information sciences, vol. 8, no. 3, pp. 199-249, 1975.
  21. A. Erasmus, T. D. Brunet, and E. Fisher, "What is interpretability?" Philosophy & Technology, vol. 34, no. 4, pp. 833-862, 2021.
  22. C.-K. Yeh, A. Taly, M. Sundararajan, F. Liu, and P. Ravikumar, "First is better than last for language data influence," Advances in Neural Information Processing Systems, vol. 35, pp. 32 285-32 298, 2022.
  23. S. Basu, P. Pope, and S. Feizi, "Influence functions in deep learning are fragile," arXiv preprint arXiv:2006.14651, 2020. · arXiv
  24. A. Søgaard et al., "Revisiting methods for finding influential examples," arXiv preprint arXiv:2111.04683, 2021. · arXiv
  25. E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, "On the dangers of stochastic parrots: Can language models be too big?" in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 2021, pp. 610-623.
  26. P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar et al., "Holistic evaluation of language models," arXiv preprint arXiv:2211.09110, 2022. · arXiv
  27. J. Ni, C. Qu, J. Lu, Z. Dai, G. Hernandez Abrego, J. Ma, V. Zhao, Y. Luan, K. Hall, M.-W. Chang, and Y. Yang, "Large dual encoders are generalizable retrievers," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022.
  28. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, "Dense passage retrieval for open-domain question answering," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, Nov. 2020.
  29. S. Robertson, H. Zaragoza et al., "The probabilistic relevance framework: Bm25 and beyond," Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333-389, 2009.
  30. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders et al., "Webgpt: Browser-assisted question-answering with human feedback," arXiv preprint arXiv:2112.09332, 2021. · arXiv
  31. A. Lazaridou, E. Gribovskaya, W. Stokowiec, and N. Grigorev, "Internet-augmented language models through few-shot prompting for open-domain question answering," arXiv preprint arXiv:2203.05115, 2022. · arXiv
  32. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., "Palm: Scaling language modeling with pathways," arXiv preprint arXiv:2204.02311, 2022. · arXiv
  33. A. Ghorbani and J. Zou, "Data shapley: Equitable valuation of data for machine learning," in International Conference on Machine Learning. PMLR, 2019, pp. 2242-2251.
  34. J. Bae, N. Ng, A. Lo, M. Ghassemi, and R. B. Grosse, "If influence functions are the answer, then what is the question?" Advances in Neural Information Processing Systems, vol. 35, pp. 17 953-17 967, 2022.
  35. A. Choudhury and H. Shamszare, "Investigating the impact of user trust on the adoption and use of chatgpt: Survey analysis," Journal of Medical Internet Research, vol. 25, p. e47184, 2023.
  36. N. M. Krause, I. Freiling, B. Beets, and D. Brossard, "Fact-checking as risk communication: the multi-layered risk of misinformation in times of covid-19," Journal of Risk Research, vol. 23, no. 7-8, pp. 1052-1059, 2020.
  37. M. A. Amazeen, "Revisiting the epistemology of fact-checking," Critical Review, vol. 27, no. 1, pp. 1-22, 2015.
  38. P. Samuelson, "Generative ai meets copyright," Science, vol. 381, no. 6654, pp. 158-161, 2023.
  39. J. Sun, Q. V. Liao, M. Muller, M. Agarwal, S. Houde, K. Talamadupula, and J. D. Weisz, "Investigating explainability of generative ai for code through scenario-based design," in 27th International Conference on Intelligent User Interfaces, 2022, pp. 212-228.
  40. Q. V. Liao, D. Gruen, and S. Miller, "Questioning the ai: informing design practices for explainable ai user experiences," in Proceedings of the 2020 CHI conference on human factors in computing systems, 2020, pp. 1-15.
  41. P. Lewis, M. Ott, J. Du, and V. Stoyanov, "Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art," in Proceedings of the 3rd Clinical Natural Language Processing Workshop. Online: Association for Computational Linguistics, Nov. 2020.
  42. J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, "Biobert: a pre-trained biomedical language representation model for biomedical text mining," Bioinformatics, vol. 36, no. 4, p. 1234-1240, Sep. 2019.
  43. R. Luo, L. Sun, Y. Xia, T. Qin, S. Zhang, H. Poon, and T.-Y. Liu, "Biogpt: generative pre-trained transformer for biomedical text generation and mining," Briefings in Bioinformatics, vol. 23, no. 6, Sep. 2022.
  44. Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, "Domain-specific language model pretraining for biomedical natural language processing," ACM Transactions on Computing for Healthcare, vol. 3, no. 1, pp. 1-23, oct 2021.
  45. A. B. Abacha, W.-w. Yim, Y. Fan, and T. Lin, "An empirical study of clinical note generation from doctor-patient encounters," in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 2283-2294.
  46. M. Liu, D. Zhang, W. Tan, and H. Zhang, "DeakinNLP at ProbSum 2023: Clinical progress note summarization with rules and language ModelsClinical progress note summarization with rules and languague models," in The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks. Toronto, Canada: Association for Computational Linguistics, Jul. 2023.
  47. Y. Cao, F. Liu, P. Simpson, L. Antieau, A. Bennett, J. J. Cimino, J. Ely, and H. Yu, "Askhermes: An online question answering system for complex clinical questions," Journal of biomedical informatics, vol. 44, no. 2, pp. 277-288, 2011.
  48. K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal et al., "Towards expert-level medical question answering with large language models," arXiv preprint arXiv:2305.09617, 2023. · arXiv
  49. M. Sallam, "Chatgpt utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns," in Healthcare, vol. 11, no. 6. MDPI, 2023, p. 887.
  50. S. L. Fleming, A. Lozano, W. J. Haberkorn, J. A. Jindal, E. P. Reis, R. Thapa, L. Blankemeier, J. Z. Genkins, E. Steinberg, A. Nayak et al., "Medalign: A clinician-generated dataset for instruction following with electronic medical records," arXiv preprint arXiv:2308.14089, 2023. · arXiv
  51. P. W. W. Koh, K.-S. Ang, H. Teo, and P. S. Liang, "On the accuracy of influence functions for measuring group effects," Advances in neural information processing systems, vol. 32, 2019.
  52. S. Zhang, J. Wang, H. Jiang, and R. Song, "Locally aggregated feature attribution on natural language model understanding," arXiv preprint arXiv:2204.10893, 2022. · arXiv

Cited by

Metadata

Authors
Worledge, T., Shen, J. H., Meister, N., Winston, C., & Guestrin, C.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2311.12233
arXiv
2311.12233
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.