CoChat
Sign Up
PreprintCited by 4Spreadsheets and End-User ComputingSoftware Engineering ResearchExplainable Artificial Intelligence (XAI)

Co-audit: tools to help humans double-check AI-generated content

Open access
Gordon, Andrew D., Negreanu, Carina, Cambronero, José, Chakravarthy, Rasika, Drosos, Ian, Fang, Hao, Mitra, Bhaskar, Richardson, Hannah +4 more·2023·arXiv
DOI 10.48550/arxiv.2310.01297arXiv 2310.01297
Open this paper in CoChat — free accountOpen PDF

Abstract

Users are increasingly being warned to check AI-generated content for correctness. Still, as LLMs (and other generative models) generate more complex output, such as summaries, tables, or code, it becomes harder for the user to audit or evaluate the output for quality or correctness. Hence, we are seeing the emergence of tool-assisted experiences to help the user double-check a piece of AI-generated content. We refer to these as co-audit tools. Co-audit tools complement prompt engineering techniques: one helps the user construct the input prompt, while the other helps them check the output response. As a specific example, this paper describes recent research on co-audit tools for spreadsheet computations powered by generative models. We explain why co-audit experiences are essential for any application of generative AI where quality is important and errors are consequential (as is common in spreadsheet computations). We propose a preliminary list of principles for co-audit, and outline research challenges.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2310.01297

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 43

  1. Saleema Amershi, Daniel S. Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi T. Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for human-AI interaction. In CHI, page 3. ACM.
  2. Amin Beheshti, Jian Yang, Quan.Z Sheng, Boualem Benatallah, Fabio Casati, Schahram Dustdar, Hamid R. Motahari Nezhad, Xuyun Zhang, and Shan Xue. 2023. ProcessGPT: Transforming business process management with generative artificial intelligence. 2023 IEEE International Conference on Web Services (ICWS), pages 731-739.
  3. Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, and Katrina Panovich. 2015. Soylent: a word processor with a crowd inside. Commun. ACM, 58(8):85-94. · DOI
  4. Yi Cao and Jianwu Zhai. 2023. Bridging the gap -the impact of ChatGPT on financial research. Journal of Chinese Economic and Business Studies, 21:177 -191.
  5. Leah Chong, Guanglu Zhang, Kosa Goucher-Lambert, Kenneth Kotovsky, and Jonathan Cagan. 2022. Human confidence in artificial intelligence and in themselves: The evolu- tion and impact of confidence on adoption of AI advice. Computers in Human Behavior, 127. · DOI · Paper page
  6. Thomas H. Davenport and DJ Patil. 2012. Data scientist: The sexiest job of the 21st century. Harvard Business Review.
  7. Ian Drosos, Nicholas Wilson, Andrew D. Gordon, Sruti Srinivasa Ragavan, and Jack Williams. 2023. FxD: a functional debugger for dysfunctional spreadsheets. In IEEE Symposium on Visual Languages and Human-Centric Computing.
  8. Katie D. Evans, Scott A. Robbins, and Joanna J. Bryson. 2023. Do we collaborate with what we design? Topics in Cognitive Science. · DOI
  9. Jerry Alan Fails and Dan R Olsen Jr. 2003. Interactive machine learning. In Proceedings of the 8th international conference on Intelligent user interfaces, pages 39-45.
  10. Kasra Ferdowsi, Jack Williams, Ian Drosos, Andrew D. Gordon, Carina Negreanu, Nadia Polikarpova, Advait Sarkar, and Ben Zorn. 2023b. ColDeco: An end user spreadsheet inspection tool for AI-generated code. In VL/HCC. IEEE.
  11. Andrew D Gordon, Claudio Russo, Marcin Szymczak, Johannes Borgström, Nicolas Rol- land, Thore Graepel, and Daniel Tarlow. 2015. Probabilistic programs as spreadsheet queries. In Programming Languages and Systems: 24th European Symposium on Pro- gramming, ESOP 2015, pages 1-25. Springer.
  12. Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. In POPL, pages 317-330. ACM.
  13. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12):248:1-248:38.
  14. Karen Sparck Jones and Julia R. Galliers. 1996. Evaluating Natural Language Processing Systems: An Analysis and Review . Springer-Verlag, Berlin, Heidelberg. · DOI
  15. Junaed Younus Khan and Gias Uddin. 2022. Automatic code documentation generation using GPT-3. In ASE, pages 174:1-174:6. ACM.
  16. Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1885-1894. PMLR.
  17. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces, pages 126-137.
  18. Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. In Advances in Neural Information Processing Systems, volume 35, pages 34586-34599.
  19. Brian Y Lim and Anind K Dey. 2009. Assessing demand for intelligibility in context- aware applications. In Proceedings of the 11th international conference on Ubiquitous computing, pages 195-204.
  20. Michael Xieyang Liu, Advait Sarkar, Carina Negreanu, Benjamin G. Zorn, Jack Williams, Neil Toronto, and Andrew D. Gordon. 2023. 'What it wants me to say': Bridging the abstraction gap between end-user programmers and code-generating large language models. In CHI, pages 598:1-598:31. ACM.
  21. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model pre- dictions. In NIPS, pages 4765-4774.
  22. Insa Mannstadt and Bella Mehta. 2023. Large language models and the future of rheuma- tology: assessing impact and emerging opportunities. Current opinion in rheumatology, pages 10-1097.
  23. Elisabeth Norman, Gerit Pfuhl, Rannveig Grøm Sæle, Frode Svartdal, Torstein Låg, and Tove Irene Dahl. 2019. Metacognition in psychology. Review of General Psychology, 23(4). · DOI
  24. Ingrid Nunes and Dietmar Jannach. 2017. A systematic review and taxonomy of explana- tions in decision support and recommender systems. User Modeling and User-Adapted Interaction, 27:393-444.
  25. Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: defining an end-to-end framework for internal algo- rithmic auditing. In FAT*, pages 33-44. ACM.
  26. Charvi Rastogi, Marco Tulio Ribeiro, Nicholas King, Harsha Nori, and Saleema Amershi. 2023. Supporting human-AI collaboration in auditing LLMs with LLMs. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, AIES '23, page 913-926, New York, NY, USA. Association for Computing Machinery. · DOI
  27. Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should I trust you?": Explaining the predictions of any classifier. In KDD, pages 1135-1144. ACM.
  28. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902-4912, Online. Association for Computational Linguistics. · DOI · Paper page
  29. Chris Russell. 2019. Efficient search for diverse coherent explanations. In FAT, pages 20-28. ACM.
  30. Arleen Salles, Kathinka Evers, and Michele Farisco. 2020. Anthropomorphism in AI. AJOB Neuroscience, 11(2):88-95. · DOI · Paper page
  31. Advait Sarkar. 2022. Is explainable AI a race against model complexity? In Workshop on Transparency and Explanations in Smart Systems (TeXSS), in conjunction with ACM Intelligent User Interfaces (IUI 2022), number 3124 in CEUR Workshop Proceedings, pages 192-199.
  32. Advait Sarkar. 2023. Will Code Remain a Relevant User Interface for End-User Pro- gramming with Generative AI Models? In Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Program- ming and Software (Onward! '23), page 15, Cascais, Portugal. ACM. · DOI
  33. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ra- gavan, and Ben Zorn. 2022. What is it like to program with artificial intelligence? In Proceedings of the 33rd Annual Conference of the Psychology of Programming Interest Group (PPIG 2022).
  34. Advait Sarkar, Mateja Jamnik, Alan F Blackwell, and Martin Spott. 2015. Interactive visual machine learning in spreadsheets. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pages 159-163. IEEE.
  35. Mark Steyvers and Aakriti Kumar. 2023. Three challenges for AI-assisted decision-making. Perspectives on Psychological Science. · DOI · Paper page
  36. Heliodoro Tejeda, Aakriti Kumar, Padhraic Smyth, and Mark Steyvers. 2022. AI-assisted decision-making: a cognitive modeling approach to infer latent reliance strategies. Com- putational Brain & Behavior, 5:491-508. · DOI · Paper page
  37. Nava Tintarev and Judith Masthoff. 2010. Designing and evaluating explanations for recommender systems. In Recommender systems handbook, pages 479-510. Springer.
  38. C.-H. Tsai, S. Kadire, T. Sreeramdas, M. VanOrmer, M. Thoene, C. Hanson, A.A. Berry, and D. Khazanchi. 2023. Generating personalized pregnancy nutrition recommendations with GPT-powered AI chatbot. Proceedings of the 20th International Conference on Information Systems for Crisis Response and Management.
  39. Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable recourse in linear classification. In FAT, pages 10-19. ACM.
  40. Yingjie Xv, Canjie Peng, Zongjie Wei, Fangtong Liao, and Mingzhao Xiao. 2023. Can chat- gpt a substitute for urological resident physician in diagnosing diseases?: a preliminary conclusion from an exploratory investigation. World Journal of Urology, 41:2569 -2571.
  41. J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qiang Yang. 2023. Why Johnny can't prompt: How non-AI experts try (and fail) to design LLM prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. · DOI · Paper page
  42. Xiaoyu Zhang, Jianping Li, Po-Wei Chi, Senthil Chandrasegaran, and Kwan-Liu Ma. 2023. ConceptEVA: Concept-based interactive exploration and customization of document summaries. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. · DOI
  43. Xin Zhou, Kisub Kim, Bowen Xu, DongGyun Han, Junda He, and David Lo. 2023. Generation-based code review automation: How far are we? In ICPC, pages 215-226. IEEE.

Cited by

Sorted by citations

Authors
Gordon, A. D., Negreanu, C., Cambronero, J., Chakravarthy, R., Drosos, I., Fang, H., Mitra, B., Richardson, H., Sarkar, A., Simmons, S., Williams, J., & Zorn, B.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2310.01297
arXiv
2310.01297
OpenAlex
W6884737364
Cited by
4
References
0
Citations by year
2026: 2 · 2024: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
  • KiltHub Repository (cc-by)
Funders
Microsoft Research
Topics
  • Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
AI-generated content, generative AI, large language models, prompt engineering, output quality, correctness checking, design principles

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review