CoChat
Sign Up
Cited by 15Robotic Process Automation ApplicationsBusiness Process Modeling and AnalysisArtificial Intelligence Applications

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Automating the Enterprise with Foundation Models

Open access
Wornow, Michael, Narayan, Avanika, Opsahl-Ong, Krista, McIntyre, Quinn, Shah, Nigam, Ré, Christopher·2024·Proceedings of the VLDB Endowment
DOI 10.14778/3681954.3681964
Open this paper in CoChat — free accountOpen PDF

Abstract

Automating enterprise workflows could unlock $4 trillion/year in productivity gains. Despite being of interest to the data management community for decades, the ultimate vision of end-to-end workflow automation has remained elusive. Current solutions rely on process mining and robotic process automation (RPA), in which a bot is hard-coded to follow a set of predefined rules for completing a workflow. Through case studies of a hospital and large B2B enterprise, we find that the adoption of RPA has been inhibited by high set-up costs (12--18 months), unreliable execution (60% initial accuracy), and burdensome maintenance (requiring multiple FTEs). Multimodal foundation models (FMs) such as GPT-4 offer a promising new approach for end-to-end workflow automation given their generalized reasoning and planning abilities. To study these capabilities we propose ECLAIR, a system to automate enterprise workflows with minimal human supervision. We conduct initial experiments showing that multimodal FMs can address the limitations of traditional RPA with (1) near-human-level understanding of workflows (93% accuracy on a workflow understanding task) and (2) instant set-up with minimal technical barrier (based solely on a natural language description of a workflow, ECLAIR achieves end-to-end completion rates of 40%). We identify human-AI collaboration, validation, and self-improvement as open challenges, and suggest ways they can be solved with data management techniques.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14778/3681954.3681964

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Coding agents have been applied to offensive security, theorem proving, and clinical tasks.

    “Coding agents have also been applied to offensive security [11, 37, 60], theorem proving [44], and clinical tasks [38, 43, 52].”

    From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [52]

    Section 3.1 discusses automation of hospital revenue-cycle workflows, and Section 5 gives physician sign-off before prescribing medication as a possible ECLAIR workflow. But the paper's preliminary evaluations (Section 4) use web-navigation tasks; our copy does not show coding agents being applied to clinical tasks. The clinical example is prospective, not a reported application. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 77

  1. Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes Byron David Chelsea Finn Chuyuan Fu Keerthana Gopalakrishnan Karol Hausman et al. 2022. Do as i can not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 (2022). · arXiv
  2. Automation Anywhere. 2020. https://www.automationanywhere.com/company/press-room/global-research-reveals-worlds-most-hated-office-tasks
  3. Rim Assouel, Tom Marty, Massimo Caccia, Issam H Laradji, Alexandre Drouin, Sai Rajeswar, Hector Palacios, Quentin Cappart, David Vazquez, Nicolas Chapados, et al. 2023. The Unsolved Challenges of LLMs as Generalist Web Agents: A Case Study. In NeurIPS 2023 Foundation Models for Decision Making Workshop.
  4. Adriano Augusto, Raffaele Conforti, Marlon Dumas, Marcello La Rosa, Fabrizio Maria Maggi, Andrea Marrella, Massimo Mecella, and Allar Soo. 2018. Automated discovery of process models from event logs: Review and benchmark. IEEE transactions on knowledge and data engineering 31, 4 (2018), 686--705.
  5. 10.1007/978-3-642-22655-7_23 · DOI
  6. Matthew Bayley and Ed Levine. 2013. Hospital revenue cycle operations: opportunities created by the ACA. Management (2013).
  7. 10.14778/2735703.2735708 · DOI
  8. Amanda Bergson-Shilcock and Roderick Taylor. 2023. Closing the Digital" Skill" Divide: The Payoff for Workers, Business, and the Economy. National Skills Coalition (2023).
  9. Alessandro Berti and Mahnaz Sadat Qafari. 2023. Leveraging Large Language Models (LLMs) for Process Mining (Technical Report). arXiv preprint arXiv:2307.12701 (2023). · arXiv
  10. 10.14778/2824032.2824100 · DOI
  11. Rishi Bommasani Drew A Hudson Ehsan Adeli Russ Altman Simran Arora Sydney von Arx Michael S Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021). · arXiv
  12. Fabio Casati and Ming-Chien Shan. 2000. Process automation as the foundation for e-business. In VLDB. Citeseer 688--691.
  13. 10.1007/978-3-030-58779-6_15 · DOI
  14. Cristiano André da Costa, Uélison Jean Lopes dos Santos, Eduardo Souza dos Reis, Rodolfo Stoffel Antunes, Henrique Chaves Pacheco, Thaynã da Silva França, Rodrigo da Rosa Righi, Jorge Luis Victória Barbosa, Franklin Jebadoss, Jorge Montalvao, et al. 2023. Intelligent methods for business rule processing: State-of-the-art. arXiv preprint arXiv:2311.11775 (2023). · arXiv
  15. Laila Dahabiyeh and Omar Mowafi. 2023. Challenges of using RPA in auditing: A socio-technical systems approach. Intelligent Systems in Accounting, Finance and Management (2023).
  16. Xiang Deng Yu Gu Boyuan Zheng Shijie Chen Samuel Stevens Boshi Wang Huan Sun and Yu Su. 2023. Mind2Web: Towards a Generalist Agent for the Web. arXiv:2306.06070 [cs.CL] · arXiv
  17. Norman Di Palo, Arunkumar Byravan, Leonard Hasenclever, Markus Wulfmeier, Nicolas Heess, and Martin Riedmiller. 2023. Towards a unified agent with foundation models. arXiv preprint arXiv:2307.09668 (2023). · arXiv
  18. 10.1145/3576047 · DOI
  19. Dirk Fahland, Fabian Fournier, Lior Limonad, Inna Skarbovsky, and Ava JE Swevels. 2024. How well can large language models explain business processes? arXiv preprint arXiv:2401.12846 (2024). · arXiv
  20. 10.33736/ijbs.4301.2021 · DOI
  21. 10.1109/WACVW60836.2024.00102 · DOI
  22. Hiroki Furuta, Ofir Nachum, Kuang-Huei Lee, Yutaka Matsuo, Shixiang Shane Gu, and Izzeddin Gur. 2023. Multimodal Web Navigation with Instruction-Finetuned Foundation Models. arXiv preprint arXiv:2305.11854 (2023). · arXiv
  23. Diimitrios Georgakopoulos, Mark Hornick, and Amit Sheth. 1995. An overview of workflow management: From process modeling to workflow automation infrastructure. Distributed and parallel Databases 3 (1995), 119--153.
  24. Michael Grohs, Luka Abb, Nourhan Elsayed, and Jana-Rebecca Rehse. 2023. Large Language Models can accomplish Business Process Management Tasks. In International Conference on Business Process Management. Springer, 453--465.
  25. Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. 2023. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arXiv:2307.12856 (2023). · arXiv
  26. Hongliang He Wenlin Yao Kaixin Ma Wenhao Yu Yong Dai Hongming Zhang Zhenzhong Lan and Dong Yu. 2024. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. arXiv:2401.13919 [cs.CL] · arXiv
  27. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023). · arXiv
  28. 10.1145/2463676.2467802 · DOI
  29. Peter C Humphreys, David Raposo, Tobias Pohlen, Gregory Thornton, Rachita Chhaparia, Alistair Muldal, Josh Abramson, Petko Georgiev, Adam Santoro, and Timothy Lillicrap. 2022. A data-driven approach for learning to control computers. In International Conference on Machine Learning. PMLR, 9466--9482.
  30. 10.1007/978-3-030-30429-4_19 · DOI
  31. 10.1145/306101.306112 · DOI
  32. Moe Kayali, Anton Lykov, Ilias Fountalis, Nikolaos Vasiloglou, Dan Olteanu, and Dan Suciu. 2023. CHORUS: Foundation Models for Unified Data Discovery and Exploration. arXiv preprint arXiv:2306.09610 (2023). · arXiv
  33. 10.51542/ijscia.v4i4.3 · DOI
  34. 10.1007/s12599-020-00641-4 · DOI
  35. Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M Mitchell, and Brad A Myers. 2019. Interactive task and concept learning from natural language instructions and gui demonstrations. arXiv preprint arXiv:1909.00031 (2019). · arXiv
  36. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118 (2023). · arXiv
  37. 10.14778/3554821.3554888 · DOI
  38. Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, et al. 2023. Bolaa: Benchmarking and orchestrating llm-augmented autonomous agents. arXiv preprint arXiv:2308.05960 (2023). · arXiv
  39. 10.1145/3183713.3196908 · DOI
  40. Pedro Mejia-Alvarez, Luis Eduardo Leyva-del Foyo, and Arnaldo Diaz-Ramirez. 2018. Interrupt Handling Schemes in Operating Systems. Springer.
  41. 10.1016/j.procs.2023.01.287 · DOI
  42. 10.18653/v1/2023.findings-emnlp.461 · DOI
  43. 10.14778/3574245.3574258 · DOI
  44. R OpenAI. 2023. GPT-4 technical report. arXiv (2023), 2303--08774.
  45. Long Ouyang Jeffrey Wu Xu Jiang Diogo Almeida Carroll Wainwright Pamela Mishkin Chong Zhang Sandhini Agarwal Katarina Slama Alex Ray et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022) 27730--27744.
  46. Andrew Pavlo, Gustavo Angulo, Joy Arulraj, Haibin Lin, Jiexi Lin, Lin Ma, Prashanth Menon, Todd C Mowry, Matthew Perron, Ian Quah, et al. 2017. Self-Driving Database Management Systems.. In CIDR, Vol. 4. 1.
  47. 10.14778/3476311.3476411 · DOI
  48. 10.1016/j.accinf.2023.100641 · DOI
  49. 10.14778/2809974.2809977 · DOI
  50. 10.1007/978-3-030-40172-6 · DOI
  51. Yara Rizk, Praveen Venkateswaran, Vatche Isahagian, Austin Narcomey, and Vinod Muthusamy. 2023. A Case for Business Process-Specific Foundation Models. In International Conference on Business Process Management. Springer, 44--56.
  52. Tara Safavi and Danai Koutra. 2021. Relational world knowledge representation in contextual language models: A review. arXiv preprint arXiv:2104.05837 (2021). · arXiv
  53. 10.32628/CSEIT2062106 · DOI
  54. Henriika Sarilo-Kankaanranta and Lauri Frank. 2021. The Slow Adoption Rate of Software Robotics in Accounting and Payroll Services and the Role of Resistance to Change in Innovation-Decision Process. In Conference of the Italian Chapter of AIS. Springer, 201--216.
  55. 10.1016/B978-155860869-6/50086-X · DOI
  56. Fred Schulte and Erika Fry. 2019. Death by 1,000 clicks: Where electronic health records went wrong. Kaiser Health News 18 (2019).
  57. Peter Shaw, Mandar Joshi, James Cohan, Jonathan Berant, Panupong Pasupat, Hexiang Hu, Urvashi Khandelwal, Kenton Lee, and Kristina Toutanova. 2023. From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces. arXiv preprint arXiv:2306.00245 (2023). · arXiv
  58. Yongliang Shen Kaitao Song Xu Tan Dongsheng Li Weiming Lu and Yueting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. arXiv:2303.17580 [cs.CL] · arXiv
  59. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning.(2023). arXiv preprint cs.AI/2303.11366 (2023).
  60. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models finetuned with human feedback. arXiv preprint arXiv:2305.14975 (2023). · arXiv
  61. Wil MP Van der Aalst. 2014. Process mining in the large: a tutorial. Business Intelligence: Third European Summer School, eBISS 2013, Dagstuhl Castle, Germany, July 7--12, 2013, Tutorial Lectures 3 (2014), 33--76.
  62. Maxim Vidgof, Stefan Bachhofner, and Jan Mendling. 2023. Large Language Models for Business Process Management: Opportunities and Challenges. arXiv preprint arXiv:2304.04309 (2023). · arXiv
  63. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023). · arXiv
  64. Zihao Wang Shaofei Cai Anji Liu Yonggang Jin Jinbing Hou Bowei Zhang Haowei Lin Zhaofeng He Zilong Zheng Yaodong Yang et al. 2023. Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models. arXiv preprint arXiv:2311.05997 (2023). · arXiv
  65. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824--24837.
  66. Judith Wewerka and Manfred Reichert. 2020. Robotic Process Automation-A Systematic Literature Review and Assessment Framework. arXiv preprint arXiv:2012.11951 (2020). · arXiv
  67. 10.1145/3544548.3581158 · DOI
  68. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023). · arXiv
  69. Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. 2024. OS-Copilot: Towards Generalist Computer Agents with Self-Improvement. arXiv preprint arXiv:2402.07456 (2024). · arXiv
  70. An Yan Zhengyuan Yang Wanrong Zhu Kevin Lin Linjie Li Jianfeng Wang Jianwei Yang Yiwu Zhong Julian McAuley Jianfeng Gao et al. 2023. Gpt-4v in wonderland: Large multimodal models for zero-shot smartphone gui navigation. arXiv preprint arXiv:2311.07562 (2023). · arXiv
  71. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441 (2023). · arXiv
  72. Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. AppAgent: Multimodal Agents as Smartphone Users. arXiv preprint arXiv:2312.13771 (2023). · arXiv
  73. Yining Ye Xin Cong Shizuo Tian Jiannan Cao Hao Wang Yujia Qin Yaxi Lu Heyang Yu Huadong Wang Yankai Lin et al. 2023. ProAgent: From Robotic Process Automation to Agentic Process Automation. arXiv preprint arXiv:2311.10751 (2023). · arXiv
  74. Liangzhao Zeng, Boualem Benatallah, Phuong Nguyen, and Anne HH Ngu. 2001. Agflow: Agent-based cross-enterprise workflow management system. In VLDB. 697--698.
  75. Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan, et al. 2024. UFO: A UI-Focused Agent for Windows OS Interaction. arXiv preprint arXiv:2402.07939 (2024). · arXiv
  76. Boyuan Zheng Boyu Gou Jihyung Kil Huan Sun and Yu Su. 2024. GPT-4V(ision) is a Generalist Web Agent if Grounded. arXiv:2401.01614 [cs.IR] · arXiv
  77. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854 (2023). · arXiv

Cited by

Sorted by citations

Metadata

Authors
Wornow, M., Narayan, A., Opsahl-Ong, K., McIntyre, Q., Shah, N., & Ré, C.
Affiliations
Stanford University (US)
ORCID
Venue
Proceedings of the VLDB Endowment
Publisher
Association for Computing Machinery (ACM)
Published
2024-07
Volume
17
Issue
11
Pages
2805-2812
ISSN
2150-8097 (ISSN-L)
DOI
10.14778/3681954.3681964
OpenAlex
W4402043034
Cited by
15
References
8
Citations by year
2026: 6 · 2025: 8 · 2024: 1
FWCI
3.07
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the VLDB Endowment (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (1763315, 1563078, 1937301)
  • Accenture
  • VMware
  • National Institutes of Health (N000141712266)
  • Office of Naval Research (N000141712266)
  • National Institute of Biomedical Imaging and Bioengineering (U54EB020405)
  • U.S. Army Combat Capabilities Development Command
Topics
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Robotic Process Automation Applications
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Business Process Modeling and Analysis
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
multimodal foundation models, GPT-4, robotic process automation, process mining, ECLAIRs, human-AI collaboration, workflow validation, self-improving AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review