The DOI matches its DataCite record.
Parrot: Efficient Serving of LLM-based Applications with Semantic Variable
Open accessAbstract
The rise of large language models (LLMs) has enabled LLM-based applications (a.k.a. AI agents or co-pilots), a new software paradigm that combines the strength of LLM and conventional software. Diverse LLM applications from different tenants could design complex workflows using multiple LLM requests to accomplish one task. However, they have to use the over-simplified request-level API provided by today's public LLM services, losing essential application-level information. Public LLM services have to blindly optimize individual LLM requests, leading to sub-optimal end-to-end performance of LLM applications. This paper introduces Parrot, an LLM service system that focuses on the end-to-end experience of LLM-based applications. Parrot proposes Semantic Variable, a unified abstraction to expose application-level knowledge to public LLM services. A Semantic Variable annotates an input/output variable in the prompt of a request, and creates the data pipeline when connecting multiple LLM requests, providing a natural way to program LLM applications. Exposing Semantic Variables to the public LLM service allows it to perform conventional data flow analysis to uncover the correlation across multiple LLM requests. This correlation opens a brand-new optimization space for the end-to-end performance of LLM-based applications. Extensive evaluations demonstrate that Parrot can achieve up to an order-of-magnitude improvement for popular and practical use cases of LLM applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Agent-aware, compute-centric schedulers decide when each request runs, but they do not control how the resulting KV Caches are computed or stored.
“These systems decide when each request runs, but they do not control how the resulting KV Caches are computed or stored.”
From TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing (2026) · cites this paper as [17]
Section 5.4 supports the first part: Parrot schedules requests and assigns them to LLM engines. But it contradicts the claimed limitation: the scheduler co-locates requests with a common prefix to use context forking and reduce redundant computation and GPU memory transactions. Section 5.3 also describes sharing prefix KV caches to save GPU memory. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredTeola, Alto, and Parrot assume static or deterministically defined DAGs and cannot work with dynamic agent workloads like ReAct-styled ones whose dependency graphs evolve at runtime.
“One shared limitation of Teola, Parrot, and Alto is that they all assume static or deterministically defined DAGs and could not work with dynamic agent workloads like ReAct-styled ones whose dependency graphs evolve at runtime.”
From Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live (2025) · cites this paper as [34]
Section 6 (Discussion; page unavailable) says Parrot does not currently support cloud-side orchestration involving dynamic control flow; such functions still require client-side execution. But §4.2 says Parrot adds requests to its DAG as they arrive, so the claim that it assumes a static DAG is too strong. The paper does not specifically assess ReAct-style workloads or establish that they cannot work with Parrot at all. (LLM, conf=0.86)
Experimental — accuracy not yet measuredAyo, Alto, and Parrot assume static workflows and do not apply to dynamic agents.
“Ayo [49], Alto [46], and Parrot [34] assume static workflows and do not apply to dynamic agents.”
From Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live (2025) · cites this paper as [34]
Section 6 (page unavailable) says Parrot does not currently handle dynamic control flow in cloud-side orchestration; that execution remains client-side. But §8.4 (page unavailable) evaluates Parrot on an interactive MetaGPT multi-agent workflow and describes its handling of dynamically generated variables. The blanket claim that Parrot does not apply to dynamic agents therefore overstates the limitation. (LLM, conf=0.88)
Experimental — accuracy not yet measuredParrot, Autellix, Teola, ScaleSim, and Tokencake optimize scheduling, prefetching, or execution order for agent applications.
“Parrot [17], Autellix [22], Teola [31], ScaleSim [25], and Tokencake [2] optimize scheduling, prefetching, or execution order for agent applications.”
From TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing (2026) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Research works printed but not linked: 35
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, San- jay Ghemawat, Geoffrey Irving, Michael Isard, Man- junath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: A system for Large- Scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265-283, Savannah, GA, November 2016. USENIX Association.
- Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S Gulavani, Alexey Tu- manov, and Ramachandran Ramjee. Taming throughput- latency tradeoff in llm inference with sarathi-serve. arXiv preprint arXiv:2403.02310, 2024. · arXiv
- Ganesh Ananthanarayanan, Srikanth Kandula, Albert Greenberg, Ion Stoica, Yi Lu, Bikas Saha, and Edward Harris. Reining in the outliers in Map-Reduce clusters using mantri. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10), Van- couver, BC, October 2010. USENIX Association.
- Zhengda Bian, Hongxin Liu, Boxiang Wang, Haichen Huang, Yongbin Li, Chuanrui Wang, Fan Cui, and Yang You. Colossal-ai: A unified deep learning system for large-scale parallel training. CoRR, abs/2110.14883, 2021. · arXiv
- Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. Clipper: A Low-Latency online prediction serving system. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17), pages 613-627, Boston, MA, March 2017. USENIX Association.
- Tri Dao. Flashattention-2: Faster attention with bet- ter parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023. · arXiv
- Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Process- ing Systems, volume 35, pages 16344-16359.
- Robert Grandl, Mosharaf Chowdhury, Aditya Akella, and Ganesh Ananthanarayanan. Altruistic scheduling in Multi-Resource clusters. In 12th USENIX Sympo- sium on Operating Systems Design and Implementa- tion (OSDI 16), pages 65-80, Savannah, GA, November 2016. USENIX Association.
- Robert Grandl, Srikanth Kandula, Sriram Rao, Aditya Akella, and Janardhan Kulkarni. GRAPHENE: Packing and Dependency-Aware scheduling for Data-Parallel clusters. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 81-97, Savannah, GA, November 2016. USENIX Asso- ciation.
- Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Liu, and Chuanxiong Guo. Tiresias: A GPU cluster manager for distributed deep learning. In 16th USENIX Sympo- sium on Networked Systems Design and Implementation (NSDI 19), pages 485-500, Boston, MA, February 2019. USENIX Association.
- Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace. Serving DNNs like clockwork: Performance predictability from the bottom up. In 14th USENIX Sym- posium on Operating Systems Design and Implementa- tion (OSDI 20), pages 443-462. USENIX Association, November 2020.
- Mingcong Han, Hanze Zhang, Rong Chen, and Haibo Chen. Microsecond-scale preemption for concurrent GPU-accelerated DNN inferences. In 16th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI 22), pages 539-558, Carlsbad, CA, July 2022. USENIX Association.
- Connor Holmes, Masahiro Tanaka, Michael Wyatt, Am- mar Ahmad Awan, Jeff Rasley, Samyam Rajbhan- dari, Reza Yazdani Aminabadi, Heyang Qin, Arash Bakhtiari, Lev Kurilenko, et al. Deepspeed-fastgen: High-throughput text generation for llms via mii and deepspeed-inference. arXiv preprint arXiv:2401.08671, 2024. · arXiv
- Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, et al. Metagpt: Meta programming for multi-agent collaborative frame- work. arXiv preprint arXiv:2308.00352, 2023. · arXiv
- Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceed- ings of the 2nd ACM SIGOPS/EuroSys European Con- ference on Computer Systems 2007, EuroSys '07, page 59-72, New York, NY, USA, 2007. Association for Com- puting Machinery.
- Suhas Jayaram Subramanya, Daiyaan Arfeen, Shouxu Lin, Aurick Qiao, Zhihao Jia, and Gregory R. Ganger. Sia: Heterogeneity-aware, goodput-optimized ml-cluster scheduling. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP '23, page 642-657, New York, NY, USA, 2023. Association for Computing Machinery.
- Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory man- agement for large language model serving with page- dattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP '23, page 611-626, New York, NY, USA, 2023. Association for Computing Machinery.
- Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Al- paServe: Statistical multiplexing with model parallelism for deep learning serving. In 17th USENIX Sympo- sium on Operating Systems Design and Implementa- tion (OSDI 23), pages 663-679, Boston, MA, July 2023. USENIX Association.
- Ashraf Mahgoub, Karthick Shankar, Subrata Mitra, Ana Klimovic, Somali Chaterji, and Saurabh Bagchi. SONIC: Application-aware data passing for chained serverless applications. In 2021 USENIX Annual Tech- nical Conference (USENIX ATC 21), pages 285-301. USENIX Association, July 2021.
- Ashraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Sameh Elnikety, Somali Chaterji, and Saurabh Bagchi. ORION and the three rights: Sizing, bundling, and prewarming for serverless DAGs. In 16th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI 22), pages 303-320, Carlsbad, CA, July 2022. USENIX Association.
- Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, Amar Phanishayee, and Matei Zaharia. Heterogeneity-Aware cluster scheduling policies for deep learning workloads. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pages 481-498. USENIX Association, November 2020.
- Flemming Nielson, Hanne R Nielson, and Chris Hankin. Principles of program analysis. Springer, 2015.
- Christopher Olston, Fangwei Li, Jeremiah Harmsen, Jor- dan Soyke, Kiril Gorovoy, Li Lao, Noah Fiedel, Sukriti Ramesh, and Vinu Rajashekhar. Tensorflow-serving: Flexible, high-performance ml serving. In Workshop on ML Systems at NIPS 2017, 2017.
- Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Aashaka Shah, Saeed Maleki, and Ricardo Bian- chini. Splitwise: Efficient generative llm inference using phase splitting. arXiv preprint arXiv:2311.18677, 2023. · arXiv
- Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 2023. · arXiv
- Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. CoRR, abs/1909.08053, 2019. · arXiv
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Perric Cistac, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-Art Natural Language Processing. pages 38-45. Association for Computational Linguistics, Oc- tober 2020.
- Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xi- aoyun Zhang, and Chi Wang. Autogen: Enabling next- gen llm applications via multi-agent conversation frame- work. arXiv preprint arXiv:2308.08155, 2023. · arXiv
- Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv, 2023.
- Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soo- jeong Kim, and Byung-Gon Chun. Orca: A distributed serving system for Transformer-Based generative mod- els. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 521-538, Carlsbad, CA, July 2022. USENIX Association.
- Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In Proceedings of the 5th European Conference on Computer Systems, EuroSys '10, page 265-278, New York, NY, USA, 2010. Association for Computing Machinery.
- Matei Zaharia, Andy Konwinski, Anthony D Joseph, Randy H Katz, and Ion Stoica. Improving mapreduce performance in heterogeneous environments. In 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 08), San Diego, CA, 2008.
- Hong Zhang, Yupeng Tang, Anurag Khandelwal, Jin- grong Chen, and Ion Stoica. Caerus: NIMBLE task scheduling for serverless analytics. In 18th USENIX Symposium on Networked Systems Design and Imple- mentation (NSDI 21), pages 653-669. USENIX Associ- ation, April 2021.
- Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Francis C.M. Lau, Yuqi Wang, Yifan Xiong, and Bin Wang. HiveD: Sharing a GPU cluster for deep learning with guarantees. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pages 515-532. USENIX Association, November 2020.
- Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. Dist- serve: Disaggregating prefill and decoding for goodput- optimized large language model serving. arXiv preprint arXiv:2401.09670, 2024. · arXiv
Cited by
Sorted by citations
LLM Inference Serving: Survey of Recent Advances and Opportunities
Li, B., Jiang, Y., Gadepally, V. N., & Tiwari, D. (2024) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec62836.2024.10938426 · Cited by 108
Prompt Flow Integrity to Prevent Privilege Escalation in LLM AgentsPreprint
Kim, J., Choi, W., & Lee, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.15547 · Cited by 98
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Yao, J., Li, H., Liu, Y., Ray, S., Cheng, Y., Zhang, Q., Du, K., Lu, S., & Jiang, J. (2025) · Proceedings of the Twentieth European Conference on Computer Systems · doi:10.1145/3689031.3696098 · Cited by 75
MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs
Wang, Q., Wang, T., Tang, Z., Li, Q., Chen, N., Liang, J., & He, B. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.259 · Cited by 61
METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
Ray, S., Pan, R., Gu, Z., Du, K., Feng, S., Ananthanarayanan, G., Netravali, R., & Jiang, J. (2025) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3731569.3764855 · Cited by 38
Zhao, Y., Yang, S., Zhu, K., Zheng, L., Kasikci, B., Zhou, Y., Xing, J., & Stoica, I. G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.16102 · Cited by 23
Thus Spake Long-Context Large Language ModelPreprint
Liu, X., Li, R., Huang, M., Liu, Z., Song, Y., Guo, Q., He, S., Wang, Q., Li, L., Liu, Q., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.17129 · Cited by 19
KubeIntellect: A Modular LLM-Orchestrated Agent Framework for End-to-End Kubernetes Management
Ardebili, M. S., & Bartolini, A. (2026) · Journal of Grid Computing · doi:10.1007/s10723-026-09837-6 · Cited by 16
Jenga: Effective Memory Management for Serving LLM with Heterogeneity
Zhang, C., Du, K., Liu, S., Kwon, W., Mo, X., Wang, Y., Liu, X., You, K., Li, Z., Long, M., et al. (2025) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3731569.3764823 · Cited by 16
Self-citationTaming the Titans: A Survey of Efficient LLM Inference ServingPreprint
Zhen, R., Li, J., Ji, Y., Yang, Z., Liu, T., Xia, Q., Duan, X., Wang, Z., Huai, B., & Zhang, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.19720 · Cited by 15
Towards End-to-End Optimization of LLM-based Applications with Ayo
Tan, X., Jiang, Y., Yang, Y., & Xu, H. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3676641.3716278 · Cited by 13
Self-citationJenga: Effective Memory Management for Serving LLM with HeterogeneityPreprint
Zhang, C., Du, K., Liu, S., Kwon, W., Mo, X., Wang, Y., Liu, X., You, K., Li, Z., Long, M., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18292 · Cited by 11
Self-citationCHASE: A Native Relational Database for Hybrid Queries on Structured and Unstructured DataPreprint
Ma, R., Zhang, K., He, Z., Jing, Y., Wang, X. S., & Chen, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.05006 · Cited by 10
MVVM: Deploy Your AI Agents-Securely, Efficiently, EverywherePreprint
Yang, Y., Hu, A., Zheng, Y., Zhao, B., Zhang, X., Xiang, D., Chu, K., Zhang, W., & Quinn, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15894 · Cited by 9
Self-citationCannikin: No Lagger of SLO in Concurrent Multiple LoRA LLM Serving
Zhu, R., Jiang, Z., Zhang, Z., Liu, X., Liu, X., & Jin, X. (2025) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2025.3590014 · Cited by 7
IoTCoder: A Copilot for IoT Application Development
Shen, L., & Zheng, Y. (2024) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3636534.3697447 · Cited by 6
Software Performance Engineering for Foundation Model-Powered Software
Zhang, H., Chang, S., Leung, A., Thangarajah, K., Chen, B., Lutfiyya, H. L., & Hassan, A. E. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3794844 · Cited by 4
TSGuard: Automated User-Centric Incident Diagnosis for AI Workloads in the Cloud
Yang, Y., Deng, Y., Xiong, Y., Li, B., Xu, H. L., & Cheng, P. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797149 · Cited by 3
Self-citationHeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding
Liu, S., Ye, Y., Zhu, Q., Cao, Z., & He, Y. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.589 · Cited by 3
Mercy, I., Nainwal, A., Balasubramani, R., Nandankar, P. V., Parikh, S. M., & Taqui, S. N. (2024) · International Conference on Sustainable Expert Systems (ICSES) · doi:10.1109/icses63445.2024.10763331 · Cited by 3
GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI PipelinesPreprint
Wong, M., Butler, U., Farkash, E., Tammana, P., Sivaraman, A., & Netravali, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.15712 · Cited by 2
Evaluating and Enhancing the Vulnerability Reasoning Capabilities of Large Language ModelsPreprint
Lu, L., Zhao, Y., Rao, H., Zhang, K., & Wang, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.06687 · Cited by 2
Serving Agentic Workflows with a Physical-Plan Compiler and Adaptive RuntimePreprint
Qian, J., Zhang, Y., Yang, H., Tao, C., Kundu, S., Wan, Z., & Krishna, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.02942 · Cited by 2
Crossflow: Prefill-Decode Elasticity for Agentic LLM ServingPreprint
Xu, Y., Ardestani, E., Fu, W., Schätz, M., Malladi, K. T., Shu, Z., Aziz, A., Kanaujia, S., Mathews, A., & Tang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.27085 · Cited by 2
Semantic Scheduling for LLM InferencePreprint
Hua, W., Ding, D., Gu, Y., Ren, Y., Mei, K., Ma, M., & Wang, W. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.12204 · Cited by 1
SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineering
Pan, Z., Wang, Q., Hu, Z., Chen, C., Guan, Y., Zhou, Y., Swanson, S., & Ding, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.00151
“Parrot [37] and Ayo [61] model static agent workflows as directed acyclic graphs (DAGs) and apply a range of optimizations over the execution graph.”
Self-citationTokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
Bian, Z., Wu, F., Zhang, C., Dong, H., Liang, Y., & Zhuo, Y. (2026) · arXiv · doi:10.48550/arxiv.2604.03143
Self-citationContinuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
Li, H., He, R., Mang, Q., Zhang, Q., Mao, H., Chen, X., Zhou, H., Zhang, H., Cheung, A., Gonzalez, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.02230
BlendServe: Optimizing Offline Inference with Resource-Aware Batching
Zhao, Y., Yang, S., Zhu, K., Zheng, L., Kasikci, B., Qiao, Y., Zhou, Y., Xing, J., & Stoica, I. (2026) · Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3779212.3790133
Gim, I., & Zhong, L. (2025) · Workshop on Hot Topics in Operating Systems · doi:10.1145/3713082.3730398
Li, R., & Perry, J. (2025) · doi:10.31224/5507
Alchemist: Towards the Design of Efficient Online Continual Learning SystemPreprint
Huang, Y., Liu, Y., Gunawi, H. S., Li, B., & Changho, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.01066
SLA-Awareness for AI-assisted codingPreprint
Thangarajah, K., Leung, A., Chen, B., & Hassan, A. E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.19876
Adaptive optimization for hybrid vector-relational queries via cost-aware plan switching
Kumar, A., Rajput, K., Upadhyay, U., Malik, R., Roy, S., & Sharma, L. (2026) · Scientific Reports · doi:10.1038/s41598-026-58877-1
DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM ServingPreprint
Shen, J., Wadlom, N., Su, Z., & Lu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.31047
Efficient Agentic LLM Serving over SSD-based Sparse KV StoragePreprint
He, W., Zhang, P., Hu, X., Wang, C., Hou, J., Cheng, Y., Sun, P., & Fu, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36938
Metadata
- Authors
- Lin, C., Han, Z., Zhang, C., Yang, Y., Yang, F., Chen, C., & Qiu, L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Microsoft (United States) (US)
- ORCID
- Yang, Yuqing
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2405.19888
- OpenAlex
- W4399252473
- Cited by
- 8
- References
- 0
- Citations by year
- 2025: 5 · 2024: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Keywords
- AI agents, data flow analysis, prompt engineering, data pipelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
