Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 34 of the 47 sources its claims cite were checked for support.

Real-Time Execution of Action Chunking Flow Policies

Open access
Black, Kevin, Galliker, Manuel, Levine, Sergey · 2025 · Advances in Neural Information Processing Systems 38

Abstract

Modern AI systems, especially those interacting with the physical world, increasingly require real-time performance. However, the high latency of state-of-the-art generalist models, including recent vision-language action models (VLAs), poses a significant challenge. While action chunking has enabled temporal consistency in high-frequency control tasks, it does not fully address the latency problem, leading to pauses or out-of-distribution jerky movements at chunk boundaries. This paper presents a novel inference-time algorithm that enables smooth asynchronous execution of action chunking policies. Our method, real-time chunking (RTC), is applicable to any diffusion- or flow-based VLA out of the box with no re-training. It generates the next action chunk while executing the current one, "freezing" actions guaranteed to execute and "inpainting" the rest. To test RTC, we introduce a new benchmark of 12 highly dynamic tasks in the Kinetix simulator, as well as evaluate 6 challenging real-world bimanual manipulation tasks. Results demonstrate that RTC is fast, performant, and uniquely robust to inference delay, significantly improving task throughput and enabling high success rates in precise tasks $\unicode{x2013}$ such as lighting a match $\unicode{x2013}$ even in the presence of significant latency. See https://pi.website/research/real_time_chunking for videos.

Overview

CoChat Summary

TL;DR

The paper introduces real-time chunking (RTC), an inference-time method that executes action-chunking diffusion/flow policies asynchronously by treating the next chunk as an inpainting problem. Across a 12-task Kinetix benchmark and 6 real-world bimanual tasks, RTC improves robustness to inference delay and task throughput without retraining.

Key findings

  • In simulation, RTC is the most robust to inference delay among compared methods, outperforming bidirectional decoding (BID) and temporal ensembling (TE); soft masking improves performance especially at lower delays and execution horizons.
  • In real-world experiments with π0.5, RTC achieves the best average task throughput at all inference delays, with statistically significant gains at +100 ms and +200 ms injected latency; synchronous inference degrades linearly, while both TE variants fail at those delays due to high oscillations triggering protective stops.
  • RTC is completely robust to injected delay in the real-world setting, showing no degradation, and completes tasks faster than synchronous inference even when inference pauses are removed; in the precision-sensitive light candle task, RTC shows a large final-score advantage.
  • Figure 1 reports RTC performs the same robot motion 20% faster than synchronous inference and smoother than all competing methods, including temporal ensembling.

Methods

  • RTC is an inference-time algorithm for asynchronous execution of action-chunking policies: it generates the next action chunk while executing the current one, freezes actions guaranteed to execute due to inference delay, and inpaints the rest using a guidance-based flow-matching inpainting procedure with soft masking and guidance weight clipping (β = 5).
  • Simulated benchmark: 12 highly dynamic tasks in the Kinetix simulator, with expert policies trained via RPO, action-chunking flow policies with prediction horizon H = 8 and a 4-layer MLP-Mixer, and 2048 rollouts per data point; delays simulated between 0 and 4.
  • Real-world evaluation: π0.5 VLA with H = 50, Δt = 20 ms, n = 5 denoising steps, model latency 76 ms for baselines and 97 ms for RTC; 6 challenging bimanual manipulation tasks, 10 trials per task and method for 480 episodes and 28 hours of robot execution time, with injected latency of +100 ms and +200 ms.

Limitations

  • The paper states RTC adds significant computational overhead compared to sampling directly from the base policy, is applicable only to diffusion- and flow-based policies, and its real-world experiments do not cover legged locomotion even though it is represented in simulation.
  • The paper states that the data used to train π0.5 and the real robot runtime code are not released because they are proprietary; only the simulated benchmark code is provided.
  • Evident from the text: real-world results use a single base policy (π0.5) and 10 trials per task/method, so generalization across policies and statistical precision are limited beyond what is reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/085713-1122

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Top: Real-time chunking (RTC) enables the robot to perform highly dexterous and dynamic tasks, such as lighting a match-even in the presence of inference delays in excess of 300 milliseconds, corresponding to more than 30% of the model's prediction horizon. Bottom: RTC performs the same robot motion 20% faster than synchronous inference [5, 30, 8, 24, 31, 59], and smoother than all competing methods, including temporal ensembling [68]. The shown positions, velocities, and accelerations correspond to the shoulder joint of one arm, and are taken from the first 10 seconds of a real autonomous match-lighting rollout.
Figure 1 Top: Real-time chunking (RTC) enables the robot to perform highly dexterous and dynamic tasks, such as lighting a match-even in the presence of inference delays in excess of 300 milliseconds, corresponding to more than 30% of the model's prediction horizon. Bottom: RTC performs the same robot motion 20% faster than synchronous inference [5, 30, 8, 24, 31, 59], and smoother than all competing methods, including temporal ensembling [68]. The shown positions, velocities, and accelerations correspond to the shoulder joint of one arm, and are taken from the first 10 seconds of a real autonomous match-lighting rollout.From the PDF
Figure 2 An illustration of a typical bifurcation between consecutive chunks. Inference is started between timesteps 3 and 4. The original chunk that was executing, { a t } (black), had planned to go above the obstacle while the newly generated chunk { a ′ t } (red) goes below the obstacle. However, { a ′ t } is not available until d = 7 steps later. A naive asynchronous algorithm might jump from a 10 to a ′ 11 , inducing a very high, outof-distribution acceleration. Temporal ensembling [68], i.e., interpolating between chunks, reduces the acceleration but produces poor actions.
Figure 2 An illustration of a typical bifurcation between consecutive chunks. Inference is started between timesteps 3 and 4. The original chunk that was executing, { a t } (black), had planned to go above the obstacle while the newly generated chunk { a ′ t } (red) goes below the obstacle. However, { a ′ t } is not available until d = 7 steps later. A naive asynchronous algorithm might jump from a 10 to a ′ 11 , inducing a very high, outof-distribution acceleration. Temporal ensembling [68], i.e., interpolating between chunks, reduces the acceleration but produces poor actions.From the PDF
Figure 3 A diagram illustrating how action generation attends to the previous action chunk in real-time chunking. If inference starts after the execution of a -1 and the inference delay is d = 4 , then the newly generated chunk will not be available until after a 3 is consumed. Therefore, a 0:3 are 'frozen' and are attended to with a full guidance weight of 1. In the intermediate region, a 4:10 , actions from the previous chunk are available but may be updated, since inference will have finished before a 4 is needed. This region is attended to with an exponentially decreasing guidance weight. Finally, the last s = 5 actions are beyond the end of the previous chunk, and need to be freshly generated. The execution horizon, s , is a hyperparameter constrained by d ≤ s ≤ H -d .
Figure 3 A diagram illustrating how action generation attends to the previous action chunk in real-time chunking. If inference starts after the execution of a -1 and the inference delay is d = 4 , then the newly generated chunk will not be available until after a 3 is consumed. Therefore, a 0:3 are 'frozen' and are attended to with a full guidance weight of 1. In the intermediate region, a 4:10 , actions from the previous chunk are available but may be updated, since inference will have finished before a 4 is needed. This region is attended to with an exponentially decreasing guidance weight. Finally, the last s = 5 actions are beyond the end of the previous chunk, and need to be freshly generated. The execution horizon, s , is a hyperparameter constrained by d ≤ s ≤ H -d .From the PDF
Figure 4 A comparison of naive inpainting (hard masking) and our proposed soft masking method: note that hard masking does not match the frozen region very well and produces faster changes in direction.
Figure 4 A comparison of naive inpainting (hard masking) and our proposed soft masking method: note that hard masking does not match the frozen region very well and produces faster changes in direction.From the PDF
Figure 5 Top left: Kinetix environments; each involves getting a green object on the left to touch a blue one on the right. Bottom left: Execution horizon vs. solve rate with a fixed inference delay of 1. Only RTC and BID take full advantage of faster updates, showing strictly increasing performance with decreasing execution horizon. Right: Inference delay vs. solve rate with a fixed execution horizon of s = max( d, 1) . RTC outperforms all baselines. Furthermore, soft masking (Sec. 3.2) improves performance at lower inference delays and execution horizons. Each data point represents 2048 trials, and 95% Wilson score intervals are shaded in.
Figure 5 Top left: Kinetix environments; each involves getting a green object on the left to touch a blue one on the right. Bottom left: Execution horizon vs. solve rate with a fixed inference delay of 1. Only RTC and BID take full advantage of faster updates, showing strictly increasing performance with decreasing execution horizon. Right: Inference delay vs. solve rate with a fixed execution horizon of s = max( d, 1) . RTC outperforms all baselines. Furthermore, soft masking (Sec. 3.2) improves performance at lower inference delays and execution horizons. Each data point represents 2048 trials, and 95% Wilson score intervals are shaded in.From the PDF
Figure 6 Top: Controller steps (equivalent to elapsed time with inference pauses removed multiplied by 50Hz) vs. cumulative progress for each task, aggregated across all delays. Progress is measured in discrete steps corresponding to the subsections of each task. Left: Time (including inference pauses) vs. cumulative progress aggregated across all tasks. The x-axis is log scale to better show progress during both short and long-horizon tasks. Right: Inference delay vs. average throughput, defined as the proportion of task completed divided by duration of episode averaged over episodes. Error bars are ± 1 SEM. Average throughput gives a balanced view of both speed and performance for each method. Neither TE variant can run at +100 or +200ms of injected latency, causing such high oscillations that the robot's protective stop is triggered.
Figure 6 Top: Controller steps (equivalent to elapsed time with inference pauses removed multiplied by 50Hz) vs. cumulative progress for each task, aggregated across all delays. Progress is measured in discrete steps corresponding to the subsections of each task. Left: Time (including inference pauses) vs. cumulative progress aggregated across all tasks. The x-axis is log scale to better show progress during both short and long-horizon tasks. Right: Inference delay vs. average throughput, defined as the proportion of task completed divided by duration of episode averaged over episodes. Error bars are ± 1 SEM. Average throughput gives a balanced view of both speed and performance for each method. Neither TE variant can run at +100 or +200ms of injected latency, causing such high oscillations that the robot's protective stop is triggered.From the PDF
Figure 7 Top left: The graph of the value 1 -τ τ · r 2 τ from Eq. 2, which we clip at β . At τ = 0 , clipping is needed to make the value finite. With 5 denoising steps, if β ≥ 4 . 25 , the clipping only determines the guidance weight for the first step ( τ = 0 ). Top right: An ablation of β in our simulated benchmark. Increasing β provides no marginal benefit beyond β = 5 . Bottom left: Example real robot action chunks generated from the same noise with 5 denoising steps ( n = 5 ) and 100 denoising steps ( n = 100 ), with lower opacities corresponding to higher guidance weight clipping ( β = { 5 , 20 , 50 , 150 } ). With 5 denoising steps, the generated action chunks diverge when β is too high. Bottom right: β vs. maximum acceleration (second discrete difference) for a batch of 325 action chunks generated with d = 15 and n = 5 . Higher β leads to more jerkiness, a proxy for out-of-distribution actions.
Figure 7 Top left: The graph of the value 1 -τ τ · r 2 τ from Eq. 2, which we clip at β . At τ = 0 , clipping is needed to make the value finite. With 5 denoising steps, if β ≥ 4 . 25 , the clipping only determines the guidance weight for the first step ( τ = 0 ). Top right: An ablation of β in our simulated benchmark. Increasing β provides no marginal benefit beyond β = 5 . Bottom left: Example real robot action chunks generated from the same noise with 5 denoising steps ( n = 5 ) and 100 denoising steps ( n = 100 ), with lower opacities corresponding to higher guidance weight clipping ( β = { 5 , 20 , 50 , 150 } ). With 5 denoising steps, the generated action chunks diverge when β is too high. Bottom right: β vs. maximum acceleration (second discrete difference) for a batch of 325 action chunks generated with d = 15 and n = 5 . Higher β leads to more jerkiness, a proxy for out-of-distribution actions.From the PDF
Figure 8 Left: Simulated ablation over different schedules for soft masking weights (Eq. 5). Exponential decay performs the best overall, although linear decay is very close behind. Right: Comparison with the inpainting algorithm from Diffuser [26], which overwrites a portion of the action chunk with the desired actions at each denoising step. While this simpler (and cheaper) inpainting method still provides some benefit, it is outperformed by our guidance-based approach.
Figure 8 Left: Simulated ablation over different schedules for soft masking weights (Eq. 5). Exponential decay performs the best overall, although linear decay is very close behind. Right: Comparison with the inpainting algorithm from Diffuser [26], which overwrites a portion of the action chunk with the desired actions at each denoising step. While this simpler (and cheaper) inpainting method still provides some benefit, it is outperformed by our guidance-based approach.From the PDF

Verification

Claims the paper makes

14 Valid · 7 Not valid · 4 Not checked · 76 Own results and interpretations

42 more claims

Create a free account to see all 101 claims and to verify the 9 references still open.

See all claims — free account
  • Probably not a substantial claim. 33
  • It may state two separate claims as one. 25
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 6

  1. [2][2] Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: a survey. Journal of machine learning research , 18(153):1-43, 2018. -
  2. [8][8] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-languageaction models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 , 2023. - · arXiv
  3. [14][14] Donald R. Coughanowr and Steven E. LeBlanc. Process Systems Analysis and Control , chapter 18. McGraw-Hill, New York, 3rd edition, 2009. ISBN 978-0073397894. -
  4. [22][22] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020. -
  5. [44][44] Atharva Mete, Haotian Xue, Albert Wilcox, Yongxin Chen, and Animesh Garg. Quest: Selfsupervised skill abstractions for learning continuous control, 2024. URL https://arxiv. org/abs/2407.15840 . -
  6. [55][55] Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations , 2023. -

Cited by

See all 207

Metadata

Authors
Black, K., Galliker, M., & Levine, S.
Affiliations
  • Physical Intelligence
  • University of California, Berkeley (US)
Venue
Advances in Neural Information Processing Systems 38
Published
2025
Pages
37596-37620
ISSN
1049-5258 (ISSN-L)
OpenAlex
W4417097926
Cited by
12
References
0
Citations by year
2026: 12
FWCI
26.12
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
action chunking, real-time execution, vision-language-action models, flow-based generative models, diffusion models, inference latency, bimanual manipulation, temporal coherence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Real-Time Execution of Action Chunking Flow Policies. https://app.cochat.ai/papers/10.52202/085713-1122
The paper
Black, K., Galliker, M., & Levine, S. (2025). Real-Time Execution of Action Chunking Flow Policies. Advances in Neural Information Processing Systems 38. https://doi.org/10.52202/085713-1122

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.