Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

It may state two separate claims as one.

Parallel decoding with action chunking boosts inference efficiency, improves success rates on downstream tasks, and enables greater flexibility in model input-output specifications.

“Our study reveals several key insights that build on each other: (1) parallel decoding with action chunking not only boosts inference efficiency but also improves success rates on downstream tasks while enabling greater flexibility in the model's input-output specifications;”

• AssumptionRests on an assumption: The three design components investigated are key for vision-language-action fine-tuning.

It may state two separate claims as one.

Policies fine-tuned with a simpler L1 regression objective can match more complex approaches in task performance while achieving significantly greater inference efficiency.

“Through controlled experiments, we show that policies fine-tuned with a simpler L1 regression objective can match more complex approaches in task performance while achieving significantly greater inference efficiency.”

It may state two separate claims as one.

The enhanced fine-tuned OpenVLA policies outperformed the best fine-tuned π0 policies as well as MDT and Seer policies.

“Notably, our enhanced ne-tuned OpenVLA policies outperform even the best ne-tuned 0 policies [3, 38] which bene t from a base model with larger-scale pretraining and a more sophisticated learning objective ( ow matching [25]) as well as Multimodal Diffusion Transformer (MDT) [40] and Seer [50] policies.”

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [38]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [40]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [50]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: For satisfactory deployment, robot policies must demonstrate reliable task execution.

It may state two separate claims as one.

In LIBERO, OpenVLA-OFT achieves 97.1% average success across four task suites, outperforming fine-tuned OpenVLA at 76.5% and π0 at 94.2% while achieving a 26 times speedup in action generation with 8-step action chunks.

“In LIBERO, OpenVLA-OFT establishes a new state of the art by achieving 97.1% average success rate across four task suites, outperforming both fine-tuned OpenVLA policies [23] (76.5%) and π 0 policies [3] (94.2%) while achieving a 26× speedup in action generation with 8-step action chunks.”

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: The LIBERO simulation benchmark and the ALOHA tasks are suitable for evaluating vision-language-action fine-tuning recipes.

It may state two separate claims as one.

The primary diffusion variant required 50 denoising steps and therefore had high latency.

“The primary diffusion variant requires 50 denoising steps (since this is the number of steps speci ed at train time) and thus suffers from high latency.”

• AssumptionRests on an assumption: Efficient inference is crucial for deploying vision-language-action models on high-frequency control robots.

It may state two separate claims as one.

Parallel decoding reduced latency and increased throughput by a factor of four by replacing seven sequential decoder forward passes with one pass.

“Results in Table II show that parallel decoding reduces latency and increases throughput by 4 by replacing 7 sequential forward passes through the decoder portion of the policy with a single pass.”

• AssumptionRests on an assumption: Efficient inference is crucial for deploying vision-language-action models on high-frequency control robots.

It may state two separate claims as one.

OpenVLA-OFT sets a new state of the art on LIBERO, raising OpenVLA's average success across four task suites from 76.5% to 97.1% and increasing action generation throughput by 26 times.

“We propose OpenVLA-OFT, an instantiation of this recipe, which sets a new state of the art on the LIBERO simulation benchmark, significantly boosting OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26×.”

• AssumptionRests on an assumption: The LIBERO simulation benchmark and the ALOHA tasks are suitable for evaluating vision-language-action fine-tuning recipes.

It may state two separate claims as one.

In real-world evaluations, the fine-tuning recipe enables OpenVLA to execute dexterous high-frequency control tasks on a bimanual ALOHA robot and outperform other fine-tuned vision-language-action models and strong imitation learning policies trained from scratch by up to 15 percentage points absolute in average success rate.

“OpenVLA-OFT+ successfully executes dexterous bimanual tasks like folding clothes and manipulating target food items based on the user's prompt (see Figure 1), outperforming both fine-tuned VLAs (π 0 and RDT-1B) and prominent imitation learning policies trained from scratch (Diffusion Policy and ACT) by up to 15% (absolute) in average success rate.”

• AssumptionRests on an assumption: The LIBERO simulation benchmark and the ALOHA tasks are suitable for evaluating vision-language-action fine-tuning recipes.

It may state two separate claims as one.

Parallel decoding paired with action chunking achieves 26 to 43 times throughput with much lower latency, 0.07 ms for single-arm tasks with one input image and 0.321 ms for bimanual tasks with three input images.

“Our parallel decoding approach, when paired with action chunking, achieves significantly greater speedups: 26× to 43× throughput with much lower latency (0.07 ms for single-arm tasks with one input image and 0.321 ms for bimanual tasks with three input images).”

It may state two separate claims as one.

Fine-tuning a vision-language-action model with an L1 regression objective yields performance comparable to diffusion-based fine-tuning while offering faster training convergence and inference speed.

“(3) fine-tuning the VLA with an L1 regression objective yields comparable performance to diffusion-based fine-tuning while offering faster training convergence and inference speed.”

• AssumptionRests on an assumption: The three design components investigated are key for vision-language-action fine-tuning.

It may state two separate claims as one.

Despite the longer input sequence, the fine-tuned OpenVLA policy maintained high throughput of 71.4 Hz and low latency of 0.112 seconds.

“Despite this substantial increase in input sequence length, the ne-tuned OpenVLA policy maintains high throughput (71.4 Hz) and low latency (0.112 sec), as shown in Table II.”

• AssumptionRests on an assumption: Efficient inference is crucial for deploying vision-language-action models on high-frequency control robots.

It may state two separate claims as one.

Executing full action chunks before replanning improved both speed and performance.

“For methods using action chunking, we set chunk size to K = 8 to match the Diffusion Policy baseline [5], and execute full chunks before replanning, which we nd improves both speed and performance.”

• cites [5]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

ACT produced less precise actions and achieved the lowest overall performance.

“ACT, while able to complete basic tasks, produces less precise actions and achieves the lowest overall performance.”

• AssumptionRests on an assumption: For satisfactory deployment, robot policies must demonstrate reliable task execution.
• AssumptionRests on an assumption: The four ALOHA tasks are representative of dexterous, bimanual manipulation tasks.

It may state two separate claims as one.

Fine-tuned VLA policies generally outperformed from-scratch baselines in both task execution and language following.

“Fine-tuned VLA policies generally outperform the from-scratch baselines in both task execution and language following, consistent with prior ndings [27,3].”

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: For satisfactory deployment, robot policies must demonstrate reliable task execution.
• AssumptionRests on an assumption: The four ALOHA tasks are representative of dexterous, bimanual manipulation tasks.

It may state two separate claims as one.

For chunk size K, parallel decoding predicts KD actions in one forward pass, increasing throughput K-fold with minimal latency impact.

“For chunk size K, the model predicts KD actions in one forward pass, increasing throughput K-fold with minimal latency impact.”

It may state two separate claims as one.

L1 regression with an MLP action head maintains the efficiency benefits of parallel decoding while potentially improving action precision.

“The model is trained to minimize the mean L1 difference between predicted and ground-truth actions, maintaining the efficiency benefits of parallel decoding while potentially improving action precision.”

It may state two separate claims as one.

The new fine-tuning recipe enables real-time robot control with strong task performance and language following ability.

“With 25-timestep action chunks, OpenVLA-OFT+ achieves 43× faster throughput than base OpenVLA, demonstrating that our new fine-tuning recipe enables real-time robot control with strong task performance and language following ability.”

• AssumptionRests on an assumption: The LIBERO simulation benchmark and the ALOHA tasks are suitable for evaluating vision-language-action fine-tuning recipes.

It may state two separate claims as one.

Language inputs may only be critical at specific moments in a task, making it challenging to train the model to focus on language inputs without special techniques.

“Furthermore, language inputs may only be critical at specific moments in a task-for example, after grasping the spoon and deciding which ingredient to scoop in the”

It may state two separate claims as one.

The use of action chunking leads to smoother and more reliable task execution.

“This improvement is particularly pronounced in LIBERO-Long, suggesting that action chunking helps capture temporal dependencies [28] and reduce compounding errors [41], which ultimately leads to smoother and more reliable task execution.”

Probably not a substantial claim.

OpenVLA is a 7B-parameter manipulation policy created by fine-tuning the Prismatic vision-language model on 1M episodes from the Open X-Embodiment dataset.

“We use OpenVLA [23] as our representative base VLA, a 7B-parameter manipulation policy created by fine-tuning the Prismatic VLM [21] on 1M episodes from the Open X-Embodiment dataset [34].”

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [21]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [34]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Conditional denoising diffusion modeling requires multiple forward passes during inference, 50 diffusion steps in this implementation, impacting deployment latency even with parallel decoding.

“While this approach offers potentially more expressive action modeling, it requires multiple forward passes during inference (50 diffusion steps in our implementation), impacting deployment latency even with parallel decoding.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

LoRA fine-tuning enables effective adaptation of vision-language-action models to single-arm robots operating at low control frequencies below 10 Hz.

“While Kim et al. [23] study various parameter update strategies and from their findings show that LoRA finetuning enables effective adaptation to single-arm robots operating at low control frequencies (< 10 Hz), their analysis does not extend to bimanual robots with high control frequencies (25-50+ Hz), a more complex control scenario.”

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Existing approaches that fine-tune vision-language-action models using the base autoregressive training recipe face slow inference speed of 3-5 Hz unsuitable for high-frequency control and unreliable task execution on bimanual manipulators.

“Existing approaches that fine-tune VLAs using the base model's autoregressive training recipe face two key limitations: slow inference speed (3-5 Hz) unsuitable for high-frequency control, and unreliable task execution on bimanual manipulators [54,27,3].”

• cites [54]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: High-frequency bimanual robot control requires control frequencies of 25-50+ Hz.
• AssumptionRests on an assumption: The three design components investigated are key for vision-language-action fine-tuning.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

OpenVLA's original training formulation uses autoregressive prediction of 7 discrete robot action tokens per timestep: 3 for position control, 3 for orientation control, and 1 for gripper control.

“OpenVLA's original training formulation uses autoregressive prediction of 7 discrete robot action tokens per timestep: 3 for position control, 3 for orientation control, and 1 for gripper control.”

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Parallel decoding allows OpenVLA to generate action chunks with only a small latency increase, which enhances flexibility in model outputs.

“As explained in Section IV-B and validated by our ef ciency evaluations in the prior section, parallel decoding enables OpenVLA to generate action chunks with minimal latency increase, thereby enhancing exibility in model outputs .”

• AssumptionRests on an assumption: Efficient inference is crucial for deploying vision-language-action models on high-frequency control robots.

It may state two separate claims as one.

For chunk size K and action dimensionality D, OpenVLA requires KD sequential decoder forward passes versus D without chunking, causing a K-fold latency increase that makes action chunking impractical for high-frequency robots under the original formulation.

“For a chunk size of K timesteps and action dimensionality D, OpenVLA requires KD sequential decoder forward passes versus just D passes without chunking.This K-fold increase in latency makes action chunking impractical for high-frequency robots under the original formulation.”

It may state two separate claims as one.

Feature-wise linear modulation is used only for the ALOHA experiments because multiple camera viewpoints lead to a larger presence of spurious correlations in visual inputs.

“We only use FiLM for the ALOHA experiments discussed in Section VI, where multiple camera viewpoints lead to a larger presence of spurious correlations in visual inputs.”

• AssumptionRests on an assumption: The three design components investigated are key for vision-language-action fine-tuning.

Probably not a substantial claim. It may state two separate claims as one.

Diffusion-based vision-language-action models achieve higher action throughput than autoregressive models by generating multi-timestep action chunks simultaneously, but introduce computational trade-offs through slower training and multiple denoising or integration steps at inference.

“While these diffusion-based VLAs achieve higher action throughput than autoregressive VLAs by generating multitimestep action chunks simultaneously, they introduce computational trade-offs through slower training and multiple denoising or integration steps at inference time.”

Probably not a substantial claim. It may state two separate claims as one.

Parallel decoding enables the model to map input embeddings to the predicted output sequence in a single forward pass, unlike autoregressive generation which requires sequential token prediction.

“Unlike autoregressive generation which requires sequential token prediction, parallel decoding enables the model to map input embeddings to the predicted output sequence in a single forward pass.”

Probably not a substantial claim.

The unified latent representation enables the model to attend to all available information when generating actions.

“This unified latent representation enables the model to attend to all available information when generating actions.”

It may state two separate claims as one.

Action chunking helps the policy capture temporal dependencies and reduces compounding errors.

“This improvement is particularly pronounced in LIBERO-Long, suggesting that action chunking helps capture temporal dependencies [28] and reduce compounding errors [41], which ultimately leads to smoother and more reliable task execution.”

• cites [28]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [41]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

The Optimized Fine-Tuning recipe combines parallel decoding, action chunking, continuous action representation, and L1 regression to improve inference efficiency, policy performance, and input-output flexibility.

“Our empirical analysis informs an Optimized Fine-Tuning (OFT) recipe that integrates parallel decoding, action chunking, a continuous action representation, and a simple L1 regression-based learning objective to altogether improve inference efficiency, policy performance, and flexibility in the model's input-output specifications.”

• AssumptionRests on an assumption: OpenVLA is a representative base vision-language-action model for studying adaptation design decisions.
• AssumptionRests on an assumption: The three design components investigated are key for vision-language-action fine-tuning.

It may state two separate claims as one.

Both LoRA and full fine-tuning of autoregressive vision-language-action models often perform unsatisfactorily on bimanual manipulation tasks.

“However, their autoregressive action generation remains too slow (3-5 Hz) for high-frequency control (25-50+ Hz), and both LoRA and full fine-tuning of autoregressive VLAs often yield unsatisfactory performance in bimanual manipulation tasks [54,27,3].”

• cites [54]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

The paper's approach fine-tunes the base vision-language-action model end-to-end with techniques that enable high efficiency without needing a separate controller.

“Lastly, unlike prior work that relies on a separate faster low-level control policy in addition to a slower VLA [36], in this work we fine-tune the base VLA end-to-end with techniques that enable high efficiency without needing a separate controller.”

• cites [36]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim.

Action chunking, meaning predicting and executing a sequence of future actions without intermediate replanning, improves policy success rates across many manipulation tasks.

“Prior works have shown that action chunking-i.e., predicting and executing a sequence of future actions without intermediate replanning-improves policy success rates across many manipulation tasks [56,5,28].”

• cites [56]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [5]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [28]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

OpenVLA employs next-token prediction with cross-entropy loss as its learning objective.

“It employs next-token prediction with cross-entropy loss as its learning objective, similar to language models.”

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Even though the base model was simpler and pretrained on less data, the alternative VLA adaptation design decisions allowed fine-tuned OpenVLA policies to achieve a new state of the art on LIBERO.

“Even with a simpler base model pretrained on less data than more recent VLAs, we nd that our alternative VLA adaptation design decisions empower ne-tuned OpenVLA policies to establish a new state of the art on the LIBERO benchmark.”

• AssumptionRests on an assumption: For satisfactory deployment, robot policies must demonstrate reliable task execution.

It may state two separate claims as one.

The ALOHA tasks are novel dexterous manipulation tasks that OpenVLA had never encountered during pretraining, which used only single-arm robot data.

“We evaluate on novel dexterous manipulation tasks that have never been encountered before during OpenVLA's pretraining (which only involves singlearm robot data).”

It may state two separate claims as one.

These design choices work together to produce strong policies that can be deployed at high frequencies while keeping the algorithm simple.

“These design choices work together to produce strong policies that can be deployed at high frequencies while maintaining algorithmic simplicity.”

It may state two separate claims as one.

The ALOHA setup differs substantially from OpenVLA's pretraining in control frequency, action space, and input modalities.

“This setup differs signi cantly from OpenVLA's pretraining, which includes single-arm robot data only, a single camera viewpoint from a third-person camera, no robot state inputs, low-frequency control (3-10 Hz), and relative end-effector pose actions.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

OpenVLA results from this work were averaged over 500 trials per task suite, with 10 tasks and 50 episodes each.

“OpenVLA results from this work are averaged over 500 trials for each task suite (10 tasks 50 episodes).”

It may state two separate claims as one.

The Optimized Fine-Tuning (OFT) recipe combines three components: parallel decoding with action chunking, continuous action representation, and an L1 regression objective.

“Based on the demonstrated improvements in task performance, inference ef ciency, and model input-output exibility, we propose anOptimized Fine-Tuning (OFT) recipe for VLA adaptation that combines three key components: 1) parallel decoding with action chunking 2) continuous action representation 3) L1 regression objective”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The evaluation used a predetermined rubric that assigned scores for partial task completion.

“To provide ne-grained assessment, we use a predetermined rubric that assigns scores for partial task completion (see Appendix F for details).”

• AssumptionRests on an assumption: The predetermined rubric validly scores partial task completion.

Probably not a substantial claim. The paper may give no reason why it holds.

The alternative fine-tuning formulation, together with additional model inputs and outputs, creates a distribution shift between the base VLA's pretraining and fine-tuning.

“Given that the alternative ne-tuning formulation, along with additional model inputs and outputs, induces a distribution shift between the base VLA's pretraining and netuning, one reasonable question is whether the base VLA's pretrained representations are helpful and have any in uence on the results we have reported.”