CoChat

Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture

Open access
Lu, Hong, Li, Hengxu, Shahani, Prithviraj Singh, Herbers, Stephanie, Scheutz, Matthias · 2025 · 2025 IEEE International Conference on AI and Data Analytics (ICAD)

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.

Abstract

Vision-language-action (VLA) models hold promise as generalist robotics solutions by translating visual and linguistic inputs into robot actions, yet they lack reliability due to their black-box nature and sensitivity to environmental changes. In contrast, cognitive architectures (CA) excel in symbolic reasoning and state monitoring but are constrained by rigid predefined execution. This work bridges these approaches by probing OpenVLA's hidden layers to uncover symbolic representations of object properties, relations, and action states, enabling integration with a CA for enhanced interpretability and robustness. Through experiments on LIBERO-spatial pick-and-place tasks, we analyze the encoding of symbolic states across different layers of OpenVLA's Llama backbone. Our probing results show consistently high accuracies$(>0.90)$for both object and action states across most layers, though contrary to our hypotheses, we did not observe the expected pattern of object states being encoded earlier than action states. We demonstrate an integrated DIARCOpenVLA system that leverages these symbolic representations for real-time state monitoring, laying the foundation for more interpretable and reliable robotic manipulation.

Overview

CoChat Summary

TL;DR

This paper probes OpenVLA’s hidden layers to test whether symbolic object and action states can be decoded from its activations, then integrates the best-performing probes into the DIARC cognitive architecture for real-time symbolic monitoring. It finds high probe accuracies across most layers but does not confirm the hypothesized layer-wise ordering of object versus action state encoding.

Key findings

  • Linear probes trained on OpenVLA’s 33 hidden layers achieve accuracies above 0.90 for most layers across object relation, object property, action status, and action subgoal categories.
  • Layer 0 probes perform significantly worse across all categories, which the authors attribute to early layers encoding low-level features rather than high-level visual-semantic relations.
  • Contrary to H1 and H2, the authors did not observe object states encoded earlier than action states; they suggest limited variation in the LIBERO-spatial training data may have washed out the expected layer-wise difference.
  • The object relation `on-table(tabletop-object)` and object property `turned-on(on-off-object)` were dropped from training due to low variance, since most objects remain on-table and the only on-off object remains off.

Methods

  • The authors extract activations from OpenVLA’s Llama 2 7B backbone, which has 32 transformer blocks plus an initial embedding layer, yielding 33 hidden states indexed 0–32, each a 4096-dimensional vector.
  • They use LIBERO-spatial, a suite of 10 pick-and-place tasks, and collect 5 successful episodes per task by repeatedly querying OpenVLA; hidden-layer embeddings are paired with same-timestep ground-truth symbolic states.
  • They train 2 × 33 = 66 linear probes for multi-label prediction of object states and action states using binary cross-entropy loss with Adam; object states have 224 atoms and action states have 12 atoms. Preprocessing includes episode-level train/test splitting, filtering labels with frequency below 1% or above 99%, no class balancing, and no feature standardization.

Limitations

  • The authors state the training data are not diverse enough: object placements are the same across the 10 LIBERO-spatial tasks except for the two black bowls, the robot always picks one of the two black bowls, and the place target is always the plate, reducing variation in object and action states and potentially inflating probe accuracies.
  • The authors state that more and better data are needed to test the hypotheses, and they plan future work with variable objects, layouts, and goals.
  • Evident from the text: the study is limited to simulated LIBERO-spatial pick-and-place tasks and a single OpenVLA checkpoint, so generalization to other tasks, environments, or real-world manipulation is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/icad65464.2025.11114062

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 9 Not checked · 22 Own results and interpretations

4 more claims

Create a free account to see all 31 claims and to verify the 9 references still open.

See all claims — free account
  • It may state two separate claims as one. 15
  • Probably not a substantial claim. 12

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 18 references

Research works printed but not linked: 3

  1. [4] V. N. Gudivada, 'Chapter 1 -cognitive computing: Concepts, architectures, systems, and applications,' in Handbook of Statistics , ser. Cognitive Computing: Theory and Applications, V. N. Gudivada, V. V. Raghavan, V. Govindaraju, and C. R. Rao, Eds. Elsevier, vol. 35, pp. 3-38. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/S0169716116300451 -
  2. [9] G. Bajaj, K. Pearce, S. Kennedy, O. Larue, A. Hough, J. King, C. Myers, and S. Parthasarathy, 'Generating chunks for cognitive architectures,' vol. 2, no. 1, pp. 246-252, number: 1. [Online]. Available: https://ojs.aaai.org/index.php/AAAI-SS/article/view/27683 -
  3. [15] S. Qi, Z. Cao, J. Rao, L. Wang, J. Xiao, and X. Wang, 'What is the limitation of multimodal LLMs? a deeper look into multimodal LLMs through prompt probing,' vol. 60, no. 6, p. 103510. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306457323002479 -

Cited by

See all 13

Metadata

Authors
Lu, H., Li, H., Shahani, P. S., Herbers, S., & Scheutz, M.
Affiliations
Tufts University (US)
Venue
2025 IEEE International Conference on AI and Data Analytics (ICAD)
Publisher
IEEE
Published
2025-06-24
Pages
1-8
OpenAlex
W4413361394
Cited by
13
References
3
Citations by year
2026: 8 · 2025: 5
FWCI
14.33
Type
conference paper
Language
en
Open access
Locations
  • IEEE International Conference on AI and Data Analytics (ICAD) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
vision-language-action models, OpenVLA, cognitive architecture, symbolic state representation, interpretability, LIBERO, pick-and-place, robotic manipulation, state monitoring

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture. https://app.cochat.ai/papers/10.1109/icad65464.2025.11114062
The paper
Lu, H., Li, H., Shahani, P. S., Herbers, S., & Scheutz, M. (2025). Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture. 2025 IEEE International Conference on AI and Data Analytics (ICAD). https://doi.org/10.1109/icad65464.2025.11114062

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.