Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Abstract
Vision-language-action (VLA) models hold promise as generalist robotics solutions by translating visual and linguistic inputs into robot actions, yet they lack reliability due to their black-box nature and sensitivity to environmental changes. In contrast, cognitive architectures (CA) excel in symbolic reasoning and state monitoring but are constrained by rigid predefined execution. This work bridges these approaches by probing OpenVLA's hidden layers to uncover symbolic representations of object properties, relations, and action states, enabling integration with a CA for enhanced interpretability and robustness. Through experiments on LIBERO-spatial pick-and-place tasks, we analyze the encoding of symbolic states across different layers of OpenVLA's Llama backbone. Our probing results show consistently high accuracies$(>0.90)$for both object and action states across most layers, though contrary to our hypotheses, we did not observe the expected pattern of object states being encoded earlier than action states. We demonstrate an integrated DIARCOpenVLA system that leverages these symbolic representations for real-time state monitoring, laying the foundation for more interpretable and reliable robotic manipulation.
Overview
CoChat Summary
TL;DR
This paper probes OpenVLA’s hidden layers to test whether symbolic object and action states can be decoded from its activations, then integrates the best-performing probes into the DIARC cognitive architecture for real-time symbolic monitoring. It finds high probe accuracies across most layers but does not confirm the hypothesized layer-wise ordering of object versus action state encoding.
Key findings
- Linear probes trained on OpenVLA’s 33 hidden layers achieve accuracies above 0.90 for most layers across object relation, object property, action status, and action subgoal categories.
- Layer 0 probes perform significantly worse across all categories, which the authors attribute to early layers encoding low-level features rather than high-level visual-semantic relations.
- Contrary to H1 and H2, the authors did not observe object states encoded earlier than action states; they suggest limited variation in the LIBERO-spatial training data may have washed out the expected layer-wise difference.
- The object relation `on-table(tabletop-object)` and object property `turned-on(on-off-object)` were dropped from training due to low variance, since most objects remain on-table and the only on-off object remains off.
Methods
- The authors extract activations from OpenVLA’s Llama 2 7B backbone, which has 32 transformer blocks plus an initial embedding layer, yielding 33 hidden states indexed 0–32, each a 4096-dimensional vector.
- They use LIBERO-spatial, a suite of 10 pick-and-place tasks, and collect 5 successful episodes per task by repeatedly querying OpenVLA; hidden-layer embeddings are paired with same-timestep ground-truth symbolic states.
- They train 2 × 33 = 66 linear probes for multi-label prediction of object states and action states using binary cross-entropy loss with Adam; object states have 224 atoms and action states have 12 atoms. Preprocessing includes episode-level train/test splitting, filtering labels with frequency below 1% or above 99%, no class balancing, and no feature standardization.
Limitations
- The authors state the training data are not diverse enough: object placements are the same across the 10 LIBERO-spatial tasks except for the two black bowls, the robot always picks one of the two black bowls, and the place target is always the plate, reducing variation in object and action states and potentially inflating probe accuracies.
- The authors state that more and better data are needed to test the hypotheses, and they plan future work with variable objects, layouts, and goals.
- Evident from the text: the study is limited to simulated LIBERO-spatial pick-and-place tasks and a single OpenVLA checkpoint, so generalization to other tasks, environments, or real-world manipulation is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 9 Not checked · 22 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our probing results show consistently high accuracies ( > 0 . 90 ) for both object and action states across most layers”
- Also stated in: § V. RESULTS AND DISCUSSION · p. 7 · hedged
“The accuracies are above 0.90 for most layers, indicating that the OpenVLA indeed encodes some object relation, object property, action status, and action subgoal features.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § V. RESULTS AND DISCUSSION · p. 7 · hedged
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, the objects in the 10 simulated LIBERO-spatial tasks have the same placements across tasks except for the two black bowls.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The layer 0 probes perform significantly worse across all categories.”
- Also stated in: § C. Probe Training and Evaluation · p. 7 · plain
“However, the first layer shows notably lower performance, aligning with its expected role in encoding lower-level features rather than high-level semantic relationships.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § C. Probe Training and Evaluation · p. 7 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate an integrated DIARCOpenVLA system that leverages these symbolic representations for real-time state monitoring, laying the foundation for more interpretable and reliable robotic manipulation.”
- Also stated in: § A. DIARC-OpenVLA Integration · p. 2 · plain
“The DIARC-OpenVLA integration bridges OpenVLA's continuous policy outputs and hidden-layer embeddings with DIARC's symbolic reasoning modules.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § A. DIARC-OpenVLA Integration · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“though contrary to our hypotheses, we did not observe the expected pattern of object states being encoded earlier than action states.”
- Also stated in: § V. RESULTS AND DISCUSSION · p. 7 · plain
“We do not observe the hypothesized pattern of higher object state accuracies in earlier layer probes versus that of later layer probes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § V. RESULTS AND DISCUSSION · p. 7 · plain
4 more claims
Create a free account to see all 31 claims and to verify the 9 references still open.
- It may state two separate claims as one. 15
- Probably not a substantial claim. 12
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ V. RESULTS AND DISCUSSION · p. 7
“therefore their corresponding atoms never change truth values.”
§ A. Probe Training Data Collection · p. 5
“In frame 3, the on-table(bowl 1) object relation becomes false as the bowl is lifted off the tabletop.”
Citations
Cites
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · cited 1×
“One commonly used approach to evaluate LLMs is prompt-based probing in which an LLM is prompted to fill in the blanks in the prompt [12].”
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., & Stone, P. (2023) · arXiv · doi:10.48550/arxiv.2306.03310 · Cited by 135
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., & Stone, P. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1939 · Cited by 135
Implicit Representations of Meaning in Neural Language Models
Li, B. Z., Nye, M., & Andreas, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.143 · Cited by 84 · cited 3×
“For example, Li et al. train semantic probes to predict object properties and relations as they evolve throughout a discourse [16].”
Prompting as Probing: Using Language Models for Knowledge Base Construction
Alivanistos, D., Santamaría, S. B., Cochez, M., Kalo, J.-C., van Krieken, E., & Thanapalasingam, T. (2022) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2208.11057 · Cited by 74
(2018) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-97550-4 · Cited by 72
An Overview of the Distributed Integrated Cognition Affect and Reflection DIARC Architecture
Scheutz, M., Williams, T., Krause, E., Oosterveld, B., Sarathy, V., & Frasca, T. (2019) · Intelligent Systems, Control and Automation: Science and Engineering · doi:10.1007/978-3-319-97550-4_11 · Cited by 69 · cited 1×
“On the other hand, traditional Cognitive Architectures (CA), also known as symbolic architectures, excel in dependable, symbol-based reasoning but are constrained by their reliance on predefined rules and coded policy execution [4], [5].”
Self-citationOpenVLA: An Open-Source Vision-Language-Action Model
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.09246 · Cited by 62
Research works printed but not linked: 3
- [4] V. N. Gudivada, 'Chapter 1 -cognitive computing: Concepts, architectures, systems, and applications,' in Handbook of Statistics , ser. Cognitive Computing: Theory and Applications, V. N. Gudivada, V. V. Raghavan, V. Govindaraju, and C. R. Rao, Eds. Elsevier, vol. 35, pp. 3-38. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/S0169716116300451 -
- [9] G. Bajaj, K. Pearce, S. Kennedy, O. Larue, A. Hough, J. King, C. Myers, and S. Parthasarathy, 'Generating chunks for cognitive architectures,' vol. 2, no. 1, pp. 246-252, number: 1. [Online]. Available: https://ojs.aaai.org/index.php/AAAI-SS/article/view/27683 -
- [15] S. Qi, Z. Cao, J. Rao, L. Wang, J. Xiao, and X. Wang, 'What is the limitation of multimodal LLMs? a deeper look into multimodal LLMs through prompt probing,' vol. 60, no. 6, p. 103510. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306457323002479 -
Cited by
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., & Zhu, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3609980 · Cited by 255
Context-Aware Small Language Models for Real Time Strategy Prediction in Strategy Game
Admajindra, C. H., & Haryanto, H. (2025) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis68501.2025.11327013
“Benchmark environments in the gaming community, such as robot soccer simulations or industrial assembly-inspired scenarios, typically evaluate one action per timestep rather than sets of actions occurring concurrently [7].”
Li, Z., Li, L., & Kang, C. (2026) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac71005.2026.11615505
Metadata
- Authors
- Lu, H., Li, H., Shahani, P. S., Herbers, S., & Scheutz, M.
- Affiliations
- Tufts University (US)
- Venue
- 2025 IEEE International Conference on AI and Data Analytics (ICAD)
- Publisher
- IEEE
- Published
- 2025-06-24
- Pages
- 1-8
- OpenAlex
- W4413361394
- Cited by
- 13
- References
- 3
- Citations by year
- 2026: 8 · 2025: 5
- FWCI
- 14.33
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE International Conference on AI and Data Analytics (ICAD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- vision-language-action models, OpenVLA, cognitive architecture, symbolic state representation, interpretability, LIBERO, pick-and-place, robotic manipulation, state monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture. https://app.cochat.ai/papers/10.1109/icad65464.2025.11114062
- The paper
- Lu, H., Li, H., Shahani, P. S., Herbers, S., & Scheutz, M. (2025). Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture. 2025 IEEE International Conference on AI and Data Analytics (ICAD). https://doi.org/10.1109/icad65464.2025.11114062
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.