E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

May combine two claims.

Traditional mesh-based representations often suffer from texture degradation and geometric oversmoothing, whereas 3DGS preserves the photometric fidelity needed for reliable viewpoint-dependent reasoning.

“Unlike traditional mesh-based representations that often suffer from texture degradation and geometric oversmoothing, 3DGS preserves the photometric fidelity necessary for reliable viewpoint-dependent reasoning.”

• cites [13] Ginting, M.F., et al.: Enter the mind palace: reasoning and planning for long-term active embodied question answering. In: Proceedings of the Conference on Robot Learning (CoRL) (2025) sourceCould not verify

  The Related Work section (p. 3) discusses dense 3D representations and scene graphs, while Mind Palace Generation (p. 4) describes viewpoint images stored in a scene graph. Neither passage compares meshes with 3DGS or addresses texture degradation, oversmoothing, or photometric fidelity. Searches for 3DGS, Gaussian splatting, and mesh rendering found no passage supporting the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

3D Gaussian Splatting provides photorealistic free-viewpoint rendering that maintains fine visual details such as small text and subtle attributes, which are often lost in mesh-based simulators.

“3D Gaussian Splatting enables photorealistic free-viewpoint rendering that preserves fine-grained visual details (e.g., small text and subtle attributes) often degraded in mesh-based simulators, thereby allowing the construction of questions that cannot be answered from a single view and instead require active inspection across viewpoints in 5-DoF.”

• Also stated in: § 1 Introduction · plain

  “3DGS enables photorealistic free-viewpoint rendering that preserves subtle visual details, such as fine-grained text and brand logos, which are often degraded in traditional mesh-based representations; importantly, this high-fidelity rendering makes it possible to define tasks where critical visual evidence is only observable from specific viewpoints, thereby necessitating active viewpoint control”

• cites [13] Ginting, M.F., et al.: Enter the mind palace: reasoning and planning for long-term active embodied question answering. In: Proceedings of the Conference on Robot Learning (CoRL) (2025) sourceCould not verify

  Section 4.1 (p. 4) describes storing images from multiple viewpoints, and Section 4.2 (p. 5) describes exploring those viewpoints to answer questions. Neither passage addresses 3D Gaussian Splatting, photorealistic rendering, preservation of small text or subtle attributes, or a comparison with mesh-based simulators. Searches for “Gaussian Splatting” and “mesh-based simulators” found no passage supporting that attributed claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

The dataset covers diverse scene categories and has a wide distribution of question and answer lengths.

“Fig. 4 shows that the dataset covers diverse scene categories and exhibits a wide distribution of question and answer lengths.”

May combine two claims.

After filtering, the dataset contains 99 scenes, 2,014 episodes derived from 1,290 unique question-answer pairs.

“After meticulous filtering, our dataset contains 99 scenes, 2,014 episodes derived from 1290 unique question-answer pairs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The human annotation process required approximately 1,120 hours of manual effort.

“The overall human annotation process required approximately 1,120 hours of manual effort.”

May combine two claims.

All evaluated state-of-the-art VLMs show a large performance gap compared to humans despite strong 2D reasoning, indicating limitations in active perception and coherent viewpoint planning under full 5-DoF viewpoint changes.

“Despite strong 2D reasoning ability, all models exhibit a substantial gap from humans, highlighting limitations in active perception and coherent viewpoint planning specifically under full 5-DoF viewpoint changes.”

May combine two claims.

The distribution of actions indicates that every action type is used, even those outside planar movement, which points to a need for 5-DoF active viewpoint control.

“In addition, the action distribution in Fig. 4(c) shows that all action types are well utilized, including those beyond planar navigation, suggesting the need for 5-DoF active viewpoint control.”

May combine two claims.

The upward movement action is used most often, and looking down is more common than looking up, which implies that useful visual evidence is frequently found by viewing targets from the side or from a slightly higher angle rather than from below.

“Moreover, move_up is the most frequent action, and look_down appears more often than look_up, suggesting that relevant visual evidence is often obtained by observing target objects from lateral or slightly elevated viewpoints rather than from below.”

May combine two claims.

The backward movement action occurs more often than forward movement, which suggests that moving away from the target to gain context is frequently needed.

“Notably, move_backward is more frequent than move_forward, indicating that gaining context by moving away from the target is often necessary.”

May combine two claims.

E3VS-Bench contains 99 high-fidelity 3D scenes reconstructed with 3D Gaussian Splatting and 2,014 question-driven episodes.

“E3VS-Bench consists of 99 high-fidelity 3D scenes reconstructed using 3D Gaussian Splatting and 2,014 question-driven episodes.”

• Also stated in: § 1 Introduction · plain

  “The benchmark contains 99 high-fidelity reconstructed scenes and 2,014 question-driven episodes.”

• Also stated in: § 3.3 Dataset Statistics and Analysis · plain

  “After meticulous filtering, our dataset contains 99 scenes, 2,014 episodes derived from 1290 unique question-answer pairs.”

Classified as probably not a substantial claim. May combine two claims.

The starting camera position is centered on the target object, yet other objects in the scene may partially or completely block it.

“The initial viewpoint v 0 is centered on the target object; however, the target may be partially or fully occluded by other objects in the scene.”

May combine two claims.

Gemini 3.0 Flash terminates efficiently, requiring only 6.60 steps on average, whereas Qwen3-VL-8B takes nearly twice as many steps (12.06).

“Gemini 3.0 Flash terminates efficiently, requiring only 6.60 steps on average, whereas Qwen3-VL-8B takes nearly twice as many steps (12.06), suggesting that Qwen3-VL-8B tends to continue exploring even when sufficient visual evidence is already available.”

May combine two claims.

The benchmark consists of 99 high-fidelity scenes and 2,014 episodes covering diverse reasoning types, including object search, spatial reasoning, attribute recognition, and counting.

“The benchmark consists of 99 high-fidelity scenes and 2,014 episodes covering diverse reasoning types, including object search, spatial reasoning, attribute recognition, and counting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Random Action baseline achieves an average score of 2.28.

“Random Action | 2.38 | 2.18 | 2.60 | 2.20 | 2.50 | 2.00 | 2.28 | 10.15 | | 0.39”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

Probably not a substantial claim: may not pass S3 (load-bearing).

Several models demonstrate strong 2D visual reasoning ability.

“While several models demonstrate strong 2D visual reasoning ability, all evaluated systems exhibit a substantial performance gap compared to human performance on E3VS.”

May combine two claims.

Gemini 3.0 models achieve relatively high performance on OA, while other models perform at or below the Random Action baseline.

“Empirically, while Gemini 3.0 models achieve relatively high performance on OA, other models perform at or below the Random Action baseline.”

May combine two claims.

More input frames do not significantly impact the VLM judge score, but consistently lead to more efficient navigation (fewer steps) and safer trajectories (lower collision rates).

“While more frames do not significantly impact the VLM judge score, they consistently lead to more efficient navigation (fewer steps) and safer trajectories (lower collision rates).”

May combine two claims.

Multi-frame inputs do not significantly improve VLM Judge scores, but consistently reduce navigation steps and collision rates.

“As shown in Fig. 6, multi-frame inputs do not significantly improve VLM Judge scores, but consistently reduce navigation steps and collision rates.”

May combine two claims.

Models already struggle in the 2D single-view setting, and performance further degrades in the full E3VS setting.

“Table 3 indicates that models already struggle in the 2D single-view setting, and performance further degrades in the full E3VS setting (Table 2).”

Not yet classified.

The VQA at Start performance of Qwen3-VL-8B is comparable to the E3VS performance of Gemini 3.0 Flash evaluated with GPT 5.1 as the judge.

“Empirically, we observe that the VQA at Start performance of Qwen3-VL-8B is comparable to the E3VS performance of Gemini 3.0 Flash evaluated with GPT 5.1 as the judge.”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

Gemini 3.0 Flash achieves the highest average score among evaluated models (2.79).

“Gemini 3.0 Flash | 3.21 | 2.82 | 3.18 | 3.53 | 2.75 | 1.88 | 2.79 | 11.29 | | 0.43”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

Probably not a substantial claim: may not pass S3 (load-bearing).

E3VS-Bench contains 2,014 episodes.

“E3VS-Bench | ✓ | 3D-GS SceneSplat++ | 6 DoF | 2,014”

May combine two claims.

Answers in E3VS-Bench are often longer and contain more descriptive expressions compared to OpenEQA.

“Compared to OpenEQA, where ground-truth answers typically consist of short phrases or single words, the answers in E3VS-Bench are often longer and contain more descriptive expressions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

VQA at Goal with GPT 5.1 achieves an average score of 3.60.

“GPT 5.1 | 4.17 | 3.36 | | 3.91 | 4.20 | 3.00 | 2.88 | | 3.60 |”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

Probably not a substantial claim: may not pass S3 (load-bearing).

VQA at Goal with Human achieves an average score of 4.12.

“Human | 4.25 | 3.82 | | 4.47 | 4.25 | 4.14 | 3.82 | | 4.12 |”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

May combine two claims.

The high-fidelity rendering from 3DGS allows the creation of tasks where critical visual evidence is only visible from specific viewpoints, thus requiring active viewpoint control in 3D space.

“thereby allowing the construction of questions that cannot be answered from a single view and instead require active inspection across viewpoints in 5-DoF.”

• Also stated in: § 1 Introduction · plain

  “importantly, this high-fidelity rendering makes it possible to define tasks where critical visual evidence is only observable from specific viewpoints, thereby necessitating active viewpoint control in 3D space.”

May combine two claims.

A large number of questions in E3VS-Bench need detailed, viewpoint-specific evidence like text, object states, or occluded areas, and these cannot be answered without moving to suitable viewpoints.

“Despite this global visibility, many questions in E3VS-Bench require fine-grained, viewpoint-dependent evidence (e.g., text, object states, or occluded regions), which cannot be resolved without moving to appropriate viewpoints.”

May combine two claims.

Existing visual search and embodied AI benchmarks, including EQA, generally use static observations or limited egocentric motion, so they do not directly assess fine-grained viewpoint-dependent effects that occur with unrestricted 5-DoF viewpoint control in real 3D environments.

“However, existing benchmarks and agents remain largely constrained to two-dimensional behaviors [23,32], and therefore do not fully capture or evaluate these capabilities in real-world 3D environments.”

• cites [23] Ma, M., et al.: SceneSplat++: a large dataset and comprehensive benchmark for language gaussian splatting. In: Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS) (2025) sourceNot checked — no full text of the source
• cites [32] Su, A., Wang, H., Ren, W., Lin, F., Chen, W.: Pixel reasoner: incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. In: Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS) (2025) sourceNot checked — no full text of the source

May combine two claims.

The episodes that remain are not solvable using only prior knowledge or the first observation; they demand active 5-DoF viewpoint changes and intentional information gathering to answer correctly.

“The remaining episodes cannot be solved from prior knowledge or the initial observation alone, and require active 5-DoF viewpoint transitions and deliberate information gathering for accurate answers.”

May combine two claims.

Counting requires evidence to be accumulated across a 5-DoF trajectory, which standard EQA approaches based on a single observation cannot handle effectively.

“Counting requires evidence to be accumulated across a 5-DoF trajectory, which standard EQA approaches based on a single observation cannot handle effectively.”

Probably not a substantial claim: may not pass S4 (based).

A viewpoint is considered answerable if the image provides sufficient visual evidence to determine the answer without relying on additional viewpoints.

“A viewpoint is considered answerable if the image provides sufficient visual evidence to determine the answer without relying on additional viewpoints.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Viewpoints that involve camera penetration through scene geometry (e.g., walls or objects) were filtered out and marked red.

“Viewpoints are marked red if they are filtered out due to camera penetration through scene geometry (e.g., walls or objects).”

May combine two claims.

OA is highly sensitive to viewpoint selection and requires both exploratory viewpoint discovery and detailed visual recognition.

“This highlights the difficulty of acquiring fine-grained visual evidence through 5-DoF active viewpoint control, suggesting that OA is highly sensitive to viewpoint selection and requires both exploratory viewpoint discovery and detailed visual recognition.”

Classified as probably not a substantial claim.

The evaluation protocol requires the judge to verify whether the predicted answer is supported by the available visual evidence in the end image, which further increases the difficulty of automated assessment.

“In addition, our evaluation protocol requires the judge to verify whether the predicted answer is supported by the available visual evidence in the end image, which further increases the difficulty of automated assessment.”

May combine two claims.

OS requires not only recognizing the target object but also understanding its spatial relations and placement in the scene.

“In contrast, OS requires not only recognizing the target object but also understanding its spatial relations and placement in the scene.”

May combine two claims.

The model is not only capable of recognizing spatial relationships but also of selecting viewpoints that improve the observability of such relationships.

“This suggests that the model is not only capable of recognizing spatial relationships but also of selecting viewpoints that improve the observability of such relationships.”

Probably not a substantial claim: may not pass S3 (load-bearing).

3D scene curation for E3VS-Bench was performed from SceneSplat++.

“(1) 3D scene curation from SceneSplat++ [20]”

• cites [20] Li, G., Xu, J., Zhao, Y., Peng, Y.: DyFo: a training-free dynamic focus visual search for enhancing LMMs in fine-grained visual understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9098–9108 (2025) sourceNot checked — no full text of the source

May combine two claims.

The additional spatial reasoning in OS likely increases task complexity and leads to lower performance.

“This additional spatial reasoning likely increases task complexity and leads to lower performance.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Answers are required to be concise, objective, and based only on visually observable information.

“Answers are required to be concise, objective, and based only on visually observable information.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This criterion also implicitly removes many invalid viewpoints, such as those outside the scene or penetrating scene geometry.

“This criterion also implicitly removes many invalid viewpoints, such as those outside the scene or penetrating scene geometry, where the target object cannot be clearly observed.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Human annotators verify automatically generated QA pairs and remove invalid questions, and correct answers when necessary.

“In Step 3 of the dataset construction pipeline, human annotators verify automatically generated question-answer (QA) pairs and remove invalid questions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

QA generation for E3VS-Bench was performed using a VLM.

“(2) QA generation using a VLM”

May combine two claims.

The quality of geometric evidence is highly viewpoint-dependent; comparing object heights is significantly easier from a lateral viewpoint than from an oblique or top-down view.

“Crucially, the quality of such geometric evidence is highly viewpoint-dependent; for example, comparing object heights is significantly easier from a lateral viewpoint than from an oblique or top-down view.”

May combine two claims.

A Spearman correlation of ρ = 0.54 with human evaluation supports the validity of the VLM-as-a-judge metric, indicating reasonable agreement with human judgments.

“The validity of this metric is supported by a Spearman correlation coefficient of ρ = 0.54 with human evaluation, suggesting reasonable agreement with human judgments.”

Classified as probably not a substantial claim.

Answerability filtering was performed to remove questions solvable without viewpoint transitions.

“(5) answerability filtering to remove questions solvable without viewpoint transitions”

Classified as probably not a substantial claim.

OA questions require identifying fine-grained, instance-specific properties of a target object, such as text, material, color, or subtle visual features.

“OA questions require identifying finegrained, instance-specific properties of a target object, such as text, material, color, or subtle visual features.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Invalid QA filtering for E3VS-Bench was performed with human verification.

“(3) invalid QA filtering with human verification”

• AssumptionRests on an assumption: The human verification of invalid QA filtering is reliable.

Classified as probably not a substantial claim.

Viewpoint-specific phrases were removed from counting questions to reformulate them to reflect structural properties of target objects.

“This modification reformulates the questions to reflect the structural properties of the target objects rather than a specific viewpoint (Fig. 13).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Annotators identify viewpoints that are physically implausible or visually corrupted and mark them as invalid.

“Annotators also identify viewpoints that are physically implausible or visually corrupted.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Current VLMs possess strong 2D image recognition capabilities.

“This suggests that while current VLMs possess strong 2D image recognition capabilities, they show”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

Classified as probably not a substantial claim.

Since answers in E3VS-Bench are open-vocabulary, exact string matching is inadequate for evaluation.

“Since answers in E3VS-Bench are open-vocabulary, exact string matching is inadequate for evaluation.”

Classified as probably not a substantial claim.

The end image corresponds to the final observation used by the agent to generate its answer, enabling the judge to verify whether the prediction is supported by the available visual evidence.

“The end image corresponds to the final observation used by the agent to generate its answer, enabling the judge to verify whether the prediction is supported by the available visual evidence.”

May combine two claims.

Models still struggle either to identify the appropriate target viewpoint or to predict the sequence of actions required to reach it, even when the desired viewpoint is relatively clear.

“This indicates that models still struggle either to identify the appropriate target viewpoint or to predict the sequence of actions required to reach it, even when the desired viewpoint is relatively clear.”

May combine two claims.

Initial viewpoints are often insufficient, which necessitates active exploration.

“The large performance gap between 'VQA at Start' and 'VQA at Goal' indicates that initial viewpoints are often insufficient, necessitating active exploration.”

• AssumptionRests on an assumption: The performance difference between VQA at Start and VQA at Goal is due to the sufficiency of the initial viewpoint.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Steps and Coll. denote the average number of Navigation Steps and collision rate, respectively.

“Steps and Coll.denote the average number of Navigation Steps and collision rate respectively.”

Classified as probably not a substantial claim.

SR questions require understanding geometric relationships such as relative positions, distances, and size comparisons between objects.

“SR questions require understanding geometric relationships such as relative positions, distances, and size comparisons between objects.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Viewpoints not filtered out are marked green as valid initial viewpoints for E3VS.

“Otherwise, they are marked green to indicate valid initial viewpoints for E3VS.”

May combine two claims.

The model can actively adjust its viewpoint to acquire geometrically informative observations, effectively integrating viewpoint selection as part of SR.

“These results indicate that the model can actively adjust its viewpoint to acquire geometrically informative observations, effectively integrating viewpoint selection as part of SR.”

Classified as probably not a substantial claim. May combine two claims.

Viewpoints that satisfy the same_object condition (target object class clearly visible) are retained; others are discarded.

“We retain viewpoints that satisfy the same_object condition, which requires that the target object class is clearly visible in the image rather than ambiguous.”

The quote only partly states this claim.

The preservation of sharp textures for small text by 3DGS is crucial for viewpoint-dependent visual reasoning.

“which is crucial for viewpointdependent visual reasoning.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Each example in the E3VS dataset illustrates a distinct reasoning type that requires viewpoint control in reconstructed 3D environments.

“Each example illustrates a distinct reasoning type that requires viewpoint control in reconstructed 3D environments.”

May combine two claims.

Once the target object is identified, it is often clear whether the current viewpoint is sufficient to determine its state, and the viewpoint required to resolve the uncertainty is relatively predictable.

“Furthermore, once the target object is identified, it is often clear whether the current viewpoint is sufficient to determine its state, and the viewpoint required to resolve the uncertainty is relatively predictable (e.g., as shown in Fig. 5).”

Not yet classified.

Because the dataset filtering removed episodes that GPT-5.1 could answer from the starting viewpoint, employing another model for answer generation would add model-specific bias and harm comparability.

“Since episodes answerable by GPT-5.1 from the start viewpoint were removed during dataset filtering (see Section 3.2), using a different model for answer generation would introduce model-specific bias and compromise comparability.”

Classified as probably not a substantial claim.

Viewpoint labeling was performed to identify answerable viewpoints for E3VS-Bench.

“(4) viewpoint labeling to identify answerable viewpoints”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The evaluation uses VLM judge scores on a scale of 1 to 5, where higher scores indicate better performance.

“All values except Steps and Coll.are VLM judge scores on a scale of 1 to 5 where a higher score indicates better performance.”

• AssumptionRests on an assumption: The VLM-as-a-judge scoring provides a valid measure of answer correctness.

May combine two claims.

Qwen3-VL-8B has a stronger ability to recognize fine-grained object properties and make stopping decisions based on such cues.

“implying a stronger ability to recognize fine-grained object properties and make stopping decisions based on such cues.”

Classified as probably not a substantial claim. May combine two claims.

Annotators label candidate viewpoints associated with each question to identify which viewpoints allow the question to be answered and to remove physically invalid viewpoints.

“After QA verification and answer correction, annotators label the candidate viewpoints associated with each question.”

May combine two claims.

OST questions typically involve binary decisions, which can often be predicted from prior knowledge or dataset bias even without sufficient visual evidence.

“OST questions typically involve binary decisions (e.g., open vs. closed), which can often be predicted from prior knowledge or dataset bias even without sufficient visual evidence.”

May combine two claims.

This characteristic makes exact or near-exact matching more difficult and can naturally reduce the correlation between automated judging and human evaluation.

“This characteristic makes exact or near-exact matching more difficult and can naturally reduce the correlation between automated judging and human evaluation.”

Classified as probably not a substantial claim.

The goal image is additionally provided to account for valid but unannotated answers.

“The goal image is additionally provided to account for valid but unannotated answers.”

Classified as probably not a substantial claim.

The evaluation covers six question types: Object Search (OS), Object State (OST), Object Attribute (OA), Context-guided Search (CGS), Spatial Reasoning (SR), and Counting (CNT).

“Model | OS | OST | OA | CGS | SR | CNT | Avg. | Steps | Coll.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The paper evaluates various baseline models across six fine-grained question types.

“We evaluate various baseline models across six fine-grained question types.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study conducted an ablation on the effect of the thinking process on E3VS-Bench.

“Ablation study on the effect of the thinking process on E3VS-Bench.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The paper's dataset construction pipeline for E3VS-Bench consists of five stages.

“The pipeline consists of five stages: (1) 3D scene curation from SceneSplat++ [20], (2) QA generation using a VLM, (3) invalid QA filtering with human verification, (4) viewpoint labeling to identify answerable viewpoints, and (5) answerability filtering to remove questions solvable without viewpoint transitions.”