Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 16 of the 29 sources its claims cite were checked for support.

Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation

Open access
Schmalstieg, Fabian, Honerkamp, Daniel, Welschehold, Tim, Valada, Abhinav·2023·IEEE Robotics and Automation Letters

Abstract

Existing object-search approaches enable robots to search through free pathways, however, robots operating in unstructured human-centered environments frequently also have to manipulate the environment to their needs. In this work, we introduce a novel interactive multi-object search task in which a robot has to open doors to navigate rooms and search inside cabinets and drawers to find target objects. These new challenges require combining manipulation and navigation skills in unexplored environments. We present HIMOS, a hierarchical reinforcement learning approach that learns to compose exploration, navigation, and manipulation skills. To achieve this, we design an abstract high-level action space around a semantic map memory and leverage the explored environment as instance navigation points. We perform extensive experiments in simulation and the real world that demonstrate that, with accurate perception, the decision making of HIMOS effectively transfers to new environments in a zero-shot manner. It shows robustness to unseen subpolicies, failures in their execution, and different robot kinematics. These capabilities open the door to a wide range of downstream tasks across embodied AI and real-world use cases.

Overview

CoChat Summary

TL;DR

HIMOS is a hierarchical reinforcement learning approach for interactive multi-object search in which a mobile manipulator opens doors and searches cabinets/drawers to find target objects. In simulation it achieves 94.6% average success on seen and 95.6% on unseen apartments for 1–6 objects, and transfers zero-shot to a real Toyota HSR robot with 23/30 (76.7%) success.

Key findings

  • HIMOS outperforms baselines in simulation: average success 94.6% seen and 95.6% unseen for 1–6 target objects, versus Greedy 91.8%/92.7% and SGoLAM+ 89.8%/93.0%; SPL is 55.7 seen and 60.6 unseen.
  • Ablations show frontier exploration is critical: removing it drops average success to 66.5% (unseen; 64.2% seen). Removing local exploration causes a 12.0 ppt average success drop, and removing invalid action masking lowers success/SPL while masking improves convergence speed by up to 2.5×.
  • Real-world HSR experiments complete 23/30 episodes (76.7%) across 1–6 target objects, with 2 collisions, 3 interaction failures, and 2 navigation failures; the high-level policy is robust to subpolicy failures and unseen subpolicies.
  • The learned decision making transfers zero-shot to the real world, unseen environments, unseen subpolicies, and different robot kinematics, given accurate perception.

Methods

  • Task: goal-conditional POMDP where an agent must find k objects out of c categories; success requires seeing each object and navigating within 1.3 m, with a 1,000-timestep limit; doors and cabinets/drawers must be opened to explore.
  • HIMOS: semantic/panoptic map memory; high-level PPO policy in a semi-Markov decision process over a 12-dimensional discrete action space (local exploration, global frontier exploration, instance navigation with interaction), using pretrained low-level subpolicies, adaptive discounting, and invalid action masking.
  • Evaluation: iGibson simulation with a Fetch robot, 8 training and 7 unseen apartments, 1–6 target objects, 25 episodes per scene/object count and 3 seeds (3,600 seen and 3,150 unseen episodes per approach); real Toyota HSR in a ~180 m² office with 30 episodes; metrics are success rate and SPL.

Limitations

  • Stated: the work abstracts from real-world perception and assumes accurate semantic labels, depth/localization, object detection, and a static environment except for the agent’s own interactions; real-world experiments use AR markers and a pre-annotated semantic map, with full perception-pipeline deployment left to future work.
  • Stated: instance navigation assumes a pre-specified maximum number of possible instances; joint training of high- and low-level policies is not performed; finer failure feedback and active perception are future work.
  • Evident from the text: real-world success (76.7%) is lower than simulation (94.6%/95.6%), with irrecoverable failures including a wrist-joint safety limit, base collisions, and repeated navigation-stack failures; real-world evaluation is limited to 30 episodes in one office environment.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/lra.2023.3329619

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 We introduce the Interactive Multi-Object Search task in which an agent has to autonomously search and manipulate the environment to find a set of target objects. To succeed, the agent has to free pathways by opening doors and searching inside articulated objects such as cabinets and drawers.
Fig. 1 We introduce the Interactive Multi-Object Search task in which an agent has to autonomously search and manipulate the environment to find a set of target objects. To succeed, the agent has to free pathways by opening doors and searching inside articulated objects such as cabinets and drawers.From the PDF
Fig. 2 Schematic overview of HIMOS. A semantic map M t serves as a central memory component and is used and updated across low- as well as high-level modules. This map is extended to a partial panoptic map with instance IDs of relevant objects. Given the remaining objects g t to find, the robot state s robot,t , and the derived valid actions v t , the high-level policy acts in an abstract action space consisting of local and global exploration, navigation to mapped object instances, and a mobile manipulation policy.
Fig. 2 Schematic overview of HIMOS. A semantic map M t serves as a central memory component and is used and updated across low- as well as high-level modules. This map is extended to a partial panoptic map with instance IDs of relevant objects. Given the remaining objects g t to find, the robot state s robot,t , and the derived valid actions v t , the high-level policy acts in an abstract action space consisting of local and global exploration, navigation to mapped object instances, and a mobile manipulation policy.From the PDF
Fig. 3 Example trajectories of HIMOS in unseen apartments. Black: unexplored, blue: free space, green: walls, red: agent trace, grey: (found) target objects, other colors: miscellaneous objects. Bottom right: the agent failed to find the last object, marked by the red circle, in the given time.
Fig. 3 Example trajectories of HIMOS in unseen apartments. Black: unexplored, blue: free space, green: walls, red: agent trace, grey: (found) target objects, other colors: miscellaneous objects. Bottom right: the agent failed to find the last object, marked by the red circle, in the given time.From the PDF
Image 4
From the PDF
Fig. 1 In order to explore unstructured environments, the agent has to autonomously manipulate the environment which may include opening doors or looking into articulated objects.
Fig. 1 In order to explore unstructured environments, the agent has to autonomously manipulate the environment which may include opening doors or looking into articulated objects.From the PDF
Fig. 2 Network architectures of the high-level agent (left), the local exploration policy (middle) and the mobile manipulation policy (right). The Resnet-18 are pretrained on Imagenet. The predictive head of the low-level policy is omitted for clarity. ˆ α t is a vector of the angular current and previous predictions, ⃗ v ee , ⃗ v b and ⃗ v torso are end-effector, base and torso velocities r is a reward signal. Full details of the local and mobile manipulation policies can be found in [3] and [7].
Fig. 2 Network architectures of the high-level agent (left), the local exploration policy (middle) and the mobile manipulation policy (right). The Resnet-18 are pretrained on Imagenet. The predictive head of the low-level policy is omitted for clarity. ˆ α t is a vector of the angular current and previous predictions, ⃗ v ee , ⃗ v b and ⃗ v torso are end-effector, base and torso velocities r is a reward signal. Full details of the local and mobile manipulation policies can be found in [3] and [7].From the PDF

Verification

Claims the paper makes

10 Valid · 7 Not valid · 10 Not checked · 55 Own results and interpretations

22 more claims

Create a free account to see all 82 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 25
  • It may state two separate claims as one. 37

Create a free account to see these claims.

Citations

Cites

Show all 42 references

Research works printed but not linked: 1

  1. 10.1109/lra.2023.3336109/mm6 · doi:10.1109/lra.2023.3336109/mm6

Cited by

See all 21

Metadata

Authors
Schmalstieg, F., Honerkamp, D., Welschehold, T., & Valada, A.
Affiliations
University of Freiburg (DE)
Venue
IEEE Robotics and Automation Letters
Published
2023-12
Volume
8
Issue
12
Pages
8549-8556
ISSN
  • 2377-3766 (ISSN-L)
  • 2377-3774
OpenAlex
W4388240413
Cited by
21
References
42
Citations by year
2026: 7 · 2025: 9 · 2024: 4 · 2023: 1
FWCI
1.86
Type
journal article
Language
en
Open access
Locations
  • IEEE Robotics and Automation Letters (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • European Commission (HORIZON2020, 871449)
  • Toyota Motor Europe
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
Keywords
mobile manipulation, hierarchical reinforcement learning, door opening, zero-shot transfer, embodied AI, human-centered environments

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation. https://app.cochat.ai/papers/10.1109/lra.2023.3329619
The paper
Schmalstieg, F., Honerkamp, D., Welschehold, T., & Valada, A. (2023). Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2023.3329619

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.