CoChat
Sign Up
Cited by 1Robot Manipulation and LearningRobotic Mechanisms and DynamicsAdvanced Vision and Imaging

Dual Quaternion-Based Visual Servoing for Grasping Moving Objects

Open access
De Farias, Cristiana, Adjigble, Maxime, Tamadazte, Brahim, Stolkin, Rustam, Marturi, Naresh·2021·2021 IEEE 17th International Conference on Automation Science and Engineering (CASE)
DOI 10.1109/case49439.2021.9551631
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper presents a new dual quaternion-based formulation for pose-based visual servoing. Extending our previous work on local contact moment (LoCoMo) based grasp planning, we demonstrate grasping of arbitrarily moving objects in 3D space. Instead of using the conventional axis-angle parameterization, dual quaternions allow designing the visual servoing task in a more compact manner and provide robustness to manipulator singularities. Given an object point cloud, LoCoMo generates a ranked list of grasp and pre-grasp poses, which are used as desired poses for visual servoing. Whenever the object moves (tracked by visual marker tracking), the desired pose updates automatically. For this, capitalising on the dual quaternion spatial distance error, we propose a dynamic grasp re-ranking metric to select the best feasible grasp for the moving object. This allows the robot to readily track and grasp arbitrarily moving objects. In addition, we also explore the robot null-space with our controller to avoid joint limits so as to achieve smooth trajectories while following moving objects. We evaluate the performance of the proposed visual servoing by conducting simulation experiments of grasping various objects using a 7-axis robot fitted with a 2-finger gripper. Obtained results demonstrate the efficiency of our proposed visual servoing.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper proposes a dual quaternion-based pose-based visual servoing controller integrated with LoCoMo grasp planning and dynamic grasp re-ranking to grasp arbitrarily moving objects in 3D. In PyBullet simulations with a 7-DoF arm and three YCB objects, the full method achieved average grasp success rates of 80%, 80%, and 70%, outperforming variants without null-space control or without re-ranking.

Key findings

  • The full method, with null-space control and re-ranking enabled, achieved average success rates of [80%, 80%, 70%] across the three objects, compared with [60%, 60%, 40%] without null-space control and [50%, 30%, 30%] without re-ranking.
  • With the full method, the first two objects achieved a 100% success rate for following and grasping the objects, as reported by the authors.
  • When re-ranking was enabled during tracking, the desired grasp configuration switched to three different grasps, with the last remaining stable until grasping; orientation error remained low in both re-ranking and non-re-ranking cases.
  • The controller without null-space control sometimes did not converge due to joint constraints, and the method without re-ranking showed poor performance, which the authors attribute to lack of inverse kinematics during trajectories.

Methods

  • Simulation experiments used PyBullet with a 7-DoF robot arm and a parallel-jaw gripper; three YCB objects were tested: woodblock (YCB ID 45), mustard bottle (YCB ID 9), and scrub cleanser (YCB ID 20). A virtual RGB-D camera and ArUco fiducial markers were used for pose tracking, with point clouds stitched from three camera locations for LoCoMo.
  • The controller is a dual quaternion-based PBVS scheme in joint space, using a damped pseudo-inverse Jacobian and a null-space secondary task for joint-limit avoidance. LoCoMo generates ranked grasp and pre-grasp poses; dynamic re-ranking uses dual quaternion distance and a vantage-point tree, with hysteresis to reduce switching chattering.
  • Objects followed five trajectories: horizontal line, vertical line, diagonal line, ellipsoid with radius (15, 8) cm, and sine wave with amplitude 15 cm and frequency 0.02 Hz, each with and without arbitrary rotations. Grasping used 50 N force, followed by stability tests: lifting 20 cm at 10 cm/s, rotating from 90° to -90° at 45 deg/s, and shaking with amplitude 0.25 m and peak acceleration 10 m/s².

Limitations

  • The paper states that object pose estimation and visual tracking are not in its scope; conventional fiducial markers are used for tracking object movements. Future work includes real robot experiments and integrating object tracking with dual quaternions.
  • The evaluation is simulation-only in PyBullet, with no real-robot validation reported; this limitation is evident from the text.
  • Only three YCB objects suitable for the gripper and marker placement were tested, so performance with unmarked objects or broader object sets is not demonstrated; this limitation is evident from the text.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/case49439.2021.9551631

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 17 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Interpretationcitation
    § C. Grasp Planning and Re-ranking · p. 5

    Setting a hysteresis to the switching strategy, where the control reference pose changes only if the new grasp ranking has improved at least by a switching threshold δ, allows a smooth and dynamic reach-to-grasp trajectory by using the closest best grasp candidate available.

    “To account for that we have set a hysteresis to the switching strategy, i.e. , we only change the control reference pose if the new grasp ranking has improved at least by a switching threshold δ [7]. This strategy allows to have a smooth and dynamic reachto-grasp trajectory by using the closest best grasp candidate available.”
    Quote in the paperSubstantial claim1 of 1 sources checked
  • Premisecitation
    § I. INTRODUCTION · p. 1

    Most existing state-of-the-art grasping methods are one-shot: they perceive once and plan grasps assuming the object is stationary, then command the arm to the grasp pose without feedback.

    “Most of the existing state-of-the-art in grasping are oneshot , i.e. , they perceive once and plan grasps assuming the object is stationary; the arm is then commanded to the grasp pose without using any feedback [3].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [3] source

      The survey describes methods that estimate an object’s pose, select a grasp, and execute it (Online Object Pose Estimation, p. 7; Grasp Synthesis by Comparison, p. 11). It does not establish that most state-of-the-art methods perceive only once, assume a stationary object, or execute without feedback. Its Discussion and Conclusion (p. 17) instead identifies a number of approaches using continuous tactile or visual feedback during execution. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingown data
    § Dual Quaternion-Based Visual Servoing for Grasping Moving Objects · p. 1

    Simulation experiments of grasping various objects using a 7-axis robot fitted with a 2-finger gripper demonstrate the efficiency of the proposed visual servoing.

    “We evaluate the performance of the proposed visual servoing by conducting simulation experiments of grasping various objects using a 7-axis robot fitted with a 2-finger gripper. Obtained results demonstrate the efficiency of our proposed visual servoing.”
    Quote in the paperSubstantial claim
  • Findingreasoning
    § Dual Quaternion-Based Visual Servoing for Grasping Moving Objects · p. 1

    Dual quaternion-based visual servoing formulation enables grasping arbitrarily moving objects in 3D space.

    “we demonstrate grasping of arbitrarily moving objects in 3D space.”
    • Also stated in: § I. INTRODUCTION · p. 1 · plain
      “we present a singularity-robust dual quaternion-based visual servoing integrated with an efficient grasping algorithm to grasp arbitrarily moving objects in the robot's task space as shown in Fig. 1.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Handling arbitrarily moving objects in a 3D space is an open challenge.
  • Findingown data
    § B. Controller Convergence Analysis · p. 6

    The orientation error in both Fig. 3(d) and Fig. 3(f) remained low.

    “Particularly, we note that the orientation error in both Fig. 3(d) and Fig. 3(f) remain low.”
    Quote in the paperSubstantial claim
  • May combine two claims. 11
  • Classified as probably not a substantial claim. 15

  • May combine two claims.

    Premisecitation
    § I. INTRODUCTION · p. 1

    Existing one-shot grasping methods often fail when there are unplanned object movements or uncertainties in estimated hand-to-object relative poses.

    “However, they often fail in case of unplanned object movements or when there are uncertainties in the estimated hand to object relative poses.”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • May combine two claims.

    Findingown data
    § C. Grasping Arbitrarily Moving Objects · p. 7

    On average, for all three objects respectively, the full method achieved [80%, 80%, 70%] success, the method without null space achieved [60%, 60%, 40%] success, and the method without re-ranking enabled achieved [50%, 30%, 30%] success.

    “On average, for all three objects respectively, the full method achieved [ 80 % , 80 % , 70 %] success while the method without null space achieved [60% , 60% , 40%] success and the method without re-ranking enabled achieved [50% , 30% , 30%] success.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B. Controller Convergence Analysis · p. 6

    In case of grasp switching, the controller successfully stabilised midtrajectories and continued tracking the object.

    “In case of grasp switching, the controller successfully stabilised midtrajectories and continued tracking the object.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B. Controller Convergence Analysis · p. 6

    With re-ranking enabled, the desired configuration switched to three different grasps, with the last one remaining stable until the moment of grasping.

    “Particularly, we show the desired configuration switching to three different grasps, with the last one remaining stable until the moment of grasping (at the end of plot).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § B. Controller Design · p. 4

    The dual quaternion-based geometric Jacobian maps the coupled dual quaternion twist to the robot joint space as in [23].

    “Considering J ∈ R 6 × N DoF the dual quaternion-based geometric Jacobian, which maps the coupled dual quaternion twist to the robot joint space as in [23]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] source

  • May combine two claims.

    Interpretationreasoning
    § Dual Quaternion-Based Visual Servoing for Grasping Moving Objects · p. 1

    The robot null-space is explored by the controller to avoid joint limits and achieve smooth trajectories while following moving objects.

    “In addition, we also explore the robot null-space with our controller to avoid joint limits so as to achieve smooth trajectories while following moving objects.”
    • Also stated in: § I. INTRODUCTION · p. 2 · plain
      “the developed dual quaternion-based control strategy (while ensuring a singularity free representation) explores the robot nullspace to provide a smooth motion avoiding joint limits.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: In redundant manipulators, manipulators might get caught in singular configurations.

  • Classified as probably not a substantial claim.

    Premisecitation
    § C. Grasp Planning and Re-ranking · p. 5

    The vantage-point tree (vp-tree) provides a fast and efficient nearest neighbor algorithm that can be used with any arbitrary metric space.

    “We use a vantage-point tree (vp-tree) [25] for searching the nearest grasps to the robot's gripper. The vptree provides a fast and efficient nearest neighbor algorithm that can be used with any arbitrary metric space.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] Yianilos, 1993, Data structures and algorithms for nearest neighbor search in general metric spaces source

  • Classified as probably not a substantial claim.

    Premisecitation
    § I. INTRODUCTION · p. 2

    Dual quaternions have proven to be an efficient and useful representation for vision topics such as hand-eye calibration, simultaneous mapping and localisation, and pose estimation and tracking.

    “More specifically, they have proven to be an efficient and useful representation for many topics related to vision, such as hand-eye calibration [11], simultaneous mapping and localisation [12] and pose estimation and tracking [13]-[15].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [11] source
    • cites [12] source
    • cites [13]-[15]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § B. Method Overview · p. 3

    The two-point reach (pre-grasp then grasp) reduces the chance of collision between the end-effector and the object on its approach.

    “This two-point reach reduces the chance of collision between the end-effector and the object on its approach.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § I. INTRODUCTION · p. 2

    The proposed visual servoing framework simultaneously analyses the quality of multiple grasp configurations and their distance to the gripper in real-time, allowing the robot to switch between grasps as the object moves.

    “In other words, the proposed visual servoing framework simultaneously analyses the quality of multiple grasp configurations and their distance to the gripper. Such analysis is performed in real-time, which allows the robot to switch between grasps as the object moves.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The efficiency of existing methods for grasping moving objects depends on the availability of inverse kinematic solutions throughout the trajectory.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Controller Design · p. 4

    The stability of the closed-loop system defined by the logarithmic controller is discussed in [22].

    “The stability of such a system is discussed in [22].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

  • May combine two claims.

    Interpretationreasoning
    § I. INTRODUCTION · p. 2

    The dual quaternion-based control strategy ensures a singularity-free representation and explores the robot nullspace to provide smooth motion avoiding joint limits.

    “Therefore, the developed dual quaternion-based control strategy (while ensuring a singularity free representation) explores the robot nullspace to provide a smooth motion avoiding joint limits.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: In redundant manipulators, manipulators might get caught in singular configurations.

  • May combine two claims.

    Premisecitation
    § C. Grasp Planning and Re-ranking · p. 5

    The LoCoMo grasp planner neither requires any object 3D models nor offline training, making it easy to integrate with the visual servoing.

    “The advantage of using the LoCoMo grasp planner is twofold. Firstly, the method neither requires any object 3D models nor offline training, making it easy to integrate with our visual servoing.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [16] source

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § C. Grasp Planning and Re-ranking · p. 5

    Using the K closest grasps allows devising a more efficient algorithm, especially for real-time applications.

    “Even though all the generated grasps could be used for the re-ranking stage, using the K closest grasps allow to devise a more efficient algorithm, especially for real-time applications.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § I. INTRODUCTION · p. 2

    The dual quaternion-based PBVS scheme described on the manipulator's joint space is robust to singularities and ensures smooth motion for the manipulator while tracking moving objects.

    “A dual quaternion-based PBVS scheme described on the manipulator's joint space, which is robust to singularities. This ensures smooth motion for the manipulator while tracking moving objects.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In redundant manipulators, manipulators might get caught in singular configurations.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § A. Problem Statement · p. 2

    By using visual servoing instead of conventional path planning, the system can accomplish moving object grasping.

    “Throughout this process, by using visual servoing instead of conventional path planning, we are able to accomplish moving object grasping.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Handling arbitrarily moving objects in a 3D space is an open challenge.

  • May combine two claims.

    Premisecitation
    § A. Mathematical Preliminaries · p. 4

    Unit dual quaternions are singularity-free and can represent full 6 DoF.

    “dual quaternion, which is introduced as non-minimal, noncommutative, singularity-free description of rigid body motions [20].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [20] Selig, 2004 source

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § C. Grasping Arbitrarily Moving Objects · p. 7

    The poor performance of the method without re-ranking is due to the lack of inverse kinematics during trajectories.

    “We believe that this is due to the lack of inverse kinematics during trajectories.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § A. Simulation Setup · p. 5

    Grasp stability is evaluated using various tests as in [18].

    “Once grasped, we perform various tests as in [18] to evaluate grasp stability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § C. Grasping Arbitrarily Moving Objects · p. 6

    The two-point approach reduces the chances that the robot collides with objects.

    “As mentioned, this two-point approach reduces the chances that the robot collides with objects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § A. Simulation Setup · p. 5

    The results are reported following the evaluation protocol presented in [18].

    “We report the results following the evaluation protocol presented in [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § B. Method Overview · p. 3

    The camera used for object pose estimation is scene-mounted and hand-eye calibrated.

    “It is worth noting that we use a scene mounted and hand-eye calibrated RGB-D camera for this purpose.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Simulation Setup · p. 5

    Pybullet has been proved to be an efficient and stable simulation tool for robotics.

    “All experiments are performed using the open source Python wrapper for Bullet physics engine, Pybullet, which has been proved to be an efficient and stable simulation tool for robotics [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Coumans, 2016 source

  • Classified as probably not a substantial claim.

    Study design and conductcitation
    § B. Controller Design · p. 4

    The visually estimated poses (grasps) are transformed to the robot base frame using camera-robot calibration.

    “For this purpose, the visually estimated poses (grasps) are transformed to the base frame using camera-robot calibration. For more details, we refer the readers to [21].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [21] source
24 of 24

  • The paper relies on this without support.

    Handling arbitrarily moving objects in a 3D space is an open challenge.

    “Nevertheless, handling arbitrarily moving objects in a 3D space is an open challenge to solve mainly due to the difficulties associated with planning arm plus hand motions while simultaneously tracking the object trajectory.”

  • The paper relies on this without support.

    The efficiency of existing methods for grasping moving objects depends on the availability of inverse kinematic solutions throughout the trajectory.

    “Although, most of these methods have shown satisfactory performance in grasping moving objects, their efficiency depends on the availability of inverse kinematic solutions throughout the trajectory.”

  • The paper relies on this without support.

    In redundant manipulators, manipulators might get caught in singular configurations.

    “In addition, especially in the presence of redundant joints, manipulators might get caught in singular configurations.”

  • The paper relies on this without support.

    For dynamic grasping tasks, eye-to-hand camera configuration is often preferred because it does not suffer 'blind spots' while visual tracking.

    “For dynamic grasping tasks, the latter is often preferred as it does not suffer 'blind spots' while visual tracking and is more biologically inspired.”

  • The paper relies on this without support.

    Unit dual quaternions are more compact and less computationally demanding than traditional representations.

    “Moreover, unit dual quaternions are singularity-free and have the benefit of being more compact and less computationally demanding than traditional representations, which makes them more suitable for visual servoing tasks.”

  • The paper relies on this without support.

    Literature combining dual quaternion for both robotic control and vision applications is scarce.

    “Despite the increased interest on using dual quaternion for both robotic control and vision applications, literature which combines both is still scarce.”

  • The paper relies on this without support.

    No known instances of using dual quaternion for visual servoing are reported in the literature until very recently a method has been presented by Saltus et al. in [13].

    “No known instances of using dual quaternion for visual servoing are reported in the literature until very recently a method has been presented by Saltus et al. in [13].”

  • The paper relies on this without support.

    The five selected trajectories (horizontal line, vertical line, diagonal line, ellipsoid, sine wave, with and without rotations) are representative of arbitrarily moving objects in 3D space.

    “For this, we have identified 5 different trajectories for the object to move: (i) horizontal line from left side of the table to the right; (ii) vertical line, i.e. , object moving away from the robot; (iii) diagonal line, from left bottom to right upper corner; (iv) ellipsoid with radius (15 , 8) cm ; (v) sine wave with an amplitude and frequency of 15 cm and 0 . 02 Hz , respectively.”
    “Overall, the obtained results clearly demonstrate the efficiency of our proposed dual quaternion-based visual servoing in grasping objects that are moving arbitrarily in a 3D space.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Object-to-table mapping is reversed

    Minor

    The text assigns the mustard bottle to Table II and the scrub cleanser to Table III, but Table II is labeled for the scrub cleanser. Table III is labeled for the mustard bottle.

1 of 1

Cites

Flagged first, then by citations

2 of the 29 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Coumans, 2016, Pybullet, a python module for physics simulation for games, robotics and machine learning

Cited by

Sorted by citations

Related

Authors
De Farias, C., Adjigble, M., Tamadazte, B., Stolkin, R., & Marturi, N.
Affiliations
  • University of Birmingham,Extreme Robotics Laboratory, School of Metallurgy and Materials,Edgbaston,UK,B15 2TT
  • University of Birmingham (GB)
  • Centre National de la Recherche Scientifique (FR)
ORCID
Venue
2021 IEEE 17th International Conference on Automation Science and Engineering (CASE)
Publisher
IEEE
Published
2021-08-23
Pages
151-158
ISSN
2161-8070 (ISSN-L)
DOI
10.1109/case49439.2021.9551631
OpenAlex
W3186940826
Semantic Scholar
61adaa88ccc8eac887580071947d1e128dc70102
Cited by
1
References
29
Citations by year
2021: 1
FWCI
0.39
Type
conference paper
Language
en
Open access
Licence
Locations
  • 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (Le Centre pour la Communication Scientifique Directe) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • UK National Centre for Nuclear Robotics (NCNR)
  • EPSRC (EP/R02572X/1)
  • CRIST-ERA (EP/S032428/1)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Engineering › Control and Systems Engineering › Robotic Mechanisms and Dynamics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
dual quaternions, visual servoing, grasp planning, null space control, joint limit avoidance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review