Nonholonomic Mobile Manipulators (NMMs)

Image-based Visual Servoing Schemes


We consider the task-oriented modeling of the differential kinematics of nonholonomic mobile manipulators (NMMs). A suitable NMM Jacobian is defined that relates the available input commands to the time derivative of the task variables, and can be used to formulate and solve kinematic control problems.

When the NMM is redundant with respect to the given task, we provide an extension of two well-known redundancy resolution methods for fixed-base manipulators (Projected Gradient and Task Priority) and introduce a novel technique (Task Sequencing) aimed at improving performance, e.g., avoiding singularities.

The proposed methods are applied then to the specific case of image-based visual servoing, where the NMM image Jacobian combines the interaction matrix and the kinematic model of the mobile manipulator.


The proposed approach has been developed by A. De Luca, G. Oriolo and P. Robuffo Giordano. The details are given in this paper published in Robotica.

Recently, we also implemented these ideas on a real mobile robot equipped with a pan-tilt camera onboard. Some details of this experimental campaign are given at the bottom of the page. The results were collected in this paper submitted to ICRA'08.


In all simulations, the visual task is the regulation of two point features. These features are extracted from two (red) markers placed on a fixed cubic target.

We tested the visual servoing schemes on two different NMMs, namely a 2R and a 3R manipulators both mounted on a unicycle-like platform.

In the 3R case, the NMM is redundant for the given task and the standard Projected Gradient (PG) can be used to generate the motion.

In the 2R case, the inversion of the square task Jacobian does not yield good results because the Jacobian becomes ill-conditioned, i.e., the system is too constrained to impose an arbitrary motion to both features, although it is still possibile to move them individually. In this situation a possibile solution is to use a strategy which partially relaxes the motion constraints on the features whenever the whole task cannot be executed simultaneously. We propose two methods, namely Task Priority (TP) and Task Sequencing (TS), which can achieve this result.

Unicycle + 3R spatial manipulator

PG: Projected Gradient. The NMM correctly executes the task, while the single degree of redundancy is used to avoid the singularities of the NMM Jacobian (Webots video).

Unicycle + 2R polar manipulator

TP: Task Priority. The regulation of the first feature (lower right) is chosen as the primary task, while the regulation of the second feature is addressed as a secondary task. With this setting, the TP method is able to execute the task without conditioning problems for the full task Jacobian (Webots video).

TS: Task Sequencing. Another idea is to regulate one feature first and then move the other while keeping the first fixed. This approach, called Task Sequencing, introduces an "artificial" redundancy during the first phase even if the NMM is not redundant for the global task. In our simulation we used this additional redundancy to keep the target as much as possible in front of the NMM (Webots video). Note the different final pose of the NMM w.r.t. the previous simulation.

Experimental results

The theoreical claims and simulation results explored in the paper published in Robotca were further validated through a number of experients. We implemented the servoing schemes on the robot MagellanPro, a unicycle-like wheeled mobile platform equipped with a pan-tilt camera. Due to its design, this robot can be classified as a NMM carrying a 2R manipulator, thus perfectly fitting with the previous simulations.

We tested both the TP and the TS method, with the aim of replicating the same conditions of our simulations. The results were quite satisfactory, showing a good adherence with the expected results.

Look at this video for an external/camera view of the motion.

Back to the LabRob Home Page