Position-based and image-based visual servoing
for reaching and following paths with nonholonomic mobile robots


We present two visual servoing controllers enabling nonholonomic mobile robots with a fixed pinhole camera to reach and follow a continuous path on the ground. The controllers (position-based and image-based) utilize only a small set of features extracted from the image plane, without using the complete geometric representation of the path.

In both control schemes, we set constant linear velocity and apply a nonlinear feedback, based on the path features, to the robot angular velocity. In practice, the path following task is to drive the first visible path point and the corresponding path tangent to a desired configuration. Depending on the position of such point in the image plane, the control schemes switch between two primitive controllers: a row controller, and a column controller. In both primitive controllers, the task is to drive the path features to a desired configuration, while the point is constrained to a line in the image plane (a row of pixels in the first case, and a column of pixels in the second case). For the position-based approach, the path 3D features must be derived from the image features by considering the camera model. The image-based control scheme relies on the Image Jacobian relating the path features with the camera velocity.

A sufficient condition for asymptotic stability of both controllers is derived. It is expressed as a condition on the maximum applicable curvature (hence on the robot kinematic model) and on the path characteristics, which must be compliant with the robot nonholonomic constraint.

The performance of the two controllers is validated and compared by simulations and experiments on a car-like robot equipped with a pinhole camera. The asymptotic stability condition is verified numerically in the experiments, depending on: the controller, the robot parameters, and the desired configuration points.


The proposed approach has been developed by A. Cherubini, F. Chaumette, and G. Oriolo.


This clip shows a simulation realized within Webots. In Webots, we have drawn a circular path, and the two control schemes (position-based and image-based) have been applied starting from an initial configuration with the path intersecting the image right pixel column. Initially, the column controller is used to drive the intersection point along the right pixel column to the bottom right corner. Then, the row controller is used to drive the point along the bottom pixel row to the desired set point.


The two control schemes have been tested on a car-like vehicle with a 70° field of view, forward looking camera. The path features are derived by tracking straight lines and arcs of parabola with the ViSP visual servoing software. The tracker must be initialized by clicking on five path points oriented in the desired path verse.

For each of the two control schemes, experiments with three different initial conditions have been carried out. The three initial conditions are shown in the image below. In order to verify the robustness of the two controllers, the experiments have been repeated by considering a random calibration error on the camera parameters. Furthermore, for the calibrated camera experiments, the system sufficient asymptotic stability condition has been numerically verified .


This clip shows the position-based experiments with calibrated camera.

In this clip instead, the image-based controller is used. The camera is calibrated.

More details and material soon...
Back to the LabRob Home Page