Robust Trajectory Tracking with Optimal Visual Servoing on a Deformable Object

Abstract

This paper presents a novel approach to rigidly aligning a perspective camera to the deformation of a non-rigid object. We assume having a reference pre-planned camera trajectory viewing a non-rigid object. Our goal is to align this trajectory at the execution time given prior only on the most relevant landmark. Our method does not assume any parametric or non-parametric prior model on the physics of deformation. The proposed method is formalized as a tracking problem embedded in an optimal control framework. The tracking problem encodes visual servoing of features from the deformable object between pre-planned and execution time. The optimal control framework is formalized with a weighted least-square criterion that minimizes the distance between the reference features and the features at run time. The weights are time-dependent smooth functions that encode the relevance of the visible object’s features. We experimentally prove that our method can adapt to multiple pre-planned trajectories and multiple types of deformations without any prior knowledge of them. Robustness to noise in the detection of image features is also experimentally demonstrated.