In-place covariance operators for computer vision

Perhaps one of the most common low-level operations in computer vision is feature extraction. Indeed, there is already a large number of specific feature extraction techniques available involving transforms, convolutions, filtering or relaxation-type operators. Albeit, in this paper we explore a different approach to these more classical methods based on non-parametric in-place covariance operators and a geometric model of image data. We explore these operators as they apply to both range and intensity data and show how many of the classical features can be redefined and extracted using this approach and in more robust ways. In particular, we explore how, for range data, surface types, jumps and creases have a natural expression using first and second-order covariance operators and how these measures relate to the well-known Weingarten map. For intensity data, we show how edges, lines corners and textures also can be extracted by observing the eigenstructures of similar first- and second-order covariance operators. Finally, robustness, limitations and the non-parametric nature of this operator are also discussed and example range and intensity image results are shown.