Vision for Cognitive Colonies

Overview

The goal of the vision part of the Cognitive Colonies project is to reconstruct a representation of the environment from images taken from the robots in the colony. The reconstructed environment models may be used for route planning purposes and for presenting the user with a three-dimensional view of the environment. Three-dimensional information is recovered by matching features between images by the robots. From feature matching, accurate robot position is recovered, as well as 3-D location of the features. The multi-robot planning system is used to plan and coordinate robot paths in order to ensure field of view overlap and to maximize coverage of the environment.

Challenges

Feature matching:  Since the relative poses of the robots are not precisely known, the relative camera geometry between images is not known. As a result, features must be matched in order to recover the geometry without the benefit of stonrg constraints. This is the classical correspondence problem. In our case, approximate robot position can be used to guide the search for correspondences and to develop a robust matcher.

Reconstruction: The reconstruction of the 3-D geometry of the environment from feature matching is based on standard "structure from motion" algorithms. Challenges include ensuring reconstruction accuracy, outlier elimination, and view planning.

Map construction: The reconstructed model is a partial and sparse model of the environment. For presentation to the user, the partial reconstruction must be converted to a dense 3-D map, which involves interpolation between features, or a floor plan. Possible representations for the uncertainty of the reconstruction through uncertainty maps are also being explored.

Example:

A typical environment as seen by one of the robots is shown below.


Salient features are extracted from the images taken at different postions by different robots. After matching, the camera geometry and the positions of the robots are recovered ("cooperative stereo"). The displays below show the extracted features and the recovered depth shown as a color-coded templates.