Mars Autonomy Project Stereo Ranging

We use a stereo calibration and matching algorithm developed by the Machine Vision and Tracking Sensors Group at JPL to convert a pair of greyscale images from our cameras into a cloud of (x,y,z) points on the surface of the terrain ahead. Some of the properties that distinguish stereo ranging from other techniques are:

• It is very computationally intensive and returns large data sets.
• Lack of texture in the terrain or unusual lighting conditions can lead to gaps in the data.
• Precise camera calibration can allow much more efficient matching algorithms.

Our traversability mapper module processes range data to generate traversability maps which rate how easy it is to drive through different points. These maps are passed on to the local obstacle avoidance and global path planning modules which determine the path the rover will take.

Click for a full-size image

• Intrinsic parameters, which relate to the camera itself: the properties of the CCD and the lens. Some of these, like the focal length and size of CCD pixels, are known in advance. The radial lens distortion, however, needs to be calculated.

  If the camera had no radial distortion, the image of each plane of the cube would be a perspective transform of the grid (in particular, the dots would be in straight lines). By measuring the deviations from the expected grid, we can come up with a best-fit model of the distortion and correct for it.

• Extrinsic parameters, which describe the relative position and orientation of the individual cameras to each other and to the world. Because the actual size of the cube is precisely known, the disparity between dot positions in the left and right images lets us infer the length of the stereo baseline. We can also correct for discrepancies in the focal axis and "up" direction of the cameras.

You can try doing your own matching on the stereo pair shown above. Look at the pair and pretend you are trying to focus on an object far behind your monitor screen, until the left and right images appear to fuse in the middle. Once you have the middle image in sharp focus, you should be able to perceive depth in the scene like our stereo algorithm. You may have better luck with the full-size image, moving your head at least 3-4 feet from the monitor.

After we have ranges for points on the terrain ahead at various positions in the image, some trigonometry lets us generate (x,y,z) points in the rover's coordinate frame. The image at the lower left is a snapshot from a 3D browser looking at such a point cloud. The black patches are "shadows" behind the rocks, where no data could be collected because the terrain was obscured.

The density of range points in each grid cell provides a measure of certainty for the traversability score. The different modules respond to uncertainty in different ways. The path planner optimistically assumes that unknown areas are clear of hazards (allowing it to plan paths through unexplored areas), while the local obstacle avoidance system will not allow the vehicle to move into unknown terrain.

A sample traversability map is shown above. Traversability fades from best (green) to worst (red). Certainty fades from highest (bright red or green) to lowest (black).