We are part of the DARPA Tactical Mobile Robotics program, whose goal is to develop portable mobile robots for autonomous operation in urban environments, both indoor and outdoor. This group is part of a team that includes the Jet Propulsion Laboratory and IS Robotics. The overall goal of the project is to develop intelligent, autonomous navigation capabilities using the IS Robotics mobile platform.

Our interest is the use of visual servoing as a key driving mode for such a robot. In a typical use of the robot, the user would designate an area of interest, e.g., a door or a flight of stairs. By servoing on the image of the selected target, the robot executes the mission specified by the user. Technical issues include the selection of suitable templates to track, seamless detection and recovery in the event of loss of track, and integration with other behaviors such as obstacle avoidance. The first issue involves the automatic detection of objects of interest in images in order to facilitate user’s designation. The second issue is key in the context of this project because the robot is expected to experience substantial vibrations and shocks when conducting a typical mission.

We are conducting this work with Prof. Shree Nayar at Columbia University . We are using a version of the Columbia omnidirectional camera as the camera for this project. The omnidirectional camera allows us to select template anywhere in the environment of the robot. The Columbia vision group is working on reducing the size of the omnidirectional camera for integration on a small, portable robot such as the ISR platform.

Other driving modes are also being explored in this program at CMU, including waypoint teleoperation and map-based planning.