Robotics Research

Applications: Perception

In order to operate in an intelligent manner, a robot needs information about itself and its environment. This information is obtained through perception, a fundamentally important area of robotics which has historically been the focus of an enormous amount of research.

Perception in robotic systems generally requires data to be detected using sensors and then processed into a form which can be used effectively.Some sensors are based on principles similar to the five human senses, for example vision systems are common, as are sound and touch sensors, and recently there has been significant research into artificial smelling devices. However, sensors on robots are not limited to simply emulating human senses, and robots are commonly equipped with laser range finders, infra-red proximity detectors, GPS systems and inertial sensors. In fact devices exist to sense almost any relevant physical phenomenon.

The processing of sensor information can also be done in a myriad of ways, and is usually a fairly intensive activity requiring a significant amount of computing power. A large amount of current research is concerned with developing successful and efficient techniques for processing sensed data.

Most robots are equipped with a number of different sensors, which enables them to obtain as wide a variety of information as possible about the state of themselves and the environment. This is important because no single sensor is infallible, and generally data from the various sources must be combined in some way to arrive at a meaningful, reliable perception of the robot’s reality. This fusion of data is also an area of extensive research.

The information perceived by a robot is used in world modelling, navigation, control and coordination, learning and adaptation and human-robot interaction. The structure of a robotic system is often influenced heavily by the systems sensing and processing requirements.

Perception Research in Australia

• Researchers at the Monash University’s Intelligent Robotics Research Centre (IRRC) have extensive expertise in a range of perception related technologies, including image processing, ultrasonic sensing, tactile sensing, pattern recognition, computer vision, optical flow, range finding and olfactory sensing.
• At the Australian Centre for Field Robotics (ACFR) research is being conducted into a variety of sensor technologies, including millimetre-wave radar, laser range finder applications, GPS and SLAM aided inertial sensors and undersea sonar.
• Adel Al-Jumaily from the University of Technology, Sydney (UTS) has developed a car number plate recognition system using neural networks.
• David Austin from the Australian National University (ANU) is working on a number of projects in perception, including vision-based SLAM (Simultaneous Localisation and Mapping, a world modelling technique) and 3D object recognition.
• Nick Barnes, also from ANU, is investigating human vision using a log-polar sensor, that was developed based on the primate retina, particularly for fixation and visual interpretation of self-motion.
• Thomas Braunl and the Mobile Robot Lab at the University of Western Australia have produced a number of mobile robot platforms based on their EyeBot vision-based control board. The board is available commercially from Robot Oz.
• Peter Corke from CSIRO has been involved in a number of vision-related projects, including 3D vision for mining and other applications.
• Phillip McKerrow and the University of Wollongong Intelligent Robotics Lab are working on using sonar for plant recognition, and as an input for a landmark based navigation system. A vision system is also being developed.
• The Intelligent Systems Research Group at Deakin University has a number of projects in Visual Information Processing.
• • The Intelligent Vision Surveillance System aims to increase the effectiveness of automated surveillance using novel techniques for recognition of moving objects and adaptive detection of anomalies.
  • Vision Fusion Based Inspection allows automated visual inspection of surface finish using fusion of outputs from a commercial multi-sense camera. 3D Shape, colour, grey scale and other streams are all available simultaneously and when combined allow reliable inspection independent of shadows and other distortions.
  • Algorithms have also been developed to extract objects such as roads and buildings from remotely sensed images. A number of genetic algorithms are used in the extraction process.
  • A system has been developed to detect flaws in cast surfaces. The CASTvision system is able to dynamically adjust its viewing angle enabling it to reliable detect flaws in complex surfaces.
  • A vision based missile guidance system is also under development, with potential applications in robot hand-eye coordination and similar areas.
• The University of Queensland Robotics Laboratory has developed several vision projects, including:
• • Probabilistic landmark recognition systems for robotic mapping and localisation.
  • Rapidly learning object scene recognition.
  • Fast colour segmentation algorithms for colour based environments such as robot soccer.
  • Embedded vision hardware for fast vision processing on small mobile robots.
• The Robotic Systems Laboratory (RSL) at ANU is currently working on a number of vision projects, including a high-performance active vision system (now in its third generation), stereo tracking for head pose and gaze estimation, and a visual interface for human-robot interaction.

  

  The CEDAR active vision head at the Australian National University is a 3rd generation high-performance vision system.

• Biologically inspired visual navigation research is ongoing into robotic navigation techniques. One project contributing to this research is the Visual Circumnavigation project. This project developed a framework for deterministic interaction with objects that a robot might encounter. The robot aims to circumnavigate the object while constantly tracking its relative position. Once the circumnavigation is complete the robot has a basis for interacting with the object from any point.
  Work is also being undertaken on algorithms for robotic fixation and docking using log-polar sensors. Fixation is the task of keeping track of one target over time. Robots require fixation for navigation and for docking with other objects. A log-polar camera has the highest resolution at the center of the recorded image and reduces in resolution logarithmically towards the edges. This means that the camera records images similarly to the human eye - the peripheral vision is at a lower resolution to the vision at the center. The camera has a comparitively low pixel count due to the reduced peripheral resolution. Researchers are examining how human visual fixation works as well as how humans perceive the motion of the world as they move through it.

• 

  The layout of the pixels in a log-polar sensor. The picture demonstrates the reduction in resolution of the sensor from the center to the periphery.

  
  An image as seen by a 'normal' camera or the human eye (left) compared to the same scene of the bike wheel as seen by the log-polar sensor (center). The sensor manipulates the image so that circular elements in the picture are turned into straight lines. The part of the image highlighted by the white rectangle is part of the bicycle's tire. The image to the right is the log-polar image reconstructed into the form of the original image. The part of the image highlighted by the white rectangle is the same as the part highlighted in the central image.

• Colour constancy: The ability for robots to accurately determine the colours in their surroundings can be extremely useful for identification of objects and for navigation. Under different lighting conditions, colours look the same to our eyes even though the wavelength of the reflected light that is hitting our retinas can vary. It is a complicated procedure to correct colour information for robots under different lighting conditions. Nick Barnes and David Austin have researched colour correction for robots and are working on using stereo video cameras mounted on robots to create human-like performance for robotic colour constancy. They aim to create more stable vision in differing lighting conditions. This is a step towards robotic vision that is immune to lighting changes.

  Nick Barnes and Daniel Cameron have developed a system called KADC (Knowledge-based Autonomous Dynamic Colour Calibration) to generate a colour table of an environment purely from geometrical knowledge of a robot's surroundings. The colour table can then be dynamically updated to adjust for changes in the lighting conditions.

• 

  The experimental output under dynamic lighting changes. The images on the left show the actual image. The image in the middle is dynamically colour classified sing Nick Barne's and Daniel Cameron's KADC system, while the image on the right is statically classified. The dynamic colour classifications are more reliably similar to the actual image.