Improving visual following for Pepper the social robot

Study level


Topic status

We're looking for students to study this topic.


Dr Suman Bista
Research Fellow- Robotic Vision Evaluation and Benchmarking
Division / Faculty
Science and Engineering Faculty
Mr Gavin Suddrey
Research Engineer
Division / Faculty
Institute for Future Environments


For social robots and service robots to be able to operate effectively around people, they must be able to safely move around in environments where humans and other robots are also moving. Pepper, the social robot from Softbank Robotics is already capable of tracking and following a person, but can get confused if they move out of sight around a corner, or if there are multiple people moving in the robot’s field of view.

The aim of this project is to improve Pepper’s visual following capability so that the robot is able to reliably operate in unknown environments, and in crowded spaces.

Research activities

This project is a component of a Vison-Enabled Humanoid Robotics project, which is funded by the Queensland Government, using the Pepper robot from Softbank Robotics. The project aims to enhance future humanoid robotic platforms to help robots see and interact with their environment and interact dynamically with humans.

You will develop algorithms to improve Pepper’s visual following capability by combining the visual target data with information from the robot’s other sensors (IMU, sonars, point lasers, and depth sensor).


The project will result in a demonstration of the Pepper robot following a person around an unfamiliar indoor environment, and following a person through a crowded room.

Skills and experience

Essential skills

Excellent programming skills in Python or C++.

Desirable skills

Basic knowledge of image processing and understanding of navigational sensors such as sonars, lasers, and depth sensors.


You may be able to apply for a research scholarship in our annual scholarship round.

Annual scholarship round



Contact the supervisor for more information.