In many futuristic movies, you see robots performing countless day-to-day tasks. Well… the future is here (almost)! For a project funded by COMMIT, we helped create a robot receptionist, named R3D3 (Rolling Receptionist Robot with Double Dutch Dialogue). The aim of this project was to create a combination of a virtual human and a robot capable of verbal and non-verbal interactions with humans. Together with University of Twente’s HMI and RAM, we succeeded in building a robot platform with the technical capacities to realize such interactions.

The R3D3 prototype can drive around, adjust its height, and carries a tablet with a virtual human face. The robot includes technology for speech recognition and speech production, and has FaceReader based computer vision techniques that can recognize gender, age and emotions. In addition, the virtual avatar on the tablet can interact with people. Here we report the results of three pilot studies, carried out to evaluate the performance of the robot and investigate how people reacted to it. Each pilot tested a different target population; shop visitors, police personnel, and children.

People appreciate an avatar-based interaction

The first pilot was done at the Intratuin garden centre with help of students of TMO Fashion Business School. For evaluation purposes, we used a second prototype based on the Beam Telepresence system, provided by project partner smartrobot.solutions. The Beam robot did not have autonomous functions yet, but mimicked various degrees of robot intelligence. This way we could study how people responded to the robot. Vicar Analytics was also installed to analyze the expressions and demographics of visitors in the area around the robot. Most visitors considered interacting with the robot a positive experience. Although they still preferred a human shop assistant over a robot assistant. However, visitors did appreciate the avatar-based interaction with computer-generated speech more than with human speech. This pilot showed that a robotic shop assistant can improve the customer journey, especially in large shops or by providing live assistance by an expert.

The second pilot was done on a police conference about communication. Because the event was very crowded, the study involved short qualitative evaluations about the robot, but not with the robot. Visitors gave their opinion of the robot and suggested possible applications within the police domain. Overall many people were enthusiastic about this new technology. Most of them saw such a robot performing straightforward tasks, such giving people directions, explaining the organization, or handling simple requests. Visitors also saw the benefits of using computer vision technologies in security and detecting crowd behavior.

Children are interested in a dynamic interaction with the robot

The third pilot was done in the NEMO Science Museum. In this pilot R3D3 interacted with humans fully autonomously. In addition, we developed more features to involve the robot in the conversation. For example, the robot greeted all new users with a happy face and it could direct its head towards a specific person. R3D3 interacted with groups of different sizes, with a maximum of 10 people.

Unfortunately, previous tests indicated that the speech recognition module of R3D3 did not work very well with children. R3D3 could detect speaking but could not respond to the content. Interestingly, the most frequently asked questions of children to the robot were, “how old are you?”, and “what is your name?”. These are examples of difficult questions (for a robot) that we should program into the dialogue in the future. When the robot responded more dynamically to the situation, for example with an angry face if someone interrupted, children were sometimes surprised and intrigued by the robot’s behavior. Although the interactions also elicited confusion on some occasions, the children were very interested in the robot.

Are we ready for a robotic receptionist?

We can conclude that the responses to the robot were quite positive. In some cases, people were a bit startled by a reasonably large robot approaching them, but most people were interested in what it could do. There are still some technical difficulties though. For example, the noise from the surroundings and complexity of speech made mimicking human dialogues complicated. Creating a robotic receptionist as good as a human receptionist still lies in the future. However, these pilots indicated many possibilities where a receptionist robot could prove to be very useful. R3D3 itself will not be standing still. All the partners in this project will keep developing and evaluating R3D3, and there are several events planned to demonstrate the robot.