University of Birmingham > Talks@bham > Artificial Intelligence and Natural Computation seminars > Gaze Estimation and Perspective-taking Mechanisms in Robots and Humans

Gaze Estimation and Perspective-taking Mechanisms in Robots and Humans

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Hector Basevi.

Host: Dr Hyung Jin Chang

Abstract: In our everyday lives, we often interact with other people. Although each interaction is different and hard to predict in advance, they are usually fluid and efficient. This is because humans are remarkably good at rapidly forming models of others and adapting their actions accordingly. To form these models, humans exploit the ability to take on another person’s point of view, which is termed perspective-taking.

In this presentation, we argue that there is a need to equip robots with perspective-taking abilities to allow for more natural human-robot interactions. We present an artificial visual system that achieves this goal and allows for accurate gaze estimation in natural environments. Large camera-to-subject distances and high variations in head pose and eye gaze angles are common in such environments. This leads to two main shortfalls in state-of-the-art methods for gaze estimation: hindered ground truth gaze annotation and diminished gaze estimation accuracy as image resolution decreases with distance.

We record a novel dataset of varied gaze and head pose images in a natural environment, addressing the issue of ground truth annotation by measuring head pose using a motion capture system and eye gaze using mobile eyetracking glasses. We apply semantic image inpainting to the area covered by the glasses to bridge the gap between training and testing images by removing the obtrusiveness of the glasses. We also present a new real-time algorithm involving appearance-based deep convolutional neural networks, which demonstrates state-of-the-art performance on a number of diverse eye-gaze datasets including our own.

We further argue that it is necessary to understand the underlying mechanisms of perspective-taking in humans. To achieve this goal, we develop a computational model that contains a set of forward-models as building blocks, and an attentional component to reduce the model’s response times. The model is crucial in explaining human data in a variety of experiments and suggests that humans implement a similar attentional mechanism. Several testable predictions are put forward, including the prediction that forced early responses lead to a bias towards a person’s own perspective.

Relevant papers: Fischer & Demiris ICRA2016

Fischer, Chang & Demiris ECCV2018

Fischer & Demiris CVPR2018 workshops

Speaker’s website:

Speaker’s biography: Dr Tobias Fischer conducts interdisciplinary research at the intersection of computer vision, cognitive robotics and computational cognition. His research goal is to provide robots with perceptional abilities that allow interactions with humans in a human-like manner. To develop these perceptional abilities, Tobias believes that it is useful to study the principles used by the human visual system. He uses these principles to develop new computer vision algorithms and validates their effectiveness in intelligent robotic systems. Tobias is currently a post-doctoral researcher in the Personal Robotics Lab at Imperial College London.

Tobias recently received his PhD from Imperial College London with the thesis topic: “Perspective Taking in Robots: A Framework and Computational Model”. He received the M.Sc. degree in Artificial Intelligence from The University of Edinburgh, in August 2014, and a B.Sc. degree in Computer Engineering from Ilmenau University of Technology, Germany, in 2013. He wrote his Bachelor thesis in John Tsotsos’ Lab for Active and Attentive Vision, at the York University, Canada. From February 2012 until August 2014, he was scholarship holder at the prestigious German National Academic Foundation (Studienstiftung des deutschen Volkes), which covered full tuition fees plus a living allowance. From October 2011 to September 2012, he received the Germany Scholarship (Deutschlandstipendium) which provides support for international top-class talent.

Google Scholar profile

This talk is part of the Artificial Intelligence and Natural Computation seminars series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.


Talks@bham, University of Birmingham. Contact Us | Help and Documentation | Privacy and Publicity.
talks@bham is based on from the University of Cambridge.