Our eyes are our windows to the world. There is more behaviour from the face that gives relevant information other than facial expressions. Our gaze reveals what we are looking at and this often indicates what we find important. The size of our pupil responds to variations of lighting but also to task demands. As part of the MIT EyeReader project together with our partner Noldus, we have researched webcam-based eye tracking and pupil analysis. Standard eye-trackers use infrared cameras to assess these eye metrics and thus require specialized hardware. Eye tracking analysis from a webcam will make the applications a lot more flexible. We will continue developing the technology, but here we describe the validation results of two of the most important components of the successfully completed project.

A surprisingly low number of citizens is aware that in the Netherlands both the national and local government broadcast all of their meetings and debates. Even fewer people actually sit down to look at these live or recorded video streams, which is a pity as a majority of the discussions directly affect our daily lives. Therefore, in an attempt to increase participation and bridge the gap between government and citizens, we decided to commit our expertise towards improving the accessibility of these video recordings. Supported with funding from SBIR (Small Business Innovation Research), a government initiative to promote human-centered AI for the public sector, we conducted a 4 month study that finished with a prototype dashboard (see also VITE-A2).

Since the pandemic online meetings have become a widespread phenomenon. During these meetings it is difficult to keep track of all nonverbal behaviour. We have used our research software FaceReader to develop a new tool “VICO” to gain Visual Insights for Video Conferencing. By measuring attention, emotional expressions and reactions of the members in the call one can better understand them. This will help optimize online meetings, such as online focus groups. Students from the University of Twente Finn Prinsenberg and Michel Schmale have used the tool in a very interesting design study to help users learn to read the room in an online environment. This is a short reproduction of their project.

We have surpassed the previous version again and are very satisfied with the new FaceReader 9 release. FaceReader 9 has improved deep learning based modelling capabilities for higher quality and faster analysis. It features a completely new project analysis module which will make the analysis of your results much easier. There are also some new features and outputs, such as gaze direction, head position, heart rate variability, and operators in the custom expression module.

Unfortunately, due to the current corona crisis, events all over the world are cancelled and traveling is almost impossible. This is also a shame for young researchers that can no longer present their research at conferences. Luckily, conference organizers have also found a way to continue online. Amogh Gudi and Marian Bittner, our PhD candidate researchers, have recently presented at two online AI conferences on two very interesting topics.