The (very) long-term research challenges that motivate HOTLab research can be summarised in the following questions:
- Can machines accurately input and analyse human behavior, up to the level of emotional and social signals, in a fully automatic and non-intrusive way?
- Can we automatically simulate realistic human behavior, including believable and useful interaction of virtual with real humans?
Full answers to these questions are still in fairly distant future, so the practical research issues deal with steps that we can realistically take in this direction, as well as near-term applications of the technologies that are developed or within reach. Specific areas that we work on include:
In collaboration with partners at Linkoping University and Visage Technologies AB we are continuing to develop a monocular high-performance multi-platform face and facial feature tracking system. The main distinguishing characteristics of our approach are that the tracker is fully automatic and works with any face without any manual initialization step. It is robust, resistant to rapid changes in pose and facial expressions, does not suer from drifting, is modestly computationally expensive, does not require previous training for successful tracking, recovers quickly from any losses and is capable of switching between dierent faces within the same tracking session. Its modest computational cost makes it ideal for achieving high multi-platform performance. The tracker works both on Windows and Linux as well as on leading mobile platforms: iOS and Android. We have performed a series of tests in dierent environments and under various conditions, using most common consumer devices as well as professional hardware, such as thermal cameras.
In collaboration with partners at Linkoping University and Visage Technologies AB we are continuing to develop Face Analysis. It consists of age, gender and emotion estimation. We are using the results of Face Tracking to implement face analysis algorithms. Its modest computational cost makes it ideal for achieving high multi-platform performance. The analyser works both on Windows and Linux as well as on leading mobile platforms: iOS and Android. We have performed a series of tests in different environments and under various conditions.
In collaboration with partners at Linkoping University and Visage Technologies AB we are continuing to develop Face Recognition.
Local descriptors learning for image matching
Local descriptors are a widely used tool in computer vision and pattern recognition. Some example applications include object/scene recognition and retrieval, face verification, face alignment, image stitching, 3D shape estimation and 3D model retrieval/matching. Current best local descriptors are learned on a large dataset of matching and non-matching keypoint pairs. However, data of this kind is not always available since detailed keypoint correspondences can be hard to establish. On the other hand, we can often obtain labels for pairs of keypoint bags. For example, keypoint bags extracted from two images of the same object under different views form a matching pair, and keypoint bags extracted from images of different objects form a nonmatching pair. On average, matching pairs should contain more corresponding keypoints than non-matching pairs. We attempt to construct effective algorithms for using such data to obtain discriminative local descriptors.