Graham Taylor

Dynamical Binary Latent Variable Models
for 3D Human Pose Tracking

Graham W. Taylor, Leonid Sigal, David J. Fleet and Geoffrey E. Hinton

We introduce a new class of probabilistic latent variable model called the Implicit Mixture of Conditional Restricted Boltzmann Machines (imCRBM) for use in human pose tracking. Key properties of the imCRBM are as follows: (1) learning is linear in the number of training exemplars so it can be learned from large datasets; (2) it learns coherent models of multiple activities; (3) it automatically discovers atomic movemes; and (4) it can infer transitions between activities, even when such transitions are not present in the training set. We describe the model and how it is learned and we demonstrate its use in the context of Bayesian filtering for multi-view and monocular pose tracking. The model handles difficult scenarios including multiple activities and transitions among activities. We report state-of-the-art results on the HumanEva dataset.

Download paper in PDF format. (Pre-print)
Download supplementary material. (Details on learning)
Download slides (pdf) (keynote-09 zipped) (keynote-08 zipped)

Videos

These videos visually demonstrate the ability of our proposed approach to track in challenging Multi-view and Monocular settings.

The videos were encoded using H.264 and so to view them with this integrated player you will need a relatively new version of Adobe Flash Player (called version 9 Update 3 or v9.0.115.0 which was released on December 3, 2007).

Multi-view tracking

The player will show in this paragraph unless you do not have flash player installed

Monocular tracking

The player will show in this paragraph unless you do not have flash player installed