Parts Based Deformable Registration
The prevalence of non-rigid deformations in real-world objects have presented challenges and opportunities for computer vision researchers for over two decades. Nowhere is this more evident than in HCI applications, where humans are of primary interest, whose facial expressions, hand gestures and the full range of body motion embody intent as well as identifying characteristics useful in recognition. Much work has been aimed at capturing these deformations. Tangential to these efforts is research on the design of recognition algorithms that are invariant to such deformations.
The principal challenges in dealing with object deformations in images is how to account for large variations in appearance an object can undergo due to illumination, viewpoint, and intra-class variabilities, in addition to the non-rigid deformations themselves. In the past few years, separate streams of research on the various types of deformable objects have converged, where unified formal representations and optimisation strategies have led to a paradigm that is, in a way, object invariant. In particular, the most successful setting for this problem that has emerged amongst the various streams is that of geometry constrained parts based detection.
In this tutorial, we will review the history of research on deformable object detection and examine the development of models and algorithms as the state-of-the-art converges to a unified theory. This will include detailed analysis of the famous Active Shape Models and Active Appearance Models and how they relate to the contemporary Pictorial Structure Models. The tutorial will also identify the key challenges and open problems that remain unresolved.
Syllabus
The course will cover the following topics:
- Introduction to Deformable Parts Models
- Regularising Geometric Deformations
- Learned models (Point Distribution Models etc.)
- Generic models (Spring Models etc.)
- Holistic vs Parts Based Representations
- Active shape models
- Active appearance models
- Constrained local models
- Pictorial structure models
- Generative vs Discriminative Approaches
- Eigen-objects
- Learned convex objectives
- Features representations
- Optimisation Strategies
- Lucas-Kanade
- Convex quadratic fitting
- Constrained mean shift
- Graph-based methods
- Open Research Directions
Simon Lucey is an Assistant Research Professor at the Robotics Institute, at Carnegie Mellon University, USA (2005-present). He is currently on leave from CMU, as a Senior Research Scientist, leading the CI2CV computer vision group at the Commonwealth Science and Industrial Organisation (CSIRO) the premiere science organisation in Australia. His research interests lie primarily in applying vision and learning to register, track and recognise events in deformable objects (especially faces and bodies). He obtained his doctoral degree from the Queensland University of Technology (QUT) in 2003. He has run multiple successful workshops at CVPR (“Beyond Patches Workshop”, 2006-7) and ICCV (“Non-rigid Registration and Tracking through Learning”, 2007). He is currently the local organising chair for ICCV 2013 in Sydney Australia. Simon has served as an Associate Editor for the IEEE Transactions of Multimedia, and a Guest Editor for a special issue on “Beyond Patches” in the International Journal of Image and Video Processing (IJIVP) 2009. Dr. Lucey was the recipient of the inaugural Australian Research Council (ARC) “Futures” Fellowship (2009-2013) given to outstanding researchers to conduct their research, of areas of national importance, in Australia.
Jason Saragih is a Research Scientist at the CI2CV computer vision group at the Commonwealth Scientific and Research Organisation (CSIRO), Australia. His research in Computer Vision has focused primarily on the use of learning in deformable object registration and recognition, with particular emphasis on applications in Human-Computer Interfaces. He is the author of two publicly available deformable object modelling API’s; DeMoLib and FaceTracker. He obtained his doctoral degree from the Australian National University in 2008 and spent two years as a Post-doctoral Fellow at the Robotics Institute, at Carnegie Mellon University. He won best paper awards at the VisHCI and DMMD workshops in 2006 and 2011 respectively.