Open Collections

Abstract

This thesis addresses the problem of extracting object masks from video sequences. It presents an online, dynamic system for creating appearance masks of an arbitrary object of interest contained in a video sequence, while making minimal assumptions about the appearance and motion of the objects and scene being imaged. It examines a region-based approach, in contrast to more recently popular pixel-wise approaches to segmentation to illustrate the advantages in the reduction of the complexity of the labeling problem. The redundancy of information typically present in a pixel-wise approach is exploited by an initial oversegmentation of the current video frame. The oversegmentation procedure is based upon a modified version of the classic watershed segmentation algorithm. This oversegmentation produces a set of appearance/motion-consistent regions upon which a conditional random field is constructed. Observations at each region are collected based upon the colour statistics within a region and the motion statistics as determined by the optical flow over the region. An unparameterized model for both the object of interest and the remainder of the scene are constructed on a frame by frame basis. The conditional random field model is used in conjunction with a first order hidden markov model over the frames of the sequence. Mean field approximations for variational inference in this model produce a region-based filter framework which incorporates both spatial and temporal constraints. This framework is used to determine an appropriate labeling for each region in each frame. The reduction in the complexity of the field model produced by the regions (as opposed to pixels) results directly in a reduced cost for the labeling problem with minor effects on accuracy.