- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Bootstrapping human optical flow and pose
Open Collections
UBC Theses and Dissertations
UBC Theses and Dissertations
Bootstrapping human optical flow and pose Arko, Aritro Roy
Abstract
In this work, we propose a bootstrapping framework to enhance human optical flow and 3D human pose. We show that, for videos involving humans in scenes, we can improve both the optical flow and the pose estimation quality of humans by considering the two tasks at the same time. Generic optical flow methods perform better on humans when fine-tuned on human-centric scenes showing that the focus should be on humans when the task is human oriented. On the other hand, an overlooked assumption in recent 3D human pose estimation methods is temporal consistency. As such, we make use of existing human pose estimators and optical flow networks and improve their performance by benefitting from each other. In more detail, we optimize the pose and optical flow networks to, at inference time, agree with each other. We show that this results in state-of-the-art performance on the Human 3.6M and 3D Poses in the Wild datasets, as well as a human-related subset of the Sintel dataset, both in terms of pose estimation accuracy and the optical flow accuracy at human joint locations.
Item Metadata
Title |
Bootstrapping human optical flow and pose
|
Creator | |
Supervisor | |
Publisher |
University of British Columbia
|
Date Issued |
2022
|
Description |
In this work, we propose a bootstrapping framework to enhance human optical flow and 3D human pose. We show that, for videos involving humans in scenes, we can improve both the optical flow and the pose estimation quality of humans by considering the two tasks at the same time. Generic optical flow methods perform better on humans when fine-tuned on human-centric scenes showing that the focus should be on humans when the task is human oriented. On the other hand, an overlooked assumption in recent 3D human pose estimation methods is temporal consistency. As such, we make use of existing human pose estimators and optical flow networks and improve their performance by benefitting from each other. In more detail, we optimize the pose and optical flow networks to, at inference time, agree with each other. We show that this results in state-of-the-art performance on the Human 3.6M and 3D Poses in the Wild datasets, as well as a human-related subset of the Sintel dataset, both in terms of pose estimation accuracy and the optical flow accuracy at human joint locations.
|
Genre | |
Type | |
Language |
eng
|
Date Available |
2022-08-22
|
Provider |
Vancouver : University of British Columbia Library
|
Rights |
Attribution-NonCommercial-NoDerivatives 4.0 International
|
DOI |
10.14288/1.0417462
|
URI | |
Degree | |
Program | |
Affiliation | |
Degree Grantor |
University of British Columbia
|
Graduation Date |
2022-11
|
Campus | |
Scholarly Level |
Graduate
|
Rights URI | |
Aggregated Source Repository |
DSpace
|
Item Media
Item Citations and Data
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International