Enforcing structure in visual attention
department or school: Science, Faculty of
department or school: Computer Science, Department of
data provider: DSpace
degree campus: UBCV
creator: Khandelwal, Siddhesh Shyam
issued: 2019-10-18T17:13:15Z
issued: 2019
related degree: Master of Science - MSc
degree grantor: University of British Columbia
description: Visual attention mechanisms have proven to be integrally important constituent components of many modern deep neural architectures. They provide an efficient and effective way to utilize visual information selectively, which has shown to be especially valuable in multi-modal learning tasks. However, all prior attention frameworks lack the ability to explicitly model structural dependencies among attention variables, making it difficult to predict consistent attention masks. In this work we develop a novel structured spatial attention mechanism which is end-to-end trainable and can be integrated with any feed-forward convolutional neural network. This proposed AttentionRNN layer explicitly enforces structure over the spatial attention variables by sequentially predicting attention values in the spatial mask in a bi-directional raster-scan and inverse raster-scan order. As a result, each attention value depends not only on local image or contextual information, but also on the previously predicted attention values. Our experiments show consistent quantitative and qualitative improvements on a variety of recognition tasks and datasets, including image categorization, question answering and image generation.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/71953
note: Enforcing Structure in Visual AttentionbySiddhesh Shyam KhandelwalB.Tech., Indian Institute of Technology (IIT) Guwahati, 2016A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFMaster of ScienceinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Computer Science)The University of British Columbia(Vancouver)October 2019c© Siddhesh Shyam Khandelwal, 2019The following individuals certify that they have read, and recommend to the Fac-ulty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:Enforcing Structure in Visual Attentionsubmitted by Siddhesh Shyam Khandelwal in partial fulfillment of the require-ments for the degree of Master of Science in Computer Science.Examining Committee:Leonid Sigal, Computer ScienceSupervisorJames Little, Computer ScienceSupervisory Committee MemberiiAbstractVisual attention mechanisms have proven to be integrally important constituentcomponents of many modern deep neural architectures. They provide an efficientand effective way to utilize visual information selectively, which has shown tobe especially valuable in multi-modal learning tasks. However, all prior attentionframeworks lack the ability to explicitly model structural dependencies among at-tention variables, making it difficult to predict consistent attention masks. In thiswork we develop a novel structured spatial attention mechanism which is end-to-end trainable and can be integrated with any feed-forward convolutional neuralnetwork. This proposed AttentionRNN layer explicitly enforces structure over thespatial attention variables by sequentially predicting attention values in the spa-tial mask in a bi-directional raster-scan and inverse raster-scan order. As a result,each attention value depends not only on local image or contextual information,but also on the previously predicted attention values. Our experiments show con-sistent quantitative and qualitative improvements on a variety of recognition tasksand datasets, including ima…
has type: Thesis/Dissertation
date issued: 2019-11
is shown at: 10.14288/1.0384602
language: eng
degree discipline: Computer Science
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-NoDerivatives 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-nd/4.0/
scholar level: Graduate
type: Text
identifier URI: http://hdl.handle.net/2429/71953