Offloading embedding lookups to processing-in-memory for deep learning recommender models
department or school: Applied Science, Faculty of
department or school: Electrical and Computer Engineering, Department of
data provider: DSpace
degree campus: UBCV
creator: Zarif, Niloofar
issued: 2023-08-21T20:26:29Z
issued: 2023
related degree: Master of Applied Science - MASc
degree grantor: University of British Columbia
description: Recommender systems are an essential part of many industries and businesses. Generating accurate recommendations is critical for user engagement and business revenue. Currently, deep learning recommender models are commonly used, but they face challenges in processing and representing categorical data, which is a significant portion of the data used by these models. Embedding layers are often used to handle these complications by storing the numerical representation of different categories of a feature in a reduced vector space. Vectors representing all the categories of a feature in the reduced vector space will be stored in a tabular structure named embedding table. The operation of fetching the vector representation of a category from the embedding table and pooling them is called embedding lookup. However, embedding lookups have large memory footprints and require high memory bandwidth, leading to high latency and low throughput. We have developed a new system called PIM-Rec to address these challenges by using the first commercially available Processing-In-Memory (PIM) capable DRAM modules for embedding lookups. PIM-Rec is the first system to use such DRAM modules, and it has shown an 80% decrease in end-to-end inference cycle latency and an 80% increase in latency-bound throughput compared to the standard CPU-only implementation. PIM DRAM modules are a good candidate for handling embedding lookups, especially with the recent drastic size increase of embedding tables. Although PIM-Rec faced obstacles, it offers a realistic solution and analysis while discovering the obstacles and projecting them. This new system provides a promising solution for improving the efficiency of recommender systems and reducing the load they incur in data centers.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/85552
note: Offloading Embedding Lookups to Processing-In-Memoryfor Deep Learning Recommender ModelsbyNiloofar ZarifBSc. of Computer Engineering, Sharif University of Technology, 2019A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFMaster of Applied ScienceinTHE FACULTY OF APPLIED SCIENCE(Electrical and Computer Engineering)The University of British Columbia(Vancouver)August 2023© Niloofar Zarif, 2023The following individuals certify that they have read, and recommend to the Fac-ulty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:Offloading Embedding Lookups to Processing-In-Memory for Deep Learn-ing Recommender Modelssubmitted by Niloofar Zarif in partial fulfillment of the requirements for the de-gree of Master of Applied Science in Electrical and Computer Engineering.Examining Committee:Alexandra Fedorova, Professor, Electrical and Computer Engineering, UBCSupervisorPrashant Nair, Assistant Professor, Electrical and Computer Engineering, UBCSupervisory Committee MemberSathish Gopalakrishnan, Associate Professor, Electrical and Computer Engineer-ing, UBCSupervisory Committee MemberiiAbstractRecommender systems are an essential part of many industries and businesses.Generating accurate recommendations is critical for user engagement and businessrevenue. Currently, deep learning recommender models are commonly used, butthey face challenges in processing and representing categorical data, which is asignificant portion of the data used by these models. Embedding layers are oftenused to handle these complications by storing the numerical representation of dif-ferent categories of a feature in a reduced vector space. Vectors representing allthe categories of a feature in the reduced vector space will be stored in a tabularstructure named embedding table. The operation of fetching the vector representa-tion of a category from the embedding table and pooling them is called embeddinglookup. However, embedding lookups have large me…
has type: Thesis/Dissertation
date issued: 2023-11
is shown at: 10.14288/1.0435518
language: eng
degree discipline: Electrical and Computer Engineering
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-NoDerivatives 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-nd/4.0/
scholar level: Graduate
contributor: Fedorova, Alexandra
type: Text
identifier URI: http://hdl.handle.net/2429/85552