Overcoming catastrophic forgetting in heterogeneous federated learning
department or school: Applied Science, Faculty of
department or school: Electrical and Computer Engineering, Department of
data provider: DSpace
degree campus: UBCV
creator: Arya, Atrin
issued: 2025-01-06T19:38:17Z
issued: 2024
related degree: Master of Applied Science - MASc
degree grantor: University of British Columbia
description: Federated learning (FL) has gained prominence in the medical domain for enabling collaborative machine learning across decentralized datasets while preserving patient privacy. However, key challenges such as catastrophic forgetting and local overfitting hinder the effectiveness of FL models in heterogeneous healthcare environments. This thesis addresses these challenges through two complementary studies. The first study introduces Federated Impressions, a synthetic data generation method that extracts the knowledge of the global model without compromising privacy. By training the local model on these impressions, the method mitigates catastrophic forgetting, allowing the model to retain critical knowledge while generalizing effectively across diverse datasets. This approach ensures robustness and adaptability to evolving medical data distributions. The second study enhances FL for medical imaging by leveraging Vision Transformers (ViTs) in a DualPrompt architecture. Prompts are divided into shared and group prompts: shared prompts capture generalized medical features and are trained exclusively on Federated Impressions alongside local data to prevent overfitting, while group prompts adapt to specific client tasks using local data. This method effectively mitigates catastrophic forgetting, preserving personalization through group prompts while maintaining global generalization via shared prompts. It strikes a balance between retaining global medical knowledge and adapting to localized data distributions. Experimental evaluations on benchmark medical datasets demonstrate that the proposed approaches improve global generalization by avoiding local overfitting. These advancements make FL more robust and effective in heterogeneous medical environments, addressing critical challenges in privacy, scalability, and catastrophic forgetting. This thesis lays the groundwork for scalable and adaptive FL frameworks tailored to medical applications, fostering innovation in privacy…
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/90056
note: Overcoming Catastrophic Forgettingin Heterogeneous Federated LearningbyAtrin AryaA THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF APPLIED SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Electrical and Computer Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)December 2024© Atrin Arya 2024The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:Overcoming Catastrophic Forgetting in Heterogeneous FederatedLearningsubmitted by Atrin Arya in partial fulfillment of the requirements for thedegree of Master of Applied Science in Electrical and ComputerEngineering.Examining Committee:Purang Abolmaesumi, Professor, Electrical and Computer Engineering, UBCCo-supervisorXiaoxiao Li, Professor, Electrical and Computer Engineering, UBCCo-supervisorRobert Rohling, Professor, Electrical and Computer Engineering, UBCSupervisory Committee MemberiiAbstractFederated learning (FL) has gained prominence in the medical domain forenabling collaborative machine learning across decentralized datasets whilepreserving patient privacy. However, key challenges such as catastrophicforgetting and local overfitting hinder the effectiveness of FL models in het-erogeneous healthcare environments. This thesis addresses these challengesthrough two complementary studies.The first study introduces Federated Impressions, a synthetic data genera-tion method that extracts the knowledge of the global model without com-promising privacy. By training the local model on these impressions, themethod mitigates catastrophic forgetting, allowing the model to retain crit-ical knowledge while generalizing effectively across diverse datasets. Thisapproach ensures robustness and adaptability to evolving medical data dis-tributions.The second study enhances FL for medical imaging by leveraging VisionTransformers (ViTs) in a DualPrompt architecture. Prompts are divided…
has type: Thesis/Dissertation
date issued: 2025-05
is shown at: 10.14288/1.0447690
language: eng
degree discipline: Electrical and Computer Engineering
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-NoDerivatives 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-nd/4.0/
scholar level: Graduate
contributor: Abolmaesumi, Purang
contributor: Li, Xiaoxiao
type: Text
identifier URI: http://hdl.handle.net/2429/90056