Science, Faculty of
Computer Science, Department of
DSpace
UBCV
Guo, Kaiyun
Master of Science - MSc
University of British Columbia
Single-cell RNA sequencing has fundamentally transformed our understanding of cellular diversity within tissues. The complexity of scRNA-seq data presents several challenges, including complex batch effects, varying sequencing depths, and ambient RNA contamination. Effective data analysis requires robust computational tools, with cell type annotation being a critical component. As the scale of the scRNA-seq dataset grows, several deep-learning models have been developed to offer automated and accurate cell type annotations. However, the lack of interpretability and explainability in these black-box models makes it difficult to understand the reasoning behind their decision-making processes and introduces concerns regarding the reliability of the results. To address this drawback, we propose ProtoCloud, a generative self-explaining model based on prototypical variational autoencoders. The model simultaneously learns cell-type representational prototypes, ensuring interpretability throughout the training process. Additionally, it quantifies the certainty of each prediction by calculating a similarity score between the cell embedding and the prototypes. The classification results demonstrate that ProtoCloud outperforms non-interpretable state-of-the-art methods in cell type annotation, as evidenced by the accuracy, macro F1 score, and Cohen's kappa score on seven extit{in vivo} datasets. To facilitate the explainability of ProtoCloud, this thesis extends layerwise relevance propagation (LRP) to encompass the case of linear layers. By applying LRP on prototypes, ProtoCloud not only underscores known marker genes but also identifies potentially relevant genes and transcriptional signatures. The model has been tested under a variety of experimental conditions in different scenarios, including datasets sourced from various technologies and organs. Its performance has been proven to be robust and consistent, providing accurate and reliable explanations. Finally, ProtoCloud was applied to transfer cell type annotations in two disease contexts: 1) across seven time points following optic nerve crush on mouse retinal ganglion cells, and 2) in human eosinophilic esophagitis, a chronic, progressive, allergy-mediated inflammatory disease of the esophagus across three disease condition from three studies. These applications demonstrate ProtoCloud's broad potential for future use in various research and clinical settings.
ProtoCloud: A GenerativePrototypical Self-explanatory Modelfor Cell Type AnnotationbyKaiyun GuoA THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)August 2024© Kaiyun Guo 2024The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:ProtoCloud: A Generative Prototypical Self-explanatory Modelfor Cell Type Annotationsubmitted by Kaiyun Guo in partial fulfillment of the requirements for thedegree of Master of Science in Computer Science.Examining Committee:Jiarui Ding, Assistant Professor, Department of Computer Science, UBCSupervisorAnne Condon, Professor, Department of Computer Science, UBCSupervisory Committee MemberiiAbstractSingle-cell RNA sequencing has fundamentally transformed our understand-ing of cellular diversity within tissues. The complexity of scRNA-seq datapresents several challenges, including complex batch effects, varying sequenc-ing depths, and ambient RNA contamination. Effective data analysis re-quires robust computational tools, with cell type annotation being a criticalcomponent. As the scale of the scRNA-seq dataset grows, several deep-learning models have been developed to offer automated and accurate celltype annotations. However, the lack of interpretability and explainabilityin these black-box models makes it difficult to understand the reasoning be-hind their decision-making processes and introduces concerns regarding thereliability of the results.To address this drawback, we propose ProtoCloud, a generative self-explainingmodel based on prototypical variational autoencoders. The model simulta-neously learns cell-type representational prototypes, ensuring interpretabil-ity throughout the training process. Additionally, it quantifies the certaintyof each prediction by calculating a similarity score between the cell em-bedding and the prototypes. The classification results demonstrate thatProtoCloud outperforms non-interpretable state-of-the-art methods in celltype annotation, as evidenced by the accuracy, macro F1 score, and Co-henâs kappa score on seven in vivo datasets. To facilitate the explainabilityof ProtoCloud, this thesis extends layerwise relevance propagation (LRP)to encompass the case of linear layers. By applying LRP on prototypes,ProtoCloud not only underscores known marker genes but also identifiespotentially relevant genes and transcriptional signatures. The model hasbeen tested under a variety of experimental conditions in different scenar-ios, including datasets sourced from various technologies and organs. Itsperformance has been proven to be robust and consistent, providing accu-rate and reliable explanations. Finally, ProtoCloud was applied to transfercell type annotations in two disease contexts: 1) across seven time pointsfollowing optic nerve crush on mouse retinal ganglion cells, and 2) in humaniiiAbstracteosinophilic esophagitis, a chronic, progressive, allergy-mediated inflamma-tory disease of the esophagus across three disease condition from three stud-ies. These applications demonstrate ProtoCloudâs broad potential for futureuse in various research and clinical settings.ivLay SummaryThe emergence of single-cell RNA sequencing allows people to look into theheterogeneity of different cell types within a complex tissue. However, as acrucial first step, annotating these cell types often relies on non-interpretablemethods, raising concerns about reliability. In this thesis, a generative pro-totypical self-explanatory model, ProtoCloud, was developed to accuratelyclassify different types of cells from single-cell RNA sequencing data. Thismodel provides explainability at the global (cell) level and local (featureor gene) level to learn and interpret the unique gene expression patterns ofeach cell type. Unlike other methods, ProtoCloud is human-understandable,making it easier for researchers to understand and trust the annotation.vPrefaceThis thesis is an original and unpublished work by the author, K., Guo,under the guidance of J., Ding. The experiments and analyses presented inthis thesis were conducted solely by the author.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . xiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Single-cell transcriptomics data . . . . . . . . . . . . . . . . . 52.1.1 Single-cell RNA sequencing . . . . . . . . . . . . . . . 62.1.2 Single-cell RNA data preprocessing . . . . . . . . . . 72.1.3 Single-cell RNA sequencing downstream analysis . . . 82.2 Variational Autoencoder . . . . . . . . . . . . . . . . . . . . 102.2.1 The predecessor: autoencoders . . . . . . . . . . . . . 102.2.2 Formulation of variational autoencoders . . . . . . . . 112.2.3 Adoption of VAE in single-cell RNA sequencing data 122.3 Explainable Artificial Intelligence . . . . . . . . . . . . . . . 132.3.1 Levels of interpretability . . . . . . . . . . . . . . . . 132.3.2 Approaches of explainability . . . . . . . . . . . . . . 142.3.3 Layer-wise Relevance Propagation . . . . . . . . . . . 15viiTable of Contents3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.1 Cell type annotation . . . . . . . . . . . . . . . . . . . . . . . 173.2 Prototypical self-explaining model . . . . . . . . . . . . . . . 184 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.1 ProtoCloud architecture . . . . . . . . . . . . . . . . . . . . . 194.2 Objective function . . . . . . . . . . . . . . . . . . . . . . . . 224.3 Data augmentation . . . . . . . . . . . . . . . . . . . . . . . 254.4 Prototypical Relevance Propagation . . . . . . . . . . . . . . 265 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.1 ProtoCloud performs accurate cell type annotation . . . . . 305.2 ProtoCloud learns informational latent space . . . . . . . . . 335.3 ProtoCloud enhances explainability of prototypes with PRP 395.4 ProtoCloud yields consistent knowledge across datasets . . . 425.5 ProtoCloud justifies misclassified cells via PRP . . . . . . . . 436 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.1 Annotate optic nerve crush data with ProtoCloud . . . . . . 476.1.1 Transferring knowledge of time-course data . . . . . . 476.1.2 Identifying unassigned cells . . . . . . . . . . . . . . . 506.2 Exploring of eosinophilic esophagitis altas using ProtoCloud 526.2.1 Findings in the eosinophilic esophagitis atlas . . . . . 526.2.2 Knowledge transfer across disease conditions . . . . . 526.2.3 Refining cell type annotation . . . . . . . . . . . . . . 547 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60AppendicesA Experiment Setting . . . . . . . . . . . . . . . . . . . . . . . . 73B Supplementary Figures and Tables . . . . . . . . . . . . . . . 76viiiList of Tables2.1 A non-exhaustive summary of scRNA-seq atlases . . . . . . . 95.1 ProtoCloud performance on rare cell types . . . . . . . . . . . 325.2 Batch entropy score for different latent components . . . . . . 355.3 HRG relevance score ranking of same cell type across differentbatches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.4 HRG relevance score ranking of same cell type across differentorgans. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43A.1 RGC ONC datasets summary . . . . . . . . . . . . . . . . . . 75B.1 Number of cells assigned to each prototype . . . . . . . . . . 77B.2 Cell type distribution across different health conditions in EoE 80B.3 Summary of prediction by cell subsets . . . . . . . . . . . . . 81ixList of Figures2.1 The autoencoder framework . . . . . . . . . . . . . . . . . . . 112.2 The variational autoencoder framework . . . . . . . . . . . . 124.1 An illustration of the ProtoCloud model . . . . . . . . . . . . 205.1 ProtoCloud results on PBMC10K . . . . . . . . . . . . . . . . 295.2 Classification performance comparing to benchmark approach 315.3 Model performance analysis under varying conditions withPBMC10K . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.4 Visualizing ProtoCloud latent embedding . . . . . . . . . . . 345.5 Certainty by the end of each training stage . . . . . . . . . . 365.6 Visualization of regional subtype in latent space . . . . . . . . 385.7 Demonstrating the HRG found with PBMC30K. . . . . . . . 415.8 Workflow of how ProtoCloud detect mis-annotation . . . . . 445.9 Dot plot of identified Type1 and Type2 error cells . . . . . . 466.1 Trend of accuracy during continued training on time-coursedata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.2 UMAP visualization of time-course data with ProtoCloud . . 506.3 Dot plot of T5-RGCs over time . . . . . . . . . . . . . . . . . 516.4 Dot plot of ooDSGCs-RGCs over time . . . . . . . . . . . . . 526.5 Pie plot of predicted cell types in Clevenger et al. dataset. . . 536.6 Shifts in cellular composition across health conditions in EoE 546.7 Refined annotation from AtlasEoE to another dataset . . . . 556.8 Discoveries with HRGs in cell subsets I . . . . . . . . . . . . . 566.9 Discoveries with HRGs in cell subsets II . . . . . . . . . . . . 57B.1 The gene score of known marker genes for PBMC30K . . . . 76B.2 Prediction certainty with end-to-end training . . . . . . . . . 77B.3 PBMC10K HRG dot plot . . . . . . . . . . . . . . . . . . . . 78B.4 AtlasEoE known marker genes PRP relevance score dot plot . 79xList of AbbreviationsAE AutoencoderCCA Canonical correlation analysisCV Computer visiondpc days post crushELBO Evidence lower boundEoE Eosinophilic esophagitisGERD Gastroesophageal reflux diseaseHRG Highly relevant geneHVG Highly variable geneKL Kullback-LeiblerLLM Large language modelLRP Layer-wise relevance propagationNB Negative BinomialONC Optic nerve crushPBMC Peripheral blood mononuclear cellPCA Principal component analysisPRP Prototype relevance propagationPSEM Prototypical self-explaining modelRNA Ribonucleic acidRNA-seq RNA sequencingscATAC-seq Single-cell assay for transposase-accessible chromatin with sequencingscRNA-seq Single-cell RNA sequencingSEM Self-explaining modelt-SNE T-distributed Stochastic Neighbor EmbeddingUMAP Uniform manifold approximation and projectionUMI Unique molecular identifierVAE Variational autoencoderXAI Explainable artificial intelligencexiAcknowledgementsThroughout my graduate studies, there have been many people I must ex-press my gratitude to. I would like to express my deepest gratitude to allthose who have supported me throughout the process of researching andwriting this thesis.First and foremost, I sincerely thank my supervisor, Jiarui Ding, for hisinvaluable guidance and insightful feedback during my study. His expertiseand dedication have been instrumental in shaping the direction and qualityof this research. I am profoundly grateful for his patience and the manyhours he spent reviewing my work. Thanks to Professor Anne Condon foragreeing to be on my supervisory committee and helping me during thesubmission.Iâm also deeply thankful to my parents. Their unwavering support, bothemotionally and financially, has been the cornerstone of my life. To myfriend Xiangyu Shi, thanks for her companionship and company. Thankyou to all lab members in Dingâs group, including, but not limited to JeffreyNiu, Nikhil Shenoy, Sarah Chen, Yingfeng Lu, Minuk Ma, and Wilson Tu,for making my time in the lab an enjoyable experience.To everyone else who has been a part of my journey but I havenât named,thank you.xiiChapter 1IntroductionCells are the basic units of the structure and function of organisms, andalso the basic unit of life forms. All organisms, except viruses, are madeof cells. Building upon the foundation of conventional cell biology, the riseof genomics research techniques has facilitated a comprehensive analysis ofthe regulatory mechanisms underlying cellular processes. A key objectivein cell biology research is to map the gene expression activity underlyingfundamental cellular activities. Single-cell RNA sequencing (scRNA-seq)technology has emerged as the leading method to achieve this objective. Itprovides a distinct and comprehensive understanding of the transcriptomeat the single-cell level for millions of cells from a tissue or organ [1].As the throughput of scRNA-seq has increased, so has the demand for bioin-formatics analysis in extracting meaningful insights from various scRNA-seqdatasets. Cell types are defined as groups of cells that share similar molecu-lar or functional characteristics, distinguishing them from other groups [2].Categorizing cells into specific types allows researchers to associate molecu-lar signatures with particular cellular functions, significantly simplifying theanalysis of biological activities. The process of determining and assigningcell types to individual cells is known as cell type annotation. In this the-sis, the terms âannotationâ, âlabelâ and âclassâ are used interchangeably torefer to the cell type of a given cell. Conventional approaches for cell typeannotation of scRNA-seq data typically involve dimension reduction andclustering analysis, followed by manual annotation based on known markergenes [3]. As the granularity of cell types increases with the explorationof the field, the demand for more expert knowledge grows, making it dif-ficult to establish a definitive ground truth. Additionally, repetitive workis often required to annotate datasets collected under similar techniques,experiments, or diseases. This redundancy underscores the need for morestreamlined and efficient annotation methods to minimize duplicated efforts.There is an urgent need to leverage existing well-labelled datasets for labeltransfer in a reliable and reproducible manner, allowing for the transfer of1Chapter 1. Introductionwell-defined annotations to newly generated datasets or enhancing the res-olution of roughly labelled datasets.The intrinsic challenges of scRNA-seq are posed by high dimensionality,inherent sparsity, and batch effects. Accurate cell type annotation and athorough understanding of cell-type composition are essential for identify-ing potential therapeutic targets and formulating precision medicine strate-gies for diverse disease conditions. Cells can manifest in multiple cyclingstates and subtypes, further complicating their identification. The challengeis compounded by the fact that many subdivision cell types share markergenes, with their expression levels overlapping among different cell types andstates. This extensive diversity further complicates the annotation processand presents significant challenges in accurate cell type annotations.Deep learning methods have revolutionized the analysis of complex dataacross numerous scientific disciplines, owing to their capacity to model in-tricate patterns and relationships within large datasets. This ability is par-ticularly transformative in the field of scRNA-seq analysis, where the vastand high-dimensional nature of the data demands sophisticated computa-tional approaches. Many approaches have been proposed, and some producestate-of-the-art results on scRNA-seq data [4â9]. However, due to the black-box nature of deep learning models, the explainability and reliability of theresults still remain a problem. There is a pressing need for trustworthymodels that can not only enhance the credibility of the annotations but alsoprovide a reliable basis for downstream analyses in the field of single-cellgenomics. To tackle the above deficiencies, this thesis aims to introduce aself-explaining model (SEM) for cell type annotation in scRNA-seq data,ensuring accuracy and transparency in the results.Addressing the need for transparency and accuracy in label transfer, thisthesis emphasizes the importance of interpretability in model design. Thereis a trend of explainable machine learning to leverage a second model to ex-plain a black-box model [10]. However, this approach is not reliable and maygenerate misleading results, where the first model predicts the correct resultusing incorrect evidence [10, 11]. Unlike this shortcut, interpretable modelsare transparent and inherently explain how they reach their conclusions.Therefore, it is crucial to build explainability on top of interpretability,rather than relying on shortcuts that can lead to misleading explanations.Self-explaining models (SEMs) integrate interpretability into their architec-ture. According to the three criteria for interpretability [12]âexplicitness,faithfulness, and stabilityâa self-explaining model should possess certain2Chapter 1. Introductioncharacteristics. First, it should offer an immediate and understandable ex-planation. Second, the relevance score given to features should indicate theirtrue importance. Lastly, similar examples should yield consistent explana-tions.The self-explaining models imitate human cognition by focusing on essen-tial features and comparing them with the general features of a given class[12]. Building on this concept, we embed the prototypical network, a typicalSEM, to learn prototypes [13â15]. Prototypes explain predictions by provid-ing clear, interpretable examples that help elucidate how decisions are madeby the model. Under the context of scRNA-seq, the prototypes learned bythe model can be analogized to cell-type-specific metacells. Metacells areconstructed by aggregating several single cells with similar gene expressionprofiles, resulting in fewer but more robust representatives that effectivelyeliminate outliers and target a more compact expression manifold with ac-curate local approximations [16]. This approach is akin to full scRNA-seqanalysis but benefits from increased robustness and simplicity. The learnedprototypes function similarly to metacells, providing representative and in-terpretable clusters of cells. However, to fully interpret the decision-relevantfeatures within the black-box model, we extend our approach by integratinglayer-wise relevance propagation (LRP) [17].Layer-wise relevance propagation [17] is originally a pixel-wise decompo-sition method for non-linear classifiers used in the computer vision field,designed to provide explainability by visualizing the contribution of eachimage pixel to assist the understanding of classification decisions. In thiswork, we extend the layer-wise relevance propagation to linear models andcombine it with learned prototypes to generate ad hoc gene-wise explana-tions. By performing prototypical relevance propagation (PRP [14]), thelearned prototypes help us generate consistent explanations of the model.This approach allows us to dissect the key genes the model has learned andunderstand the rationale behind each classification decision.In this thesis, we developed ProtoCloud, a prototypical generative modeldesigned for interpretable and accurate cell type classification. ProtoCloudoffers interpretability through three key aspects: First, the model obtainslearned prototypes for each cell type; Second, the model enables a trans-parent decision-making process; and lastly, the identification of decision-relevant genes justifies the prediction for each cell. By integrating the powerof a variational autoencoder with a prototypical network, the model obtainscell type-specific latent representations, enabling reliable and explainable3Chapter 1. Introductionpredictions. The interpretability of ProtoCloud does not come at the ex-pense of accuracy. Our experiments demonstrate that ProtoCloud achieveshigher or comparable accuracy compared to analogous non-explainable base-line methods across datasets of varying sizes, technologies, and experiments.Additionally, by using uncertainty metrics, our model effectively detectsmis-annotated cells and outliers, enhancing the overall quality of the anno-tation. The explainability of ProtoCloud makes it a powerful assistant forfacilitating the dissection of the molecular mechanisms underlying complexbiological processes. We applied ProtoCloud to two complex disease sce-narios, demonstrating its capability for facilitating the annotation of newdatasets and uncovering novel marker genes.4Chapter 2BackgroundThis chapter introduces the background information that forms the basis ofthis thesis work. Section 2.1 discusses the background of single-cell tran-scriptomics data. Section 2.2 introduces the variational autoencoder frame-work that this thesis is built on. Section 2.3 will covers the current ap-proaches in explainable artificial intelligence and a summary of layer-wiserelevance propagation.2.1 Single-cell transcriptomics dataRibonucleic acid (RNA) transcription is a steady reflection of physiologicaland pathological stimuli and can be seen as a direct indicator of cell functionand state. Abnormal RNA expressionâwhether over-expressed due to cellexpansion, up-regulated, or under-expressedâis often associated with cir-cadian rhythms, environmental changes, drug treatments, or disease states[18â21]. By analyzing gene expression profiles under various conditions, re-searchers can infer the biochemical functions of genes and their broader rolesin biological processes.Bulk RNA sequencing was the predominant tool in early gene expressionstudies. It takes a population of cells to measure the average global gene ex-pression, resulting in a composite of different gene expression profiles. How-ever, this approach masks cellular heterogeneity, as different cell types thatwould have distinct functions and may react differently to stimuli, are aver-aged together in the expression profile derived from bulk RNA-seq. This lim-itation catalyzed the development of next-generation technology - single-cellRNA sequencing (scRNA-seq). Recent advancements in high-throughputcell separation technologies ([22]) enable the profiling of tens of thousandsof cells per tissue in parallel with unprecedented resolution. By bringing thegene expression to a cellular level, scRNA-seq soon became one of the mostpopular genomic tools for dissecting transcriptome heterogeneity, discover-ing rare cell types and cell states [23].52.1. Single-cell transcriptomics dataWe will briefly go through how scRNA-seq data is generated and the stan-dard computational data analysis workflow in the following sections. Wedescribe the pipeline for obtaining single-cell transcriptomics data in Section2.1.1. We then briefly talk about the standard workflow for preprocessingand analyzing this data in Section2.1.2.2.1.1 Single-cell RNA sequencingThe main workflow of scRNA-seq typically consists of sample preparation,transcription sequencing, and short-read alignment.The first step of sample preparation involves isolating single cells into in-dividual containers using different protocols. Plate-based protocols [24, 25]isolate cells into wells on a plate, are low throughout techniques allowing forprecise control and higher sensitivity. In contrast, droplet-based methods[22, 26], which capture cells in microfluidic droplets, offer a high level ofthroughput. The latest combinatorial indexing techniques, such as SPLiT-seq [27] and sci-RNA-seq [28], can scale up to hundreds of thousands of fixedcells or nuclei in a single experiment. The extracted RNA undergoes reversetranscription to generate complementary DNA (cDNA). This cDNA is thenamplified to generate enough material for sequencing.The next step, transcript quantification, varies based on the sequencinglength and can be categorized into full-length and tag-based protocols. Full-length protocols are low-throughput methods that aim to achieve uniformread coverage across the entire transcript, making it possible to detect splicevariants and alternative isoforms (e.g., Smart-seq2 [24]). However, the lackof unique molecular identifiers (UMIs) limited our ability to correct for PCRamplification bias, resulting in an uneven coverage of transcripts. Tag-basedprotocols represented by 10x Chromium [22], on the other hand, use UMIsfor transcript quantification that improves the accuracy of quantification bydistinguishing between original RNA molecules and PCR duplicates. Thedownside that was brought up was that the transcript was only sequencedon one of the ends (3â or 5â). There is no universal optimal protocol as thechoice of sequence protocol depends on the targeted organism and the spe-cific biological question being addressed [29]. For instance, droplet-basedmethods are more suitable for studying the composition of heterogeneoustissues due to their high-throughput nature and the ability to handle largecell numbers. Conversely, if the research focus is on studying different iso-forms, full-length protocols are more appropriate because of their highersensitivity in detecting transcript variation.62.1. Single-cell transcriptomics dataThe raw nucleotide sequencing reads are mapped back to their correspondinglocations in a reference genome or transcriptome. The final output is a cell-by-gene matrix where the read counts (or UMI counts) quantify the numberof copies of each gene in each cell.2.1.2 Single-cell RNA data preprocessingQuality control: low-quality cell filteringThe preliminary obtained data contains systematic and random noise thatcan obscure the biological signal. One common artifact is the formation ofdoublets, which occur when a proportion of single-cell containers (dropletsor wells) capture more than one cell or when two beads share the samebarcodes. The frequency of doublets depends on the cell density used duringthe experiment [30]. Due to the high technical noise, quality control is crucialto ensure that only high-quality scRNA-seq data are retained and used fordownstream analysis [31]. By the end of this pipeline, the raw count matrixis obtained and could be used for further analysis.Normalization: noise correctionWe normalize raw read counts to account for differences in sequencing depthand other technical variations. Common normalization techniques includecounts per million (CPM), transcripts per million (TPM), or size factor nor-malization (e.g., using DESeq2 [32]). For scRNA-seq data, cell counts aretypically normalized to a sum of 10,000, followed by a logarithm transfor-mation to highlight changes in expression.Feature selection and dimensionality reductionWith the resolution of tens of thousands of genes measured per cell, manygenes may not be informative or may only be expressed in certain cells. Thedimensionality of scRNA-seq also raises the âcurse of dimensionalityâ thatdistances between cells are similar [33]. Reducing the number of features isessential for improving computational efficiency. This can be done in twoways: feature selection and dimensionality reduction (feature extraction).Common approaches for feature selection in scRNA-seq include highly vari-able gene (HVG) selection, where genes are selected based on their mean/variancerelationship across cells, as these genes are likely to capture important bio-logical differences. Feature selection helps enhance the signal-to-noise ratio72.1. Single-cell transcriptomics dataand highlights the most biologically relevant genes, putting a basis for down-stream analysis.Subsequent to feature selection that retains a subset of informative genesfor downstream analysis, dimensionality reduction projects high-dimensionaldata into a low-dimensional space. The primary goal of dimensionality re-duction is to facilitate the identification of meaningful patterns and struc-tures within the data. A common technique for dimensionality reductionin scRNA-seq is principal component analysis (PCA), which transforms theoriginal high-dimensional data into a set of linearly uncorrelated principalcomponents that capture the most variance in the data. Subsequently, dis-tance metrics can be calculated in the low-dimensional space, indicatingthe variability between different cells. Other commonly used methods inscRNA-seq, such as t-Distributed Stochastic Neighbor Embedding (t-SNE[34]) and Uniform Manifold Approximation and Projection (UMAP [35]),are particularly useful for visualizing complex structures and are used laterin this project.2.1.3 Single-cell RNA sequencing downstream analysisThe downstream analysis of scRNA-seq data encompasses a wide range ofresearch directions, such as cell type annotation, differential expression anal-ysis, trajectory inference, and integrative analysis across multiple datasets,aimed at unravelling the complexity of cellular functions and interactionswithin heterogeneous tissues. These analyses provide insights into cell iden-tity, lineage relationships, and the molecular mechanisms driving develop-ment and disease.One iconic use of the massive scRNA-seq data is to build cell atlases (Table2.1). A cell atlas includes data on the expression levels of hundreds of thou-sands of cells across a wide range of tissues and cell types, often includingmultiple time points or developmental stages. Such a cell atlas provides acomprehensive compendium of the cell diversity in organisms and serves asa foundational tool for developmental biology, disease research, and drugdevelopment.However, scRNA-seq also posts several challenges that complicate down-stream analysis. High-dimensionality and sparsity of the data demand ro-bust statistical and computational methods. As related to this thesis, we willmainly focus on addressing the challenges in cell type annotation (Section2.1.3) and the batch effects (Section 2.1.3).82.1. Single-cell transcriptomics dataProject Ref Species Year Datasets Size (Cells)Human CellAtlas[36] Homo sapiens 2017 53 61.8 milionTabula Muris [37] Mus musculus 2018 20 100,000+Fly Cell Atlas [38] Drosophilamelanogaster2022 2 580,000+Cell Atlas ofWorm[39] Caenorhabditiselegans2017 2 50,000+Plant SingleCell Atlas[40] Arabidopsisthaliana2019 13 20,000+Table 2.1: A non-exhaustive summary of scRNA-seq atlasesCell type annotationCell type identification using scRNA-seq typically consists of two stages.First, the cells undergo clustering using an unsupervised technique, whereincells are grouped based on their gene expression profiles, thus allowing themto identify separate cell populations. Out of the many available clusteringalgorithms, graph-based methods, particularly the Louvain and Leiden [41]algorithms, are highly recommended. Once clusters are identified, we assignbiological identities, including cell types, to these clusters. The standardapproach begins with identifying differentially expressed (DE) genes uniqueto each cluster by comparing gene expression profiles with gene expression.These DE genes are then matched against canonical markers from expertsor reference databases, such as CellMarker [42] and PanglaoDB [43], toannotate the clusters.However, the conventional annotation workflow has many limitations [33].All clustering algorithms require the manual choice of the parameters (e.g.the number of clusters for K-means clustering or the density parameters forgraph-based methods) which can significantly affect the clustering result.Annotating each cluster is another time-consuming process and requires in-depth knowledge of the marker genes of different cell types. As cell typeannotation by marker genes is independent of the clustering algorithm, it isunclear whether the cells within a cluster have been assigned the optimalclass labels.In contrast, automated procedures are favored for their speed, precision, and92.2. Variational Autoencoderenhanced user-friendliness. More related works of automated tools incorpo-rating machine learning techniques that address the above problems will bediscussed in Section 3.1.The problem of batch effectsBatch effects pose a significant challenge in scRNA-seq analysis, arising whensequencing the same material with different experimental batches or proto-cols. These effects can introduce systematic variations unrelated to the bio-logical variables of interest, potentially misleading the analysis [44]. There-fore, addressing these effects is crucial for data reusability. Data integrationtechniques aim to harmonize datasets from multiple batches or studies, elim-inating technical variations while preserving biological variability.2.2 Variational AutoencoderConventional machine-learning techniques have limited the capability of pro-cessing the data in their original form. These methods required considerableunderstanding and expertise to create suitable representations. Deep learn-ing models with their intrinsic characteristics to explore the inner relation-ships of input to learn their representations. These methods surprisinglyimproved the accuracy of various domains such as speech recognition, facerecognition, object detection, and biomedical applications [45â48].The growing application of deep learning in scRNA-seq analysis is driven bythe potential of neural networks to unravel the complexity of cellular hetero-geneity, identify rare cell types, and infer dynamic biological processes. Vari-ational autoencoders (VAEs) as a powerful and flexible generative model,have been widely used in scRNA-seq data analyses (2.2.3).2.2.1 The predecessor: autoencodersThe autoencoder (AE) is widely applied in the field of dimensionality reduc-tion and feature selection like data compression, image denoising, anomalydetection, and facial recognition. As shown in Figure 2.1, its basic formu-lation is to encode the high-dimensional input into low-dimensional latentspace with an encoder, and use the latent embedding to reconstruct thehigh-dimensional input by the decoderâs decoding process. The training ob-jective of AE is to learn the latent representations to best reconstruct theoriginal input. Although AEs are an unsupervised learning approach that102.2. Variational AutoencoderFigure 2.1: The autoencoder framework. The basic structure is to mapthe samples to the hidden variables in the low-dimensional space throughan encoder and then map the hidden variables to the reconstructed samplesthrough a decoder.does not rely on training labels to learn the latent representation, it is con-sidered a self-supervised model that uses the original input x as the groundtruth to measure the reconstruction xË.LAE = E[â¥xâ xËâ¥2]The decoder can be dropped after training in some applications, while theencoder would serve solely as a data-specific compression tool. A linkedbenefit of utilizing AEs as opposed to alternative dimensionality reductionmethods such as PCA is their ability to effectively capture intricate non-linear relationships between the observations and the latent representations.2.2.2 Formulation of variational autoencodersAlthough AEs are effective for dimensionality reduction and reconstruc-tion, they are definitive models that struggle to handle unseen data ef-fectively. Variational autoencoders (variational autoencoder (VAE)s) aredistinguished from autoencoders (autoencoder (AE)s) by learning a latentdistribution for each input data point, rather than a fixed latent vector.VAE incorporates a probabilistic approach, where the encoder maps inputsto a distribution, typically a multivariate Gaussian distribution N (µ,ÏI).The latent embedding z is sampled from this multivariate Gaussian distri-bution, for the decoder to reconstruct data (Figure 2.2). This probabilisticframework allows VAEs to generalize to unseen data better than AEs. Asthe latent sampling process is intractable, the reparameterization trick is112.2. Variational Autoencoderimplemented to help train the encoder and decoder networks by replacingthe sampling operation z â¼ q(z|x) in terms of a differentiable function ofthe form z = µ+ Ï · ϵ, where ϵ â¼ N (0, I).Figure 2.2: The variational autoencoder framework. The encoder ofa VAE models the latent multivariate normal distribution with two param-eters: a vector of means µ and a vector of standard deviations ÏI.The training objective of VAE is to maximize the log-likelihood of the ob-served data log p(x). The objective function of VAE is composed of twoparts: the reconstruction loss as in AE, and a Kullback-Leibler (KL) diver-gence term. The KL divergence quantifies the disparity between the varia-tional distribution q(z|x) and the assumed prior latent distribution f(z).LELBO = Eq(z|x)[log p(x|z)]âDKL(q(z|x) â¥ f(z))= â¥xâ xËâ¥2 â Eq(z|x)[log q(z|x)â log f(z)]Since direct maximization of log p(x) is intractable, we use q(z|x) to ap-proach the posterior distribution p(z|x). As the KL divergence is non-negative, the evidence lower bound (ELBO) serves as a lower bound forthe log-likelihood, and maximizing the ELBO indirectly maximizes the ob-jective function.2.2.3 Adoption of VAE in single-cell RNA sequencing dataMany scRNA-seq VAE models are designed to handle count data directlyas using raw counts preserves the inherent properties of the data, allowingthese statistical models to more accurately capture the underlying biologicalvariability and technical noise [4, 5, 49â52]. There has been a debate aboutwhich distribution best fits the discrete and over-dispersed nature of countdata [53â57]. The negative binomial distribution is widely accepted andcommonly used due to its flexibility in modelling overdispersion. Given that122.3. Explainable Artificial Intelligencethe NB distribution is used as the input distribution, reconstruction loss inthe ELBO is modified to use the log-likelihood of the NB distribution, wherex is the observed input, µ(z) is the mean of reconstruction by the decoderdepends on the latent variable z, and θ is the dispersion parameter:log p(x | z) = logNB(x;µ(z), θ)2.3 Explainable Artificial IntelligenceWhile automated cell-type annotation models have emerged to streamlinethe analysis, machine-learning approaches have inherent limitations thatare not readily explainable. Explainability in machine learning models canbe seen as âthe ability to explain or to present in understandable termsto a human.â [58]. Research under this paradigm is so-called eXplainableartificial intelligence (XAI [59]). Interpretability and explainability are ofteninterchangeable in use, they have different meanings in XAI: interpretabilityhelps users understand the modelâs structure, while explainability refers tothe decisions made for a prediction [60]. We will first introduce differentlevels of interpretability of the models in Section 2.3.1, and the currentdirections of aiding explainability in Section 2.3.2. Lastly, we will explainthe theorem of layer-wise relevance propagation in Section 2.3.3.2.3.1 Levels of interpretabilityResearchers [61] grouped different types of models into three categories ofinterpretability: white box, grey box, and black box models. White boxmodels are linear and monotonic, comprising regression algorithms that arehighly interpretable but generally low in accuracy. These algorithms allowusers to easily understand and inspect how input variables influence theoutput. Grey box models have medium interpretability that is nonlinearbut interpretable. Decision trees fall into this category, offering a balancewhere the internal decision processes of the model can be partially inter-preted. The majority of deep learning algorithms fall into the black boxcategory, which is both nonlinear and noninterpretable. Deep learning uti-lizes the backpropagation algorithm [62] to automatically extract, analyze,and comprehend useful information from complex data sets, this superiorability comes at the cost of no interpretability.132.3. Explainable Artificial Intelligence2.3.2 Approaches of explainabilityThe explainability strategies that are generally applied can be categorizedinto two groups based on their methodology: post-hoc explanations andself-explaining models (SEMs) [63].Self-explaining modelsself-explaining models (SEMs) are designed to provide intrinsic interpretabil-ity, making their decision-making processes understandable without the needfor additional post-hoc analysis. Typical examples of SEMs include tra-ditional machine learning methods like regression, tree-based models andrule-based models. The future direction of SEMs aiming at inherently in-terpretable deep learning models.Deep learning SEM has been getting more attention in recent years. SENN[12] introduced the first self-explainable neural network by decomposing pre-dictions into interpretable components where the explanations are intrinsicto the model. The idea of a transparency classifier is further extended toother SEMs like [63, 64]. A recent work [65] addresses the challenge of in-terpretability by jointly learning a predictive model and its interpreter. Inparallel with these, the prototypical self-explaining model (PSEM) led byProtoPNet [13] has emerged as a stem branch. More related work withPSEMs will be discussed in Section 3.2.Post-hoc explanationsPost-hoc approaches are techniques used to derive explanations for the pre-dictions of a model after it has been trained. Most existing explainablemethods belong to the type, aiming to enhance the explainability of non-interpretable models. For example, Local Interpretable Model-agnostic Ex-planations (LIME) [66] works by approximating the inputâs local approxi-mation of a complex model with a simpler model such as linear regression.SHapley Additive exPlanations (SHAP) [67], based on the concept of Shap-ley values, considers all possible combinations of feature contributions onmodel observations. Anchor [68] explains individual predictions of any blackbox classification model by perturbing the potential rule candidates.However, local interpretability corresponds to the ability to understand therationale behind individual predictions instead of the global context. Un-reliability and high variability brought by these posterior explanations maysuffer from input bias that gains local insights from specific inputs [69]. Due142.3. Explainable Artificial Intelligenceto the high dimensionality and sparsity, scRNA-seq data are more prone togenerate unstable explanations sensitive to noise or batch effects. Given thecritical nature of cell type annotations in scRNA-seq studies, solely relyingon such local interpretations can lead to misleading conclusions.2.3.3 Layer-wise Relevance PropagationLayer-wise Relevance Propagation (LRP) is a back-propagation algorithmused for explainability by decomposing the output prediction back to theinput features. As a post-hoc, model-agnostic approach, LRP interprets thedecisions of convolutional neural networks (CNN) by assigning relevancescores to input features. These scores indicate the importance of features inthe prediction, with more relevant features receiving higher relevance scores,providing a clear visualization of how specific input features contribute tothe modelâs output.In standard LRP, the final prediction output a is assigned as the initialrelevance and back propagates until the input layer is reached. The totalrelevance is conserved per layer - each neuronâs relevance score in a givenlayer is computed as the sum of the relevance scores of all neurons in the nextlayer, weighted by the neuronâs contribution to the next layerâs activation[17, 70]. The general relevance score computation for neuron i in layer l isdescribed by Equation 2.1. Here, Rli and Rl+1j represent the relevance scoresof neurons i and j in layers l and l + 1, respectively. ai represents the ithneuronâs activation. wij stands for the weights connecting the ith neuron inlayer l and the jth neuron of layer l + 1.Rli =âjaiwijâi aiwijRl+1j (2.1)LRP in the biology fieldLRP has found significant applications in the field of biology, particularly inthe analysis of biomedical images ([71â74]). These applications demonstratehow LRP can aid in the diagnosis and understanding of various diseaseconditions.Despite its success in biomedical imaging, the application of LRP in single-cell genomics remains unexplored. While LRP is designed as a post hocmethod for non-linear convolutional neural networks, it can be utilized forother forms of data with modification. Its potential to help interpret multi-layer perceptions has been explored in tabular data [75], highlighting the152.3. Explainable Artificial Intelligencesame discriminative features as SHAP and LIME but with significantly lessexecution time. Although many of the genomics methods are inspired andadapted from CV and LLM [7â9, 76, 77], the intrinsic data characteristics ofgenomics data are very different. Unlike other data types, gene expressioncounts are discrete and each gene is informational by itself, without muchstandalone context from background genes. This contrasts with pixel-baseddata in CV, where the granularity of pixel-level explanations may be lessmeaningful [14]. On the contrary, this granularity is advantageous in thecontext of scRNA-seq as it reveals the decision-sensitive genes. Hanzcer et.al [78] have explored the potential of LRP for phenotype prediction basedon gene expression.However, the standalone use of LRP still faces the same limitations as otherpost-hoc methods (Section 2.3.2). In this thesis, we aim to enhance thereliability of LRP explanations by integrating it with SEMs.16Chapter 3Related WorkIn this chapter, the related works in the field relevant to this thesis willbe discussed. Section 3.1 will discuss the main categories of automatedcell-type annotation approaches. Section 3.2 will review some prototypicalself-explaining models in other fields.3.1 Cell type annotationCell type annotation with clustering and marker identification, as introducedin Section 2.1.3, has been a conventional approach. In recent years, dozens ofautomated approaches have been developed. Xie et al. [79] further classifythese approaches based on their use of reference datasets: marker learningand classifiers learning.Marker learning methods, such as CellAssign [80], are similar to the conven-tional approach that utilizes canonical cell markers as priors to assign thetesting datasets using a probabilistic model. The classifier learning meth-ods train a classifier model to predict unseen cells. These classifiers oftenrely on various similarity or correlation matrices, such as cosine similarity[81], Pearson correlation [82], Spearman correlation [83], or self-defined scorefunctions [84]. In some cases, they just leverage the power of neural net-works to enhance classification performance, including scvis [49], ACTINN[85], MARS [86], scVI [4], scANVI [5], Seurat [87] and CellTypist [6].Non-deep learning approaches typically involve a gene selection step. Forinstance, Seurat uses the HVGs to perform PCA, projecting the referencedata onto the subspace spanned by the principal components (PCs) for com-parison with the query data, CaSTLe [88] uses highly expressed genes withthe highest mutual information between the genes and the class. CellTyp-ist employs logistic regression based on HVGs, optimized with stochasticgradient descent (SGD).173.2. Prototypical self-explaining modelDespite some methods demonstrating superior performance on test data,these methods yield limited interpretability and explainability.3.2 Prototypical self-explaining modelPrototypical models are inherently self-explainable, offering intrinsic inter-pretability through visualization. The first prototypical network, proposedin 2017 by Snell et al. [89], aimed to create an embedding space where pointsfrom the same class are close together, while those from other classes are farapart, with interpretability not being the primary focus. Following this pio-neering work, the first PSEM, ProtoPNet [14], was introduced. ProtoPNetutilizes the mean of several embedded auxiliary samples as the prototype foreach class in few-shot learning. Similarly, Li and Wang [90] use a probabilis-tic generative latent model to derive prototypes from a prior distribution.In both cases, prototypes are not optimized to resemble actual observationsand their visualizations are based on averaging the nearest training samples.Some approaches to incorporating prototypes into the autoencoder frame-work to leverage the decoding functionality to visualize the prototypes. Haseet al. [91] introduces a hierarchical network with a special prototype layer,where each unit of that layer resembles a prototype. The contributionsof a prototype are directly linked to the activation map that indicates thesalient regions of an image. ProtoVAE [15] explains the prototypes with fulltransparency and global class explanation by decoding the prototypes andvisualizing them in the input space.There is an ongoing debate on whether prototype information should be spe-cific to one class or shared across classes. Shared prototypes, as discussed by[91â93], might be more suitable for fine-grained image classification. On theother hand, class-specific prototypes as used in ProtoPNet and ProtoVAE,are analogous to metacells in scRNA-seq. Metacells are aggregate symboliccells derived from homogeneous cells with similar states [16]. They can helpovercome sparsity and noise in scRNA-seq and can provide greater stabilityfor downstream analysis.18Chapter 4ModelWe present the implementation details of ProtoCloud, a supervised, self-explainable generative model for cell type annotation. An overview ofProtoCloud is described in Figure 4.1, the formulation and objective func-tion of the model is illustrated in Sections 4.1 and 4.2. A data augmentationcomponent, which is applied to all rare cell types by default, is discussed inSection 4.3. Finally, Section 4.4 explains the relevance propagation processand how prototypical relevance propagation is built on top of it.4.1 ProtoCloud architectureThe model consists of four main components: an encoder f : RG â Rdz ,a decoder g : Rdz â RG, a prototype matrix P and a transparent linearclassifier h : RKâM â RK . In this context, G is the total number of genes,dz represents the dimensionality of the latent embedding, M refers to thenumber of prototypes per class, and K is the number of classes. Wheninitializing the model, the dataset-dependent prototype matrix P of size(KÃM)Ãdz is randomly initialized in the latent space. The total number ofprototypes is declared by the total number of cell typesK times a predefinednumber ofM prototypes per class (M â¥ 1). The pk matrix represents allMprototypes of class k, whereas pk,j is a vector that indicates the jth prototypeof class k. The prototypes share the same space as the latent embedding andare learnable parameters optimized during training. Each prototype shouldact as a potential representation of a cell type in the dataset.The inputs to ProtoCloud include the raw UMI counts along with the celltype annotations, with a total of N cells in the form of D = (xi, yi)Ni=1.Each cell xi â RG is represented by a high dimensional vector of G genesbelonging to cell type yi = k. To accommodate the inherent character-istics of scRNA-seq input, we assume it follows the negative-binomial NBdistribution that aligns with the noise characteristics of the data.194.1. ProtoCloud architectureFigure 4.1: An illustration of the ProtoCloud model. The model iscomposed of four parts: an encoder, a decoder, a prototype matrix, and aclassifier. The input of ProtoCloud are the cells in the form of a cell-by-gene count matrix, the cells are projected to the VAE latent space via theencoder. Different colours represent different cell types. The larger circlesin the latent space represent the prototype embedding from the prototypematrix. The prototype matrix is pre-initialized and shares the same latentspace as the latent embedding. Each row of the prototype matrix representsa prototype, colored by cell type. Each cell type has six prototypes bydefault. The latent embedding of cells plus the prototype matrix, are thenused for 1) reconstruction of the cells; and 2) prediction of cell types basedon its similarity to the prototypes.204.1. ProtoCloud architectureThe encoder comprises three hidden layers equipped with batch normal-ization and ReLU as the activation function to gradually distill the high-dimensional input data into a lower-dimensional latent space. On the otherhand, the asymmetric decoder is constructed with two MLP blocks, aimedat reconstructing the original data from the latent representations withoutthe risk of causing posterior collapse [94]. Upon pushing the input throughthe encoder leveraging the reparametrization trick z â¼ N (µ, IÏ), the result-ing latent embedding z is then employed in a bifurcated fashion. Firstly, itis used to reconstruct the input UMI count vector x through the decoderxË = g(z). Secondly, it is compared against the prototype matrix P, subse-quently being classified by the single-layer classifier h.A two-step training strategy is applied to better stabilize the training pro-cess and fine-tune the prototypes to further capture each classâs variability.In the first training stage, we aim to learn an informational latent spacewith a preliminary association between the data points and their respec-tive prototypes, where the cell embeddings are tightly gathered around theprototypes of their true classes. As to the second half of the training, thefocus shifts to better differentiate prototypes of the same class by addingthe orthogonal loss to the optimization step. By decoupling the whole train-ing process into two steps, ProtoCloud focuses on different aspects of theprototype generality: the first stage prioritizes inter-class compactness, andthe second stage amplifies intra-class diversity.Since the dimension of each prototype is the same as the latent embedding,the position of the prototypes provides a direct interpretation of the latentembeddings of cells. Each data point in the latent space is associated withthe nearest prototype based on Euclidean distance. This assignment guidesthe training process, encouraging a clear separation between different celltypes in the latent space.Some representation learning approaches reduce the representation dimen-sion to be smaller than the latent dimension (dz), which is already a com-pressed form of the original input dimension (G in our case) [15, 95]. Whilethis strategy can be useful for the disentanglement of essential features, thedimensionality difference may result in explanations that are limited to lo-cal patterns within the data and lack global vision. Consequently, thesemethods typically require upsampling or aggregating techniques to producecomprehensive explanations for the entire sample, which can introduce ar-tifacts and reduce the clarity of the resulting interpretations. In contrast,our model positions the prototypes in the same latent space as the latent214.2. Objective functionfeatures, allowing for a direct global explanation. The disentanglement ofknowledge is done by only comparing the similarity between the two in thefirst half of the latent dimensions. This design encourages the model toretain a rich representation of the data, achieving a structured organizationof the latent space (Section 5.3).4.2 Objective functionThe ProtoCloud objective function comprises four components: the evidencelower bound, classification loss, prototype orthogonal loss, and atomic loss.Classification lossThe classifier h is a single-layer MLP to preserve the transparency of themodel decision-making process. Where the input is the similarity scoresbased on the distance between the latent representation z and the prototypesP. Here, we want to compress the cell type variation into the first half of thelatent space z1. Specifically, the pairwise Euclidean distance is computedbetween latent embedding z1 and the jth prototype vector of class k, p1k,j ,using the first half of latent dimensions.d(z1,p1k,j) =â¥â¥z1 â p1k,jâ¥â¥2The distance-to-similarity score conversion aligns with the form that resem-bles the Cauchy distribution probability density function and yields valuesin the range (0, 1]. The instances that are close to prototypes have highersimilarity scores. The function still allows for a certain degree of similarityfor instances that are farther away, albeit at diminishing levels gradually.si(k, j) = sim(z1i ,p1k,j) =1(d(z1,p1k,j) · κ)2 + 1The κ parameter is learned during training.To strengthen the interpretability of the model, we incorporate a singlelinear classifier and evaluate it with cross-entropy loss. It is worth noticingthat although the classifier functions linearly, the process of transformingthe original count to the latent space effectively captures the intricate non-linear relationship between different cell types [12]. Moreover, it facilitatesthe linear separability of each classâs embedding and promotes the inter-class224.2. Objective functiondiversity of the prototypes.Lpred = 1NNâi=1CE(h(si),yi)Here we use yi to represent the one-hot encoding of yi.Evidence Lower BoundThe model employs a variational autoencoder framework [96, 97], a power-ful generative approach that has gained significant traction in recent yearsfor its ability to learn complex data distributions and has been proven suf-ficient to deal with sparse high-dimensional data like scRNA-seq [4, 5, 50â52]. The evidence lower bound, referred to as LELBO, is modified accordingto the scRNA-seq data characteristics and the requirements of ProtoCloudspecificity, aiming to balance the reconstruction quality and latent spacerepresentation quality.For scRNA-seq data, the UMI counts are typically assumed to follow thenegative binomial (NB) distribution as field agreement [98]. We want themodel to capture the complex nonlinear relationships between the latentvariables and the observations concerning the cell type information. Thus,the NB distribution parameter mean µi,j for gene j in cell i is specified bythe model encoder, where the dispersion Ïk,j is an independent learnableparameter to model the cell-type dispersion (µi,j > 0, Ïk,j > 0).p(xi|zi,θi) =Kâk=1Gâj=1NB(xi,j |µi,j , Ïyi,j) (4.1)Lrecon = â 1NNâi=1Eq(zi|yi,xi)[log p(xi|yi, zi,θi)] (4.2)As using standard normal prior distribution may result in a centralizedclustering of latent variables in a low-dimensional space, we choose Gaus-sian mixture distribution as the variational distribution, centering at eachprototype. The KL loss is scaled by the similarity score, si(k, j), anchoredto the belonging classâs nearest prototype, combining the prior knowledgewith information acquired from the input matrix X.234.2. Objective functionsËi(yi, j) =sim(zi,pyi,j)âMl=1 sim(zi,pyi,l)(4.3)LKL = â 1NNâi=1Mâj=1sËi(yi, j)DKL(N (µi,Ïi)||N (pyi,j , I)) (4.4)Dispersion of prototypesAdditional forces are employed to enhance the alignment of the latent em-bedding, targeting a more compact distribution around their respective pro-totypes. We introduced the orthogonal loss and atomic loss to retain intra-class diversity within each prototype group.Since prototypes are learned along with the latent embedding, they sharesimilar attributes and also suffer from the posterior collapsing problem [94].This loss term enforces that prototype vectors within a class maintain adistinct direction from one another, thereby preventing the dominance ofa single prototype and encouraging the model to capture diverse charac-teristics of the class. The function iterates through each class k, centeringthe prototypes from their mean p¯k and subsequently forming a Gram ma-trix that encapsulates deviations. By subtracting the identity matrix IMof shape M ÃM and computing the Frobenius norm, the model penalizesprototypes that are too similar.Lorth = 1KKâk=1||p¯Tk p¯k â IM ||22Regularization is also incorporated as a penalty term to enhance sparsity,assisting in identifying relevant genes contributing to the classification. Theregularization coefficient λ is set proportionally to the inverse of the numberof weights in the first layer of the encoder We1. Thus, the final orthogonalloss becomes:Lorth = 1KKâk=1||p¯Tk p¯k â IM ||22 +1Î»â¥We1â¥1 (4.5)In short, optimization of this loss promotes orthogonality among prototypeswithin the same class, reducing the information overlap of learned repre-sentations and encouraging the sparsity in the encoderâs gene extractionprocess.244.3. Data augmentationAdditionally, to further refine the specificity of the prototypes, it is essentialthat a cell embedding is sufficiently close to its corresponding prototype.To enhance the clarity of the decision boundary, we also aim for the cellembedding to be distinctly distant from any confusing typeâs prototypes. Alabel-prototype identity matrix is used as a boolean mask, where a valueof 1 in the matrix indicates the prototype contributes to the cell, 0 other-wise. By maximizing the maximum product of the similarity score and themask, we want to attract each latent to the closest prototype of its class.On the contrary, the repulsion loss is to push latent representations awayfrom prototypes that are not of their classes. Together, the two forces en-force latent embedding to stay around their own prototypes while leavingboundaries between different clusters. Furthermore, it reduces the potentialrisk of information loss that may occur due to reducing the dimensionalityof prototypes [14]. As the forces act like the inter-molecular force betweenatoms, together, the two are named atomic loss.Latomic = Lrepulsion â Lattraction (4.6)=1NNâi=1[max(si · ¬maski)âmax(si Â·maski)] (4.7)4.3 Data augmentationSingle-cell RNA sequencing typically contains data that is highly imbal-anced, with extremely rare cell types and dominant cell types. This disparityis prevalent in many integration contexts, such as developmental and can-cer biology, presenting significant challenges for machine learning algorithms[99].One common approach to address this issue is over-sampling, where rareclass populations are duplicated to create more balanced training data. How-ever, this method often leads to overfitting on rare populations, as it doesnot explore the potential data distribution of each class.To mitigate this, we generate artificial cells for rare populations using statis-tical modelling of the gene count distribution for each gene in each rare celltype. Specifically, for each rare cell type (where the ratio is less than 0.5/K),we generate artificial samples to ensure the total number of cells from thisrare population reaches 0.5N/K. We sample these artificial cells of a vectorof UMI counts of dimension G from a multinomial distribution. The multi-nomial distribution rate parameter (r) is estimated from a randomly drawn254.4. Prototypical Relevance Propagationsample with UMI count vector x from a rare population:r = (x+ 0.1)/â(x+ 0.1)Adding 0.1 to x ensures that r will be a dense vector sum to 1. By samplingfrom this multinomial distribution with rate parameter r, we can generatenon-zero counts for genes that have zero counts in x. The size parameter forthe multinomial distribution is uniformly sampled from the interval [0.5 ââ(x + 0.1),â(x + 0.1)]. This ensures the training data has a balancednumber of cells for all types, preventing the model from overfitting to themajor cell types.4.4 Prototypical Relevance PropagationIn prototypical relevance propagation (PRP), the trained prototypes in thelatent space are used to initialize the relevant input for backpropagation[14]. The similarity of the training dataâs corresponding latent embeddingto each prototype is back-propagated and averaged as the relevance-genemapping for each prototype, scaled by the similarity to the prototype. Theinitialized relevance of the jth prototype of cell type k at output layer N isreferred to as aNk,j . A single PRP can generate the input-level explanationfor one prototype.z¯k =1NNâi=1zi · 1(yi = k)aNkj =1pk,j â z¯kNote here a simplified similarity calculation is adopted to magnify the mea-sure of importance, while the similarity score si uses additional scaling fornormalization. As our goal is to find out the most relevant genes contributingtowards cell type classification, we apply the z+-rule that only the positiveweights are considered (equivalent to αβ-rule with α = 1 and β = 0) [17]:w+ij = {wij |wij > 0}Thus, the general LRP-rule mentioned in 2.3.3 becomes:Rli =âjaiw+ijâi aiw+ijRl+1j264.4. Prototypical Relevance PropagationMany small relevance values are more likely to represent noise or less impor-tant auxiliary details during back-propagation. Therefore, we incorporateda relevance filter at each layer to exclude 90% of the small relevance values,allowing only the largest relevance scores to propagate further [100]. Byexcluding small relevance values, the resulting relevance map is expected toconcentrate more on significant features, resulting in a more representativefinal relevance-gene mapping.27Chapter 5ExperimentsProtoCloud is a self-explaining model for fast and accurate cell-type knowl-edge transfer with global interpretability and explainability. The proto-types capture the biological distinctions across cell types, thereby facilitat-ing knowledge transfer across datasets from diverse biological and technicalcontexts. Additionally, leveraging the explainability of ProtoCloud withPRP, the model can reveal cell-type relevant genes, supporting a variety ofdownstream analytical tasks.ProtoCloud will demonstrate full interpretability without sacrificing accu-racy based on the aforementioned three criteria for self-explaining mod-els: explicitness, faithfulness, and stability [101]. The simple demonstrationin Figure 5.1 uses the human peripheral blood mononuclear cells dataset(PBMC10K) from 10X genomics and is included in scVI-tools [102]. Withan overall accuracy of 97.1% and macro F1 score of 0.932 on the validationset, ProtoCloud successfully learned the structure of known cell identitieswith distinct boundaries in the latent space (Figure 5.1.a). We assume thatthe cell typing with lower accuracy will likely have a larger distance and vari-ance to prototypes (Figure 5.1.c, more details in Section 5.5). To furtherdissect the model decision strategy, prototype relevance propagation (PRP)is used to highlight the top relevance genes for each cell type (Figure 5.1.d).A high overlap rate with the known marker genes of a cell type is noticedand serves for further explanation. More explorations and discoveries onreal-world in vivo datasets will be discussed in later sections.We will dive into the following perspective with different datasets to demon-strate the full potential of ProtoCloud in the following sections:1. perform accurate cell type classification compared to SOTA methods,2. learn cell-type specific, self-explainable prototypes and informationallatent space embeddings,3. reveal cell-type-specific highly relevant genes,28Chapter 5. Experiments4. justify its prediction and refine cell annotations,5. integrate reference data for new dataset annotation.ProtoCloud has several hyper-parameters. To substantiate the robustnessof our method, unless specified for some experiments, we apply the samedefault setting across all different datasets.Figure 5.1: ProtoCloud results on PBMC10K with 11,990 cellsand 3,267 genes from eight classes. a) The UMAP visualization ofProtoCloudâs latent space, points with black edges are prototypes coloredby corresponding cell types. b) The classification confusion matrix has anoverall accuracy of 0.971 and a macro F1 score of 0.932. c) The varianceof similarity score for each cell type. The cell types are ordered by classaccuracy from high (left) to low (right). d) Dot plot of the HRGs relevancescore.295.1. ProtoCloud performs accurate cell type annotation5.1 ProtoCloud performs accurate cell typeannotationWhile we have addressed the need for interpretable models, the primaryobjective of such models must remain the accurate classification of celltypes. Unfortunately, the explainability of many XAI methods has been im-proved at the cost of classification efficacy [73, 103]. Here, we demonstrateProtoCloudâs competitive performance in cell type annotation compared tothe state-of-the-art (SOTA) non-explainable counterparts.In our experimental evaluation of ProtoCloudâs capabilities in cell typeidentification and its proficiency in handling diverse scRNA-seq datasets,we embarked on a comparative analysis against three representative meth-ods: CellTypist [6], scANVI [5], and Seurat V4 [87]. The eight datasetsselected for this assessment were diverse, encompassing different species, or-gans, and sequencing technologies. Other than the previously mentionedPBMC10K as a base case, these datasets included PBMC30K [29] andAtlasRGC [104], both of which were generated using various technologiesand exhibited strong batch effects. Additionally, three datasets from theTissue Stability Cell Atlas of different organs, lung (TSCA lung), esoph-agus (TSCA oesophagus), and spleen (TSCA spleen), allowed for cross-organ comparison of cell types[105]. Furthermore, two large-scale cell at-las datasets, ICA [106] and AtlasEoE [107], were included, encompassinghundreds of thousands of cells and numerous cell types (K = 43 and 72,respectively). To quantify each modelâs cell type classification performance,we reported the average accuracy, macro F1-score, and Cohenâs kappa scoreon 5-fold cross-validation (Figure 5.2). Configured with default hyper-parameters for all datasets, ProtoCloud exhibited favorable accuracy, con-sistently matching or exceeding the performance of non-explainable bench-marks across various datasets in all evaluated metrics.To demonstrate the superior performance of ProtoCloud in identifying rarecell types (defined by minority group with portion < 0.5%), we quantifyit with macro F1-score and Cohenâs kappa score, where both metrics areminority/imbalance class sensitive. By up-sampling the cell types below acertain threshold, ProtoCloud expressed superior performance on rare typesin both metrics, providing compelling evidence of its enhanced ability to an-notate rare cell populations accurately. Specifically, our model demonstratesnotable accuracy on the rare cell types. A breakdown example of per celltype accuracy in ICA can be seen in Table 5.1. The per-type accuracy of305.1. ProtoCloud performs accurate cell type annotationFigure 5.2: Benchmarking ProtoCloud against Seurat V4, scANVI,and CellTypist in cell type annotation across eight datasets. Weevaluated (a) accuracy, (b) macro F1, and (c) Cohenâs kappa score. Thematrices are averaged after a 5-fold cross-validation, with 80% as trainingdata and 20% as validation data for each fold.315.1. ProtoCloud performs accurate cell type annotationProtoCloud is particularly high, significantly outperforming all comparablemethods. Compared to the second-best method, CellTypist, ProtoCloud didconsistently better in classifying these rare cell types, e.g., accuracy 0.938vs. 0.719 for Migratory DCs, 0.898 vs. 0.137 for CD8a/b entry. These re-sults highlighted ProtoCloudâs superior performance in identifying rare celltypes.Rare Type Percentage(%)AccuracyProtoCloud CellTypistMigratory DCs 0.049 0.938 0.719CD8a/b (entry) 0.039 0.898 0.137ILC3 0.037 0.833 0.00Immature B cells 0.030 0.934 0.406Germinal center B cells 0.026 0.806 0.159CMP 0.006 1.00 0.00Table 5.1: Comparing ProtoCloud against CellTypist on rare celltype annotations. Here we showed the classification accuracies for the sixrare cell types from the Immune Cell Atlas.ProtoCloud demonstrated a high convergence speed during the trainingphase, reaching above 90% validation accuracy at about 30 epochs (Fig-ure 5.3.a). In the interest of evaluating the modelâs robustness, we testedProtoCloudâs performance under varying conditions using the PBMC10Kdataset. We introduced perturbations by randomly altering varying frac-tions of the training labels (10%, 15%, and 20%). Despite these perturba-tions affecting the explainability of the generalized prototypes, the modelsustained a satisfying validation accuracy above 93% (Figure 5.3.b). Addi-tionally, we performed ablation studies by varying the test ratio from 0.1to 0.9. Notably, the model maintained stable performance, with accuracyranging from 97.6% to 95.3% (Figure 5.3.c). The Macro F1 after trainingon 10% of the data is 0.90. These results demonstrate ProtoCloudâs robustgeneralization ability as a data-efficient model, as it consistently performswell with small training sets and retains high accuracy even when subjectedto label noise.The pronounced performance and robustness exhibited by ProtoCloud incellular classification tasks underscore its potential as a reliable tool forscRNA-seq data analysis. This reliability underpins the subsequent elucida-tion of explainability from multiple perspectives, establishing its foundationas a dependable tool for scRNA-seq analysis.325.2. ProtoCloud learns informational latent spaceFigure 5.3: Model performance analysis under varying conditionswith PBMC10K. a) The training and validation accuracies, as well asthe training loss change over epochs. The left y-axis is the accuracy, andthe right axis is the training loss. b) Validation accuracy with an increasedportion of label perturbation. c) Validation accuracy of ablation experimentwith increasing test ratio from 0.1 to 0.9.5.2 ProtoCloud learns informational latent spaceWe next evaluated the extent to which ProtoCloud incorporates biologicalvariance and batch information into its latent space through distinct focuses.The first half of the latent dimensions is dedicated to capturing primarycell type classification information, ensuring that cell identity is accuratelyrepresented. Conversely, the subsequent dimensions integrate extensive in-formation from other biological factors and technical variability, enhancingthe modelâs robustness. The UMAP visualization of the second half of thelatent space presents a mixture of all cell types, with additional clustersrepresenting other collected information.To assess the integration of additional information in the second half ofthe latent space, we utilized two datasets, PBMC30K and AtlasRGC, bothcontaining known batch information. Using cell type and batch annotations,we visualized different components of the latent spaces from the focus ondifferent training goals separately. The first component of the latent spaceshould capture cell type variation, while the second component incorporatesother factors, such as batch effects.We next used UMAP to show the preservation of cell type information andother information in different segments of the latent space. For the visual-335.2. ProtoCloud learns informational latent spaceFigure 5.4: Visualizing ProtoCloud latent embedding. (a-c) Visual-ization of PBMC30K cells in the latent space. a) UMAP of the first half oflatent space. Left: colored by cell type; Right: colored by batch. b) UMAPof the second half latent space. Left: colored by cell type; Right: colored bybatch. c) Direct visualization of the latent dimensions 11 and 12 that com-promises batch effects. (d-f) Visualization of AtlasRGC in the latent space.d) UMAP of the first half latent space and colored by cell type. e) UMAPof the second half latent space colored by batch. f) Direct visualization ofthe latent dimensions 13 and 14 that encode batch effects.345.2. ProtoCloud learns informational latent spaceization of the first-half latent space, we saw clear separations of cell typesin the UMAP (Figure 5.4.a, left). Within each cell type, we can see someminor separation of cells from the two batches (Figure 5.4.a, right). On thecontrary, by using the second-half latent space as inputs for UMAP analysis,cells were predominantly grouped by batch, exhibiting a mixture of cell-typecolors (Figure 5.4.b). The same pattern can also be concluded from the visu-alization of AtlasRGC (Figure 5.4.d, e). A particularly intriguing discoverywas the clear batch separation observed in certain dimensions of the latentspace in direct visualizations for both datasets (Figure 5.4.c, f), indicatingthat certain dimensions of the latent space may play a crucial role in cap-turing batch-specific variations. Furthermore, we quantified the extent ofbatch effect incorporation by calculating the batch entropy score [108]. Theresults confirm that batch effects are predominantly embedded in the laterhalf of the latent, thus preserving the integrity of cell-type information inthe initial segment of the latent space (Table 5.2). This batch distinctioncould be observed on training with full genes, but using HVGs resulting inbetter separation.Dataset Batches Num. GenesBatch Entropy Score1st-half latent 2nd-half latentPBMC30K 211145 0.251 0.0093000 0.319 0.030AtlasRGC 312070 0.508 0.1673000 0.598 0.089Table 5.2: ProtoCloud batch entropy score for different latent com-ponents on PBMC30K and AtlasRGC. The batch separation is betterwhen training with highly variable genes.Once the model is trained, we obtain the representative prototypes of eachclass for further investigation. From the correlation of prototypes, it canbe seen that prototypes of the same class are highly correlated and distinctfrom those of other classes (Figure 5.7.a).The UMAP visualization of the first half of latent dimensions yields distinctclusters corresponding to different cell types. Prototypes are highlightedwith larger points with black edges, which represent the âtypicalâ profilesof each cell type (Figure 5.4.a,d). The cell embedding is evenly distributedacross the prototypes for each class (Table B.1). A more balanced andcertain knowledge allocation is promoted with a two-step training process355.2. ProtoCloud learns informational latent spacethat adequately captures intra-class diversity (see Method for details). Eachprototype represents a substantial number of cells, ensuring no prototype re-mains underutilized or irrelevant. Consequently, it facilitates the discoveryof rare subtypes that might be overlooked, thereby offering deeper insightsinto the heterogeneity of cell populations. While the accuracy did not no-tably improve in the second half of the training, the modelâs confidence in itspredictions was enhanced during the latter training phase with a witness risein the prediction certainty (Figure 5.5). To demonstrate that the enhancedcertainty is due to the two-step training strategy rather than longer trainingepochs, an experiment is conducted using an end-to-end training strategywhere the loss function considers all terms simultaneously. The certaintyachieved after stage 1 of the two-step training exceeded the final certaintyobtained with the end-to-end approach (Figure B.2).Figure 5.5: Certainty by the end of each training stage, trained onPBMC10K.We are also interested in how the latent space of ProtoCloud would in-corporate granular cell types within broader categories, where the nuancedlandscape is induced by closely related subtypes. To this extent, we con-ducted an experiment with the lung dataset from the Tissue Stability CellAtlas (marked as TSCA Lung), which encompasses various cell types thatcan be hierarchically mapped into broader major cell types [105]. We trainedthe model with cell labels remapped to their major cell types. For example,mature and naive B cells were labelled as B cells; CD4+, CD8+, divid-ing T, and regulatory T cells were all mapped to T cells. ProtoCloud stillmaintained a tight regional representation of the pre-annotated subtypes.This was evident from the clear and distinct clustering of the different cell365.2. ProtoCloud learns informational latent spacesubtypes, which preserved the integrity and separation of these groups asannotated before dimensionality reduction. For example, a clear divisioncould be seen between naive and mature B cells in the visualization of Bcells (Figure 5.6.b), the same localization of five subtypes of Dendritic cellscould be seen in Figure 5.6.c. The dividing Macrophages and T cells wereclearly separated from the rest (Figure 5.6.d-e). These findings indicatethat ProtoCloud retained an inherent understanding of the subtle diversityamong the subtypes, reserving high fidelity to the original data structure ofcell populations even when trained on more generalized labels.Taken together, ProtoCloud effectively addresses technical variability whilemaintaining high-fidelity cell type identification, thereby enhancing the modelâsoverall utility and reliability in practical applications.375.2. ProtoCloud learns informational latent spaceFigure 5.6: Visualization of regional subtype in latent space. a)UMAP showing the distribution of major cell types in the TSCA lungdataset. (b-e) UMAP visualization of granular subtypes within the majorcell types: b) B cells subdivided into mature and na¨ıve B cell populations,385.3. ProtoCloud enhances explainability of prototypes with PRPrepresented by different shades. c) Dendritic cells (DC) differentiated into5 subtypes including DC 1, DC 2, Monocyte-dividing DCs, activated DCs,and plasmacytoid DCs. d) Macrophages are divided into 3 subtypes: Divid-ing Macrophages, MARCO-negative, and MARCO-positive Macrophages.e) T cells were further classified into CD4 T cells, Cytotoxic T cells, divid-ing T cells, and regulatory T cells.5.3 ProtoCloud enhances explainability ofprototypes with PRPIn the previous sections, we examined the accuracy and adaptability ofProtoCloud across multiple datasets by conducting benchmark comparisonsand delicately constructed experiments. We showed that ProtoCloudâs uniquedesign and integration of prototypes allow it to proficiently absorb the com-prehensive biological information within the latent space. However, thenext challenge is the explainability of the model and how to further under-stand and utilize the learned prototypes. Prototype relevance propagation(PRP) [14] is used to tackle these difficulties by offering an input-level rele-vance map of the prototypes. Rather than utilizing the prior knowledge ofmarker genes, PRP allows ProtoCloud to justify itself with the most rele-vant decision-making genes. The genes with high relevance scores from thegene-level relevance map of PRP are referred to as the highly relevant genes(HRGs).We ranked the top 20 HRGs identified by ProtoCloud from each class onPBMC30K [29] and compared them with the top 50 class-specific HVGs.The mapping exhibits a high degree of overlap, showing the prototypes cap-ture the intra-variance within each class (Figure 5.7.b). Another compres-sion is made between the prototype relevance map of Cytotoxic T cells, thereconstructed prototype in cell space and the average gene expression levelof the class (Figure 5.7.c). The prototypes are decoded back to the origi-nal observational gene expression space, allowing for direct interpretation oftheir gene expression. The reconstructed gene expression of a prototype ishighly aligned with the average expression level of the UMI counts of cellsfor that prototype, while not all highly expressed genes are emphasized bythe PRP. ProtoCloud extends beyond the reliance on highly expressed genesby up-scaling the relevance of cell type marker genes and down-scaling theopposite. In short, the prototype relevance mapping does not disproportion-ately emphasize genes with high expression levels.395.3. ProtoCloud enhances explainability of prototypes with PRPTo verify that the HRGs selected by ProtoCloud are relevant to specific celltypes and are indeed characteristic of those cell types, we compared the genescores between the known marker genes for each cell type and the top 20HRGs from each class. The result shows that leveraging PRP on ProtoCloudis effectively identifying cell-type relevant genes (Figure 5.7.d). The genescores from HRGs are higher for target cell types than other types and arecomparable to those from known marker genes (Figure B.1). A visualizationof B cell HRGs shows a high expressed level in their respective cell typesand are differentially expressed from other cell types (Figure 5.7.e). Evenfor two very similar cell types, such as mature B cells and naive B cells,the difference in their rank of HRGs can highlight these differences andassist in identifying type-specific genes. For example, TCL1A, CD79B andHLA-DM are more expressed in naive B cells, while LY9 is more specificfor mature B cells (Figure 5.7.e). While some marker genes are includedamong the top 20 HRGs, we also examined the relevance scores of otherknown marker genes. The dot plot of known marker geneâs relevance scoresdemonstrates that even if they are not top-ranked, their significance canstill be identified through the PRP relevance score (Figure B.4). Together,PRP not only aids in discovering novel marker genes but also validatesthe identification of existing marker genes. A dot plot displaying all celltypes top HRGs is generated, ordered by cell type and corresponding HRGs(Figure B.3). Both the relevance dot plot and the data expression dot plotexhibit a clear diagonal trend, confirming the specificity of the HRGs chosenby ProtoCloud.In the following sections, ProtoCloud will leverage its ability to uncoverand validate novel marker genes and transcriptional signatures via PRP onreal-world, large-scale datasets.405.3. ProtoCloud enhances explainability of prototypes with PRPFigure 5.7: Demonstrating the HRG found with PBMC30K. a)The prototype correlation plot. The prototypes of the same class are highly415.4. ProtoCloud yields consistent knowledge across datasetscorrelated but not correlated to those of other classes. b) The Venn diagramshows the overlap between top HRGs and HVGs for each class. Upper-left:B cells, upper-right: Cytotoxic T cells, lower-left: CD4 T cells, lower-right:CD16+ monocytes. c) Top: the relevance of the prototype retrieved fromprototype relevance propagation; Middle: the reconstructed gene expressionof the prototype; Bottom: the average gene expression level of the cells fromthe prototypeâs class. d) The radar plot of top HRGs gene score across celltypes. e) The violin plot of top 10 HRG expression for B cell HRGs acrosscell types. f) The gene expression violin plot of differentially ranked HVGsbetween mature and naive B cells.5.4 ProtoCloud yields consistent knowledgeacross datasetsDespite the models being trained independently on each dataset, there is asignificant overlap in the HRGs identified from the same cell types acrossdifferent datasets from various technologies and organs.To ensure gene and cell type consistency on datasets from different technolo-gies, we trained the models separately on the two sub-datasets of PBMC30Kdivided by batch with identical cell types and genes. The top HRGs for thesame cell type were almost identical in both sub-datasets (Table 5.3).GeneNameRankinPBMC1RankinPBMC2GeneNameRankinPBMC1RankinPBMC2MS4A1 0 3 CD83 9 1036CD79A 1 4 EZR 10 35HLA-DRA 2 1 TNFRSF13C 11 33IGHM 3 0 LINC00926 12 16HLA-DRB1 4 8 HLA-DPA1 13 7IGKC 5 2 HLA-DQA1 14 12BANK1 6 11 HLA-DPB1 15 6IGHD 7 10 CD79B 16 5IGLC2 8 14 RALGPS2 17 22Table 5.3: The ranking of learned HRGs of B cells from datasets ofdifferent batches. Ranked by the ascending order in PBMC1. Most topHRGs in PBMC1 are also marked as most relevant in PBMC2.425.5. ProtoCloud justifies misclassified cells via PRPAdditionally, to assess if ProtoCloud aligns with the criteria for self-explainingmodels that similar examples should produce consistent explanations, we ap-plied our model to the three datasets from the Tissue Stability Cell Atlas,which included samples from different organs: lung, esophagus, and spleen([105]). For shared cell types across these organs, the PRP identified highlysimilar HRGs, although the rank of these genes may vary. For example, theHRGs for mature B cells showed a high degree of overlap in the lung andspleen datasets, capturing typical markers like CD74, CD79A, MS4A1, andIGKC (Table 5.4). This alignment consistency also shows for the CD27+ Bcells in the esophagus dataset. These findings underscore the consistency ofProtoCloud in gene expression analysis across different datasets, confirmingits ability to uncover key biological signatures irrespective of the datasetorigin.Gene NameOrganLung Spleen EsophagusCD74 0 0 0MS4A1 1 16 33HLA-DRA 2 1 3IGKC 3 3 7CD79A 4 21 9EEF1A1 5 2 4Table 5.4: The ranking of learned HRGs of mature B cells fromdatasets of different organs. Three datasets of Tissue Stability Cell Atlasfrom different organs: lung, spleen and esophagus. As the TSCA oesophagusdoes not contain mature B cells class, CD27+ B cell is used instead.By examining the differences between top relevant genes across the pro-totypes for each class, we can identify genes that might reveal potentialsubtypes within a broader cell type.5.5 ProtoCloud justifies misclassified cells viaPRPWith the ProtoCloudâs decision-relevant genes (HRGs) from PRP, our nextgoal is to interpret and justify the modelâs decision for each cell. We employthe similarity score as an indicator of uncertainty in cell-type prediction.A positive correlation is observed between confusing cell types and their435.5. ProtoCloud justifies misclassified cells via PRPminimum distance to their prototypes: the greater the distance, the lowerthe predicted class accuracy. This distance is transformed into a similarityscore for classification. These findings indicate that similarity can serve as adirect measure of prediction certainty, aiding in the identification of outliersand misclassified cells. Moreover, the log-likelihood metric quantifies themodelâs confidence in its reconstruction, instilling a high degree of certaintyin the results.Figure 5.8: Workflow of how ProtoCloud detects mis-annotation.Step 1: The similarity threshold Each dot represents a cell; the class-specificthreshold is obtained from the certainty of training data. Step 2: Classifycell prediction as certain or ambiguous. Categorize the certain cells as oneswith a similarity score below the threshold that are treated as ambiguous(colored orange), otherwise certain (blue). Step 3: If the original annotationis available, further categorize mispredicted or mis-annotated by comparingthem to the original label. We tag cells as mis-annotated for cells that weare certain with prediction, but the prediction does not align with the label(colored green). We donât adjudicate for ambiguous cells (colored red) andstill mark them as mispredicted.ProtoCloud classifies its predictions as either certain or ambiguous basedon the similarity score of each cell to the predicted class prototype in thelatent space (Figure 5.8). Cells that surpass the certainty threshold areconsidered as certain, indicating a high confidence level in these predictionsand suggesting that the modelâs decisions can be trusted. In contrast, cellsbelow this threshold are categorized under ambiguous predictions, signallinga need for cautious interpretation. On top of that, distinct types of deviancyclassification are identified by ProtoCloud, characterized by discrepancies inthe cellâs similarity score and misalignment between the relevant or markergenes of the predicted and annotated labels (Figure 5.8). These complexscenarios are further addressed by ProtoCloud, which not only justifies its445.5. ProtoCloud justifies misclassified cells via PRPdecisions but can also be used to identify mis-annotated cells, enhancing thequality of the ground-truth label.To facilitate interpretations and application, we address each scenario throughvarious approaches and provide explanatory examples with the PBMC30Kdataset. The major re-annotated pairs are summarized in Figure 5.9, andwe support our findings using known cell markers.Type 1: Mis-annotated: ProtoCloud confidently predicts a cell type thatdiffers from the datasetâs original annotation. In such situations, these cellsshould be marked as mis-annotated and reannotated with ProtoCloudâs pre-diction. The gene expression of HRGs identified by PRP shows high consis-tency with the predicted cell type rather than the original annotation. Thissuggests a high probability of inaccuracies in the datasetâs annotations. Themis-annotated cells highly expressed the shared common HRGs between theoriginal labelled cell type and the predicted cell type. However, those cellsexhibit higher expression levels for the predicted cell type-specific HRGs andlower levels for those associated with the original labels. For the examples,a Type 1 error should express a similar expression level to the predicted celltype (second row) instead of the original (first row).Type 2: Ambiguous Classification: ProtoCloud identifies a portion ofcells as ambiguous, indicating uncertainty in the prediction. The labelsof mispredicted cells in this category will not be updated. The embeddingdistances to the original label prototype and the predicted prototype are veryclose, reflected in only a slight difference in similarity scores. Although thesimilarity score might align more closely with the predicted label, our modellacks sufficient evidence from decision-making genes to make a definitiveprediction and thus labels these cases as ambiguous. There is no distinctflavored cell type by the expression of top HRGs, the accumulation of theabundant genes relevance is overwhelming for the decision. An explanationfor this is that while our model includes trained prototypes for all knowncell types, the existing labels may not cover every cell type present in theactual data. Additionally, cells undergoing differentiation or exhibiting highsimilarity may not have prominent enough gene expression to be distinctlyrepresented by a single type.Type 3: Low-quality Cells: Doublets and low-quality cells are the maincomponents of this type. This type of error has a similar behaviour as Type2 error, where the cells are marked ambiguous. As samples of this errorinclude considerable data noise, we will not specifically address them.455.5. ProtoCloud justifies misclassified cells via PRPFigure 5.9: Dot plot of identified Type1 and Type2 error cells. Therows from top to bottom, respectively, are original annotation, predictedcell type, Type 1 and Type 2 error. a) CD4+ T cells VS Cytotoxic T cells.b) Dendritic cell VS CD14+ monocytes. c) Cytotoxic T cells VS Naturalkiller cells. Type 1 error dot plot should align more with the predicted celltype than the original.46Chapter 6ApplicationBuilding on previous findings, this chapter will explore how ProtoCloud canbe utilized to dissect intricate biological processes and uncover key molecularmechanisms in challenging disease contexts. We will now apply the model totwo complex disease datasets: time-coursed optic nerve crush in Section 6.1,and allergic inflammatory disease eosinophilic esophagitis in Section 6.2.6.1 Annotate optic nerve crush data withProtoCloud6.1.1 Transferring knowledge of time-course dataTime-coursed data are collected from different experiments under a con-trolled setting at designed time points. Investigating disease time-coursedata is key to comprehending the dynamic alterations in cellular expres-sion that occur throughout the progression of a disease. The mouse retinalganglion cells (RGCs) are first-born cells that are generated at birth andmost damages are irreversible [109]. Studying how RGCs respond followingoptic nerve crush (ONC) provides insights into neurodegenerative processesin a controlled setting. One major challenge is that the disease impact ishighly variable between different cell types over time. Additionally, manycells may be dying during disease/damage progression, â¼60% of RGCs diedbetween 3 and 7 days post-crush (dpc), and reached 80% at 14dpc. Typesare characterized into three survival groups, resilient, intermediate, and sus-ceptible, based on changes in relative survival frequency rates compared tothe control group.To demonstrate ProtoCloudâs effectiveness in label transfer on such time-coursed data, we experimented with optic nerve crush data collected at seventime points: 0, 0.5, 1, 2, 4, 7, and 14dpc [104]. Label transfer from the con-trol group (0dpc) to data at later time points is performed using only theinitial labels from the control group. The architecture of ProtoCloud lends476.1. Annotate optic nerve crush data with ProtoClouditself to a seamless transfer of prototype knowledge through the utilization ofshared genes when trained further with new datasets. By aligning the sharedgenes, our model could effectively transfer the prototype knowledge from thepre-trained model, thereby enriching the analysis of the new dataset withoutstarting from scratch. We trained ProtoCloud on one dataset, applied andcontinued training it on the dataset of the next time point, grouping thepredictions by certainty as detailed in Section 5.5. This incremental train-ing functionality not only expedited the training process but also ensuredconsistency in disease cell type identification across different time points.Figure 6.1: Trend of accuracy on continued training on time-coursedata. a) The first-round accuracy at different time points. Direct applica-tion of the model trained by the previous time-point data on the dataset.The overall accuracy is the weighted accuracy of both certain and ambigu-ous predictions. b) The accuracy of ambiguous cells after continued training.The model continued to train for a few additional epochs with cells of cer-tain prediction and improved the prediction for ambiguous cells.Initially, we trained the model on the control group data. The applied pre-dictions were divided into certain and ambiguous categories, and the overallaccuracy was weighted by the accuracy of these classifications. The modelwas then further trained on the subsequent time point using certain predic-tions as training labels for a few epochs, while ambiguous cells were left outof the training data and used as a reference for validation. By repeating theabove process, ProtoCloud retained and built upon the historical informa-tion, adapting to nuanced changes over time by progressive training withsubsequent data. The accuracy of certain predictions was above 94% fortime points before 3dpc and remained above 70% even through 14dpc (Fig-ure 6.1.a). The continued training resulted in a 5-15% increase in accuracyfor these difficult samples (Figure 6.1.b).486.1. Annotate optic nerve crush data with ProtoCloud496.1. Annotate optic nerve crush data with ProtoCloudFigure 6.2: UMAP visualization of time-course data withProtoCloud. Certainly predicted cells from both the initial applicationand after continued training were plotted to visualize the results. Proto-types are in dots with black edges. Legends are ordered by survival group:from most resilient (C43) to most susceptible (C28). a) Control group data.b-e) 0.5-14dpc. An exponential growth of unassigned cells can be seen after4dpc, colored by grey points.6.1.2 Identifying unassigned cellsThe proportion of âunassignedâ cells increases over time in the original la-bels, particularly after 4dpc, where it abruptly rises to approximately 23%.ProtoCloud confidently classifies one-third to one-half of these previouslyunassigned cells (Table A.1). A significant portion of these predictions is at-tributed to the W3-like cell type C1, which also expresses the known markerTusc5 (Figure 6.2.e, 6.3.b-d). C1 was classified as a susceptible type with thesurvival rate dropping to 1.2% at 14dpc in the data annotation. However,our prediction assigns C1 a relative survival frequency of 5.7%, suggestingthat it should be considered an intermediate survival type according to theranking in the original study. Another major type observed after 7dpc inunassigned cells is a novel resilient type C10, which is transcriptionally closeto C24 and D/VooDSGCs (C16) [104]. The known marker Gpr88 is highlyexpressed in the control group, while not abundant in later phases (Figure6.4). This showcases the ProtoCloudâs robust ability to accurately classifyand adapt to complex, evolving datasets.506.1. Annotate optic nerve crush data with ProtoCloudFigure 6.3: Dot plot of T5-RGCs over time. All of these types shouldexpress the marker Tusc5. The shared and different HRGs are comparisonsbetween C1 W3L1 and C13 W3L2. a) Gene expression of HRGs at 0dpc.(b-d) Gene expression of HRGs at 4, 7, and 14dpc. The original unassignedcells predicted as C1 W3L1 are added to the class (Pred C1).516.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.4: Dot plot of ooDSGCs-RGCs over time. C10 and C24 ex-pressing Gpr88 and Fam19a4, respectively. The shared and different HRGsare comparisons between C10 and C24. a) Gene expression of HRGs at0dpc. b) Gene expression of HRGs at 7dpc. The original unassigned cellspredicted as C10 are added to the class (Pred C10).6.2 Exploring of eosinophilic esophagitis altasusing ProtoCloud6.2.1 Findings in the eosinophilic esophagitis atlasWe applied ProtoCloud to scRNA-seq data from the challenging contextof individuals with eosinophilic esophagitis (EoE). Regarding the referencedataset, the recently published esophagus cell atlas (AtlasEoE) [107] encom-passes 421,312 cells from 22 donors across three disease statuses: healthy,remission, and active). This dataset reveals compositional and cell-intrinsicchanges in numerous cell types associated with EoE.Our model achieved a notable overall validation accuracy of 94.6% and aMacro F1 score of 0.883 on this immense dataset comprising 60 prevalentcell types (n = 393,763) and 12 rare cell types (n = 1,215). Even with a smallnumber of rare cells in the validation set, the model reaches 97% accuracyin identifying these. The goal is to transfer this granular annotation to twoother EoE datasets with various disease conditions and technical platforms,aiming for finer resolution and facilitating comparisons between differentstudies.6.2.2 Knowledge transfer across disease conditionsThe Clevenger et al. dataset profiled 151,519 cells. The authors focused onthe epithelial cells, which comprise 86.83% of the dataset [110]. The cellportions in the original study are annotated by marker genes and roughly526.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.5: Pie plot of predicted cell types in Clevenger et al.dataset. The epithelium cells include quiescent basal cells, basal cells,suprabasal and apical cells.divided into eight major cell populations covering three disease conditions.EoE is characterized by hyperproliferation of the epithelium cells, includingquiescent basal cells, basal cells, suprabasal and apical cells. While theoriginal labelling was not available, our predictions identified 87.7% of thecells as epithelial cells, closely matching the published percentage (Figure6.5.With our refined annotations of the Clevenger et al. dataset, we quanti-fied cell composition shifts across the three disease conditions. The originalstudy reported enrichment of plasma B cells and Mast cells in active EoEcompared to healthy donors. Our refined mapping identified that plasmacells (IgA+, IgG+, and cycling plasma cells) are dominant in EoE, whilememory B cells are evenly distributed (Figure 6.6.a). Mast cells are signifi-cantly outnumbered in EoE patients (Figure 6.6.c). Additionally, we verifiedprominent counts of ILC2 (90%), Th2 (96%), ALOX15+ macrophages (76%)and SEPP1+ macrophages (56%) in EoE patients, whereas apical cells weremore frequent (59%) in healthy donors [107, 111, 112] (Figure 6.6.b,d, TableB.2). The Clevenger et al. dataset also included four donors with gastroe-sophageal reflux disease (GERD) for similar symptoms. We observed anenrichment of certain inflammatory T cells (Th17, MAIT, CD4+ Trm) inGERD patients (Figure 6.6.d, Table B.2).536.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.6: Bar plot of cell portion across health conditions of labeltransfer for Clevenger et al. The prevalent subsets include a) plasma cellsubsets, b) myeloid (monocyte, macrophage, DC) subsets, c) the eosinophiland two mast cell subsets, d) T/NK/lymphocyte subsets, e) stromal cellsubsets,6.2.3 Refining cell type annotationThe second target EoE dataset from Morgan et al. includes 14,242 immuneand non-immune esophageal cells, identifying remove identifying includingeight major cell types and six T cell subsets from active EoE and remis-sion patients across two disease statuses [112]. By applying the pre-trainedProtoCloud, we mapped the originally identified eight major cell types totheir respective subcategories, including several newly identified cell types(Figure 6.7.a). The breakdown visualization of prevalent cell subsets showsdistinct boundaries between different cell types (Figure 6.7.b,c).546.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.7: Label transfer from AtlasEoE to Morgan et al. a)Annotation-transfer mapping heatmap from AtlasEoE (K=72) to Morganet al. (K=8). b-c. The UMAP projection of Morgan et al. in ProtoCloudlatent space. b) UMAP colored by major compartments. c) BreakdownUMAP of 60 prevalent cell subtypes.Apical cells with a higher portion in healthy donors, are the only epithelialcells that are more abundant in diseased than remission EoE condition (Fig-556.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.8: Discoveries with HRGs in EoE cell subsets. a) Bar plot ofepithelial cells composition between conditions. b) HRG expression level boxplot colored by two diagnoses. Whiskers include all cells outboundary of thelower or upper quartile. c) Violin plot of CD4+ T cells and inflammatory-related genes.ure 6.6.a, 6.8.a). Among the top HRGs between these two disease states,FTH1 stands out, with significantly higher expression in Apical cells dur-ing the diseased state compared to remission (Figure 6.8.b). T helper type2 (Th2) cells were enriched in the active state with over-expressed genesPLA2G16, PTGS2, HPGDS [112]. Our predicted Th2 cells also differen-tially expressed these genes with EoE inflammatory response genes IL-13,IL-4, and IL-5 in the annotation transfer (Figure 6.8.c). The original datasetnotes the absence of rare, tissue-resident ILC2s [112]; however, we identifieda few of these cells (expressing marker genes KLRB1 and IL-17 RB [113])in our refined annotation.Beyond our accurate annotations and their alignment with existing EoEknowledge, the ProtoCloud interpretability can be further leveraged. TheHRGs highlight genes indicative of specific cell types correlating with dis-ease activity and abundance. For example, ProtoCloud accurately identifiesMMP12 and ALOX15 as key relevant genes for ALOX15+ macrophages(with relevance score r of 0.615 and 0.267, respectively) but not for oth-566.2. Exploring of eosinophilic esophagitis altas using ProtoCloudFigure 6.9: Discoveries with HRGs in EoE cell subsets. a) Expressionlevel violin plot of differentially relevant genes for macrophages. b) Expres-sion level violin plot of newly discovered cycling genes across all cycling cells.ers (r < 0.03) (Figure 6.9.a). C1QA, which should be highly expressed inM1 and M2 macrophages [114], showed high relevance in all but SPP1+macrophages (r â 0). We also identified several cell-cycling-related genes,TUBA1B, HMGB2, TUBB, H2AFZ and KIAA0101, commonly expressedin the cycling cells (Figure 6.9.b).These findings showcase ProtoCloudâs superiority in label transfer and itspotential to assist in the differentiation of intrinsically subtle cell types.57Chapter 7DiscussionOur study demonstrates that the proposed model, ProtoCloud, a self-explainingprototypical generative model, provides interpretable and accurate cell typeclassification in scRNA-seq data. By integrating variational autoencoderswith prototypical networks, our model effectively captures cell type-specificlatent representations, demonstrating robustness in handling variations incell-type independent gene expressions while retaining the essence of proto-type representations for interpretation.Compared to other non-explainable counterparts, ProtoCloud offers supe-rior accuracy without sacrificing performance. Our analysis confirmed therobustness and reliability of the learned prototypes across different sequenc-ing techniques, organs, and disease states. By leveraging a set of prototypesderived from reference data, the model could effectively and flexibly map thetarget dataset to these reference prototypes, facilitating robust annotationof cell types. Regarding efficiency, ProtoCloud has a high training conver-gence speed and leverages PRP to provide instant, gene-level explanations,which is particularly advantageous in scenarios involving the evolving sizeof datasets. ProtoCloud is able to dissect the batch effect to some extentwithout prior knowledge.When it comes to explainability, ProtoCloud employs PRP to provide im-mediate, gene-level explanations once the model is trained. We introducedPRP as a powerful method for explaining individual prototypes in cell typeclassification, potentially revealing new and unknown biomarkers. The gran-ularity of LRP is advantageous in the context of scRNA-seq as it reveals thedecision-sensitive genes. We have shown the differentially ranked HRGs havethe potential to aid identification in closely related cell types with severalreal-world intrinsic datasets.One thing to keep in consideration is that, although PRP is a promisingtool for understanding model decisions, it primarily highlights key genescontributing to cell type classification without dissecting the rationality of58Chapter 7. Discussionunderlying biological or environmental interactions. While we identifiedHRGs for certain cell types, the causal relationships between genes and celltypes remain unknown and require further systematic investigation.There are several thoughts on the future direction of this work. First,there could be an automatic flexible adaptiveness to the number of pro-totypes. The number of prototypes for each class is predefined and fixedthroughout the training process. The intra-variability between prototypesfor rare cell types could be small and make abundant computational re-sources, while for major cell types, more prototypes might capture sub-type representations. Another avenue for future research is the extensionof modalities and the integration of multi-modality data, such as single-cell transposase-accessible chromatin with sequencing (scATAC-seq) data.An early experiment has shown ProtoCloud could have comparable per-formance on single-cell assay for transposase-accessible chromatin with se-quencing (scATAC-seq) data. We hope the explainability can be ultimatelyextended across different modalities, allowing for a more comprehensive un-derstanding of cellular processes.59Bibliography[1] Byungjin Hwang, Ji Hyun Lee, and Duhee Bang. Single-cell rna se-quencing technologies and bioinformatics pipelines. Experimental &molecular medicine, 50(8):1â14, 2018.[2] Hongkui Zeng. What is a cell type and how to define it? Cell,185(15):2739â2755, 2022.[3] Malte D Luecken and Fabian J Theis. Current best practices insingle-cell rna-seq analysis: a tutorial. Molecular systems biology,15(6):e8746, 2019.[4] Romain Lopez, Jeffrey Regier, Michael B Cole, Michael I Jordan, andNir Yosef. Deep generative modeling for single-cell transcriptomics.Nature methods, 15(12):1053â1058, 2018.[5] Chenling Xu, Romain Lopez, Edouard Mehlman, Jeffrey Regier,Michael I Jordan, and Nir Yosef. Probabilistic harmonization andannotation of single-cell transcriptomics data with deep generativemodels. Molecular systems biology, 17(1):e9620, 2021.[6] C Domı´nguez Conde, C Xu, LB Jarvis, DB Rainbow, SB Wells,T Gomes, SK Howlett, O Suchanek, K Polanski, HW King, et al.Cross-tissue immune cell analysis reveals tissue-specific features in hu-mans. Science, 376(6594):eabl5197, 2022.[7] Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang,Junzhou Huang, Hui Lu, and Jianhua Yao. scbert as a large-scalepretrained deep language model for cell type annotation of single-cellrna-seq data. Nature Machine Intelligence, 4(10):852â866, 2022.[8] Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo,Nan Duan, and Bo Wang. scgpt: toward building a foundation modelfor single-cell multi-omics using generative ai. Nature Methods, pages1â11, 2024.60Bibliography[9] Hongzhi Wen, Wenzhuo Tang, Xinnan Dai, Jiayuan Ding, Wei Jin,Yuying Xie, and Jiliang Tang. Cellplm: pre-training of cell languagemodel beyond single cells. bioRxiv, pages 2023â10, 2023.[10] Cynthia Rudin. Stop explaining black box machine learning modelsfor high stakes decisions and use interpretable models instead. Naturemachine intelligence, 1(5):206â215, 2019.[11] Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Se-menova, and Chudi Zhong. Interpretable machine learning: Funda-mental principles and 10 grand challenges. Statistic Surveys, 16:1â85,2022.[12] David Alvarez Melis and Tommi Jaakkola. Towards robust inter-pretability with self-explaining neural networks. Advances in neuralinformation processing systems, 31, 2018.[13] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin,and Jonathan K Su. This looks like that: deep learning for inter-pretable image recognition. Advances in neural information processingsystems, 32, 2019.[14] Srishti Gautam, Marina M.-C. Ho¨hne, Stine Hansen, Robert Jenssen,and Michael Kampffmeyer. This looks more like that: Enhancingself-explaining models by prototypical relevance propagation. PatternRecognition, 136:109172, 2023.[15] Srishti Gautam, Ahcene Boubekki, Stine Hansen, Suaiba Salahuddin,Robert Jenssen, Marina Ho¨hne, and Michael Kampffmeyer. Proto-vae: A trustworthy self-explainable prototypical variational model.Advances in Neural Information Processing Systems, 35:17940â17952,2022.[16] Yael Baran, Akhiad Bercovich, Arnau Sebe-Pedros, Yaniv Lubling,Amir Giladi, Elad Chomsky, Zohar Meir, Michael Hoichman, AviezerLifshitz, and Amos Tanay. Metacell: analysis of single-cell rna-seqdata using k-nn graph partitions. Genome biology, 20:1â19, 2019.[17] Sebastian Bach, Alexander Binder, Gre´goire Montavon, FrederickKlauschen, Klaus-Robert Mu¨ller, and Wojciech Samek. On pixel-wiseexplanations for non-linear classifier decisions by layer-wise relevancepropagation. PloS one, 10(7):e0130140, 2015.61Bibliography[18] Greg Gibson. The environmental contribution to gene expression pro-files. Nature reviews genetics, 9(8):575â581, 2008.[19] William Cookson, Liming Liang, Gonc¸alo Abecasis, Miriam Moffatt,and Mark Lathrop. Mapping complex disease traits with global geneexpression. Nature Reviews Genetics, 10(3):184â194, 2009.[20] Ulrich M Zanger and Matthias Schwab. Cytochrome p450 enzymes indrug metabolism: regulation of gene expression, enzyme activities, andimpact of genetic variation. Pharmacology & therapeutics, 138(1):103â141, 2013.[21] Martin E Young. The circadian clock within the heart: potentialinfluence on myocardial gene expression, metabolism, and function.American Journal of Physiology-Heart and Circulatory Physiology,290(1):H1âH16, 2006.[22] Grace XY Zheng, Jessica M Terry, Phillip Belgrader, Paul Ryvkin,Zachary W Bent, Ryan Wilson, Solongo B Ziraldo, Tobias D Wheeler,Geoff P McDermott, Junjie Zhu, et al. Massively parallel digital tran-scriptional profiling of single cells. Nature communications, 8(1):14049,2017.[23] Xinmin Li and Cun-Yu Wang. From bulk, single-cell to spatial rnasequencing. International journal of oral science, 13(1):36, 2021.[24] Simone Picelli, AËsa K Bjo¨rklund, Omid R Faridani, Sven Sagasser,Go¨sta Winberg, and Rickard Sandberg. Smart-seq2 for sensitivefull-length transcriptome profiling in single cells. Nature methods,10(11):1096â1098, 2013.[25] Tamar Hashimshony, Naftalie Senderovich, Gal Avital, AgnesKlochendler, Yaron De Leeuw, Leon Anavy, Dave Gennert, ShuqiangLi, Kenneth J Livak, Orit Rozenblatt-Rosen, et al. Cel-seq2: sensitivehighly-multiplexed single-cell rna-seq. Genome biology, 17:1â7, 2016.[26] Evan Z Macosko, Anindita Basu, Rahul Satija, James Nemesh,Karthik Shekhar, Melissa Goldman, Itay Tirosh, Allison R Bialas,Nolan Kamitaki, Emily M Martersteck, et al. Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets.Cell, 161(5):1202â1214, 2015.62Bibliography[27] Alexander B Rosenberg, Charles M Roco, Richard A Muscat, AnnaKuchina, Paul Sample, Zizhen Yao, Lucas T Graybuck, David J Peeler,Sumit Mukherjee, Wei Chen, et al. Single-cell profiling of the devel-oping mouse brain and spinal cord with split-pool barcoding. Science,360(6385):176â182, 2018.[28] Paul Datlinger, Andre´ F Rendeiro, Thorina Boenke, Martin Senekow-itsch, Thomas Krausgruber, Daniele Barreca, and Christoph Bock.Ultra-high-throughput single-cell rna sequencing and perturbationscreening with combinatorial fluidic indexing. Nature methods,18(6):635â642, 2021.[29] Jiarui Ding, Xian Adiconis, Sean K Simmons, Monika S Kowalczyk,Cynthia C Hession, Nemanja D Marjanovic, Travis K Hughes, Marc HWadsworth, Tyler Burks, Lan T Nguyen, et al. Systematic compari-son of single-cell and single-nucleus rna-sequencing methods. Naturebiotechnology, 38(6):737â746, 2020.[30] Pierre-Luc Germain, Aaron Lun, Carlos Garcia Meixide, Will Mac-nair, and Mark D Robinson. Doublet identification in single-cell se-quencing data using scdblfinder. F1000Research, 10, 2021.[31] Geng Chen, Baitang Ning, and Tieliu Shi. Single-cell rna-seq technolo-gies and related computational data analysis. Frontiers in genetics,10:317, 2019.[32] Simon Anders and Wolfgang Huber. Differential expression analysisfor sequence count data. Nature Precedings, pages 1â1, 2010.[33] Vladimir Yu Kiselev, Tallulah S Andrews, and Martin Hemberg. Chal-lenges in unsupervised clustering of single-cell rna-seq data. NatureReviews Genetics, 20(5):273â282, 2019.[34] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data usingt-sne. Journal of machine learning research, 9(11), 2008.[35] Leland McInnes, John Healy, and James Melville. Umap: Uniformmanifold approximation and projection for dimension reduction. arXivpreprint arXiv:1802.03426, 2018.[36] Aviv Regev, Sarah A Teichmann, Eric S Lander, Ido Amit, ChristopheBenoist, Ewan Birney, Bernd Bodenmiller, Peter Campbell, Piero63BibliographyCarninci, Menna Clatworthy, Hans Clevers, Bart Deplancke, Ian Dun-ham, James Eberwine, Avigdor Eldar, Katherine Fitzgerald, PaulFlicek, Julien Gagneur, Yoav Gilad, Pierre Greve, Peter Hamilton,William B Hamilton, Robert Hinch, David A Hume, Detlev Arendt,Seungil Kim, Antonio Lanzavecchia, Sten Linnarsson, Clement Loy,Miriam Merad, James Nichols, Marloes Nry, Nikolaus Rajewsky,Glenn Randall, Mark Robinson, Orit Rozenblatt-Rosen, Rahul Satija,Ton N Schumacher, Alex K Shalek, Ehud Shapiro, Partha Sharma,John W Shin, Oliver Stegle, Michael Stratton, Michael J T Stubbing-ton, Daniel Turner, Alexander Van Oudenaarden, Andreas Wagner,Fiona M Watt, Jonathan S Weissman, Barbara Wold, Ramnik Xavier,Nir Yosef, Philip Awadalla, Wei Peng, Jun Wang, John C Marioni,and The Human Cell Atlas Consortium. The human cell atlas. eLife,6:e27041, 2017.[37] Nicholas Schaum, Jim Karkanias, Norma F Neff, Andrew P May,Stephen R Quake, Tony Wyss-Coray, Spyros Darmanis, Joshua Bat-son, Olga Botvinnik, Michelle B Chen, et al. Single-cell transcrip-tomics of 20 mouse organs creates a tabula muris: The tabula murisconsortium. Nature, 562(7727):367, 2018.[38] Hongjie Li, Jasper Janssens, Maxime De Waegeneer, Sai SarojaKolluru, Kristofer Davie, Vincent Gardeux, Wouter Saelens, Fab-rice PA David, Maria Brbic´, Katina Spanier, et al. Fly cell atlas:A single-nucleus transcriptomic atlas of the adult fruit fly. Science,375(6584):eabk2432, 2022.[39] Junyue Cao, Jonathan S Packer, Vijay Ramani, Darren A Cusanovich,Chau Huynh, Riza Daza, Xiaojie Qiu, Choli Lee, Scott N Furlan,Frank J Steemers, et al. Comprehensive single-cell transcriptional pro-filing of a multicellular organism. Science, 357(6352):661â667, 2017.[40] Seung Y Rhee, Kenneth D Birnbaum, and David W Ehrhardt. To-wards building a plant cell atlas. Trends in plant science, 24(4):303â310, 2019.[41] Vincent A Traag, LudoWaltman, and Nees Jan Van Eck. From louvainto leiden: guaranteeing well-connected communities. Scientific reports,9(1):1â12, 2019.[42] Congxue Hu, Tengyue Li, Yingqi Xu, Xinxin Zhang, Feng Li, JingBai, Jing Chen, Wenqi Jiang, Kaiyue Yang, Qi Ou, et al. Cellmarker64Bibliography2.0: an updated database of manually curated cell markers in hu-man/mouse and web tools based on scrna-seq data. Nucleic acidsresearch, 51(D1):D870âD876, 2023.[43] Oscar Franze´n, Li-Ming Gan, and Johan LM Bjo¨rkegren. Panglaodb:a web server for exploration of mouse and human single-cell rna se-quencing data. Database, 2019:baz046, 2019.[44] Jeffrey T Leek, Robert B Scharpf, He´ctor Corrada Bravo, David Sim-cha, Benjamin Langmead, W Evan Johnson, Donald Geman, KeithBaggerly, and Rafael A Irizarry. Tackling the widespread and criti-cal impact of batch effects in high-throughput data. Nature ReviewsGenetics, 11(10):733â739, 2010.[45] Huajian Fang, Guillaume Carbajal, Stefan Wermter, and Timo Gerk-mann. Variational autoencoder for speech enhancement with a noise-aware encoder. In ICASSP 2021-2021 IEEE international conferenceon acoustics, speech and signal processing (ICASSP), pages 676â680.IEEE, 2021.[46] Xianxu Hou, Linlin Shen, Ke Sun, and Guoping Qiu. Deep featureconsistent variational autoencoder. In 2017 IEEE winter conferenceon applications of computer vision (WACV), pages 1133â1141. IEEE,2017.[47] Charlie Nash and Christopher KI Williams. The shape variationalautoencoder: A deep generative model of part-segmented 3d objects.In Computer Graphics Forum, volume 36, pages 1â12. Wiley OnlineLibrary, 2017.[48] Ruoqi Wei and Ausif Mahmood. Recent advances in variational au-toencoders with representation learning for biomedical informatics: Asurvey. Ieee Access, 9:4939â4956, 2020.[49] Jiarui Ding, Anne Condon, and Sohrab P Shah. Interpretable dimen-sionality reduction of single cell transcriptome data with deep gener-ative models. Nature communications, 9(1):2002, 2018.[50] Dongfang Wang and Jin Gu. Vasc: dimension reduction and visu-alization of single-cell rna-seq data by deep variational autoencoder.Genomics, Proteomics and Bioinformatics, 16(5):320â331, 2018.65Bibliography[51] Jiarui Ding and Aviv Regev. Deep generative model embedding ofsingle-cell rna-seq profiles on hyperspheres and hyperbolic spaces. Na-ture communications, 12(1):2554, 2021.[52] Christopher Heje Grønbech, Maximillian Fornitz Vording, Pascal NTimshel, Casper Kaae Sønderby, Tune H Pers, and Ole Winther. scvae:variational auto-encoders for single-cell gene expression data. Bioin-formatics, 36(16):4415â4422, 2020.[53] Michael I Love, Wolfgang Huber, and Simon Anders. Moderated es-timation of fold change and dispersion for rna-seq data with deseq2.Genome biology, 15:1â21, 2014.[54] Chen Jia. Kinetic foundation of the zero-inflated negative binomialmodel for single-cell rna sequencing data. SIAM Journal on AppliedMathematics, 80(3):1336â1355, 2020.[55] Christoph Hafemeister and Rahul Satija. Normalization and variancestabilization of single-cell rna-seq data using regularized negative bi-nomial regression. Genome biology, 20(1):296, 2019.[56] Peter V Kharchenko, Lev Silberstein, and David T Scadden. Bayesianapproach to single-cell differential expression analysis. Nature methods,11(7):740â742, 2014.[57] Federico Agostinis, Chiara Romualdi, Gabriele Sales, and DavideRisso. Newwave: a scalable r/bioconductor package for the dimen-sionality reduction and batch effect removal of single-cell rna-seq data.Bioinformatics, 38(9):2648â2650, 2022.[58] Finale Doshi-Velez and Been Kim. Towards a rigorous science of in-terpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.[59] Alejandro Barredo Arrieta, Natalia Dı´az-Rodr´ıguez, Javier Del Ser,Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garc´ıa,Sergio Gil-Lo´pez, Daniel Molina, Richard Benjamins, et al. Explain-able artificial intelligence (xai): Concepts, taxonomies, opportunitiesand challenges toward responsible ai. Information fusion, 58:82â115,2020.[60] Hajar Hakkoum, Ibtissam Abnane, and Ali Idri. Interpretability inthe medical field: A systematic mapping and review study. AppliedSoft Computing, 117:108391, 2022.66Bibliography[61] P.H.N. Gill. Introduction to Machine Learning Interpretability.OâReilly Media, Incorporated, 2018.[62] Raul Rojas and Rau´l Rojas. The backpropagation algorithm. Neuralnetworks: a systematic introduction, pages 149â182, 1996.[63] Enyan Dai and Suhang Wang. Towards self-explainable graph neuralnetwork. In Proceedings of the 30th ACM International Conference onInformation & Knowledge Management, pages 302â311, 2021.[64] Jiaqi Wang, Huafeng Liu, Xinyue Wang, and Liping Jing. Inter-pretable image recognition by constructing transparent embeddingspace. In Proceedings of the IEEE/CVF international conference oncomputer vision, pages 895â904, 2021.[65] Jayneel Parekh, Pavlo Mozharovskyi, and Florence dâAlche´ Buc. Aframework to learn with interpretation. Advances in Neural Informa-tion Processing Systems, 34:24273â24285, 2021.[66] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. â why shouldi trust you?â explaining the predictions of any classifier. In Proceed-ings of the 22nd ACM SIGKDD international conference on knowledgediscovery and data mining, pages 1135â1144, 2016.[67] Scott M Lundberg and Su-In Lee. A unified approach to interpretingmodel predictions. Advances in neural information processing systems,30, 2017.[68] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors:High-precision model-agnostic explanations. In Proceedings of theAAAI conference on artificial intelligence, volume 32, 2018.[69] Davide Castelvecchi. Can we open the black box of ai? Nature News,538(7623):20, 2016.[70] Gre´goire Montavon, Wojciech Samek, and Klaus-Robert Mu¨ller.Methods for interpreting and understanding deep neural networks.Digital signal processing, 73:1â15, 2018.[71] Moritz Bo¨hle, Fabian Eitel, Martin Weygandt, and Kerstin Ritter.Layer-wise relevance propagation for explaining deep neural networkdecisions in mri-based alzheimerâs disease classification. Frontiers inaging neuroscience, 11:456892, 2019.67Bibliography[72] Mahbub Ul Alam, Jo´n Ru´nar Baldvinsson, and Yuxia Wang. Explor-ing lrp and grad-cam visualization to interpret multi-label-multi-classpathology prediction using chest radiography. In 2022 IEEE 35th in-ternational symposium on computer-based medical systems (CBMS),pages 258â263. IEEE, 2022.[73] Yinchong Yang, Volker Tresp, Marius Wunderle, and Peter AFasching. Explaining therapy predictions with layer-wise relevancepropagation in neural networks. In 2018 IEEE International Confer-ence on Healthcare Informatics (ICHI), pages 152â162. IEEE, 2018.[74] Fabian Eitel, Emily Soehler, Judith Bellmann-Strobl, Alexander UBrandt, Klemens Ruprecht, Rene´ M Giess, Joseph Kuchling, SusannaAsseyer, Martin Weygandt, John-Dylan Haynes, et al. Uncoveringconvolutional neural network decisions for diagnosing multiple sclero-sis on conventional mri using layer-wise relevance propagation. Neu-roImage: Clinical, 24:102003, 2019.[75] Ihsan Ullah, Andre Rios, Vaibhav Gala, and Susan Mckeever. Explain-ing deep learning models for tabular data using layer-wise relevancepropagation. Applied Sciences, 12(1):136, 2021.[76] Shenghao Li, Hui Guo, Simai Zhang, Yizhou Li, and Menglong Li.Attention-based deep clustering method for scrna-seq cell type identi-fication. PLOS Computational Biology, 19(11):e1011641, 2023.[77] Yuyao Zhai, Liang Chen, and Minghua Deng. Generalized cell typeannotation and discovery for single-cell rna-seq data. In Proceedingsof the AAAI Conference on Artificial Intelligence, volume 37, pages5402â5410, 2023.[78] Blaise Hanczar, Farida Zehraoui, Tina Issa, and Mathieu Arles. Bio-logical interpretation of deep neural network for phenotype predictionbased on gene expression. BMC bioinformatics, 21:1â18, 2020.[79] Bingbing Xie, Qin Jiang, Antonio Mora, and Xuri Li. Automatic celltype identification methods for single-cell rna sequencing. Computa-tional and Structural Biotechnology Journal, 19:5874â5887, 2021.[80] Allen W Zhang, Ciara OâFlanagan, Elizabeth A Chavez, Jamie LPLim, Nicholas Ceglia, Andrew McPherson, Matt Wiens, Pascale Wal-ters, Tim Chan, Brittany Hewitson, et al. Probabilistic cell-type as-68Bibliographysignment of single-cell rna-seq for tumor microenvironment profiling.Nature methods, 16(10):1007â1015, 2019.[81] Vladimir Yu Kiselev, Andrew Yiu, and Martin Hemberg. scmap: pro-jection of single-cell rna-seq data across data sets. Nature methods,15(5):359â362, 2018.[82] Ji-Hye Choi, Hye In Kim, and Hyun Goo Woo. sctyper: a compre-hensive pipeline for the cell typing analysis of single-cell rna-seq data.BMC bioinformatics, 21:1â8, 2020.[83] Rui Fu, Austin E Gillen, Ryan M Sheridan, Chengzhe Tian, MichelleDaya, Yue Hao, Jay R Hesselberth, and Kent A Riemondy. clustifyr:an r package for automated single-cell rna sequencing cluster classifi-cation. F1000Research, 9, 2020.[84] Yinghao Cao, Xiaoyue Wang, and Gongxin Peng. Scsa: a cell type an-notation tool for single-cell rna-seq data. Frontiers in genetics, 11:490,2020.[85] Feiyang Ma and Matteo Pellegrini. Actinn: automated identificationof cell types in single cell rna sequencing. Bioinformatics, 36(2):533â538, 2020.[86] Maria Brbic´, Marinka Zitnik, Sheng Wang, Angela O Pisco, Russ BAltman, Spyros Darmanis, and Jure Leskovec. Mars: discovering novelcell types across heterogeneous single-cell experiments. Nature meth-ods, 17(12):1200â1206, 2020.[87] Yuhan Hao, Stephanie Hao, Erica Andersen-Nissen, WilliamM. Mauck III, Shiwei Zheng, Andrew Butler, Maddie J. Lee, Aaron J.Wilk, Charlotte Darby, Michael Zagar, Paul Hoffman, Marlon Stoeck-ius, Efthymia Papalexi, Eleni P. Mimitou, Jaison Jain, Avi Srivastava,Tim Stuart, Lamar B. Fleming, Bertrand Yeung, Angela J. Rogers,Juliana M. McElrath, Catherine A. Blish, Raphael Gottardo, PeterSmibert, and Rahul Satija. Integrated analysis of multimodal single-cell data. Cell, 2021.[88] Yuval Lieberman, Lior Rokach, and Tal Shay. Castleâclassificationof single cells by transfer learning: harnessing the power of publiclyavailable single cell rna sequencing experiments to annotate new ex-periments. PloS one, 13(10):e0205499, 2018.69Bibliography[89] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networksfor few-shot learning. Advances in neural information processing sys-tems, 30, 2017.[90] Yanan Li and Donghui Wang. Zero-shot learning with generative la-tent prototype model. arXiv preprint arXiv:1705.09474, 2017.[91] Peter Hase, Chaofan Chen, Oscar Li, and Cynthia Rudin. Inter-pretable image recognition with hierarchical prototypes. In Proceed-ings of the AAAI Conference on Human Computation and Crowd-sourcing, volume 7, pages 32â40, 2019.[92] Dawid Rymarczyk, Lukasz Struski, Jacek Tabor, and BartoszZielin´ski. Protopshare: Prototype sharing for interpretable image clas-sification and similarity discovery. arXiv preprint arXiv:2011.14340,2020.[93] Meike Nauta, Ron Van Bree, and Christin Seifert. Neural prototypetrees for interpretable fine-grained image recognition. In Proceedings ofthe IEEE/CVF conference on computer vision and pattern recognition,pages 14933â14943, 2021.[94] James Lucas, George Tucker, Roger Grosse, and Mohammad Norouzi.Understanding posterior collapse in generative latent variable models;2019. In URL https://openreview. net/forum, 2019.[95] Xuejian Cui, Xiaoyang Chen, Zhen Li, Zijing Gao, Shengquan Chen,and Rui Jiang. Discrete latent embedding of single-cell chromatinaccessibility sequencing data for uncovering cell heterogeneity. NatureComputational Science, pages 1â14, 2024.[96] Diederik P Kingma and Max Welling. Auto-encoding variationalbayes. arXiv preprint arXiv:1312.6114, 2013.[97] Diederik P Kingma, Max Welling, et al. An introduction to varia-tional autoencoders. Foundations and Trends® in Machine Learning,12(4):307â392, 2019.[98] Valentine Svensson. Droplet scrna-seq is not zero-inflated. NatureBiotechnology, 38(2):147â150, 2020.[99] Hassaan Maan, Lin Zhang, Chengxin Yu, Michael Geuenich, Kieran RCampbell, and BoWang. The differential impacts of dataset imbalancein single-cell data integration. bioRxiv, pages 2022â10, 2022.70Bibliography[100] Kai Fischer. Relevance propagation with py-torch. https://kaifishr.github.io/2021/12/15/relevance-propagation-pytorch.html, 2021.[101] Sajid Ali, Tamer Abuhmed, Shaker El-Sappagh, Khan Muhammad,Jose M Alonso-Moral, Roberto Confalonieri, Riccardo Guidotti, JavierDel Ser, Natalia Dı´az-Rodr´ıguez, and Francisco Herrera. Explainableartificial intelligence (xai): What we know and what is left to attaintrustworthy artificial intelligence. Information fusion, 99:101805, 2023.[102] Adam Gayoso, Romain Lopez, Galen Xing, Pierre Boyeau, Valeh Vali-ollah Pour Amiri, Justin Hong, Katherine Wu, Michael Jayasuriya,Edouard Mehlman, Maxime Langevin, et al. A python library forprobabilistic analysis of single-cell omics data. Nature biotechnology,40(2):163â166, 2022.[103] Tuomas Oikarinen and Tsui-Wei Weng. Clip-dissect: Automatic de-scription of neuron representations in deep vision networks. arXivpreprint arXiv:2204.10965, 2022.[104] Nicholas M Tran, Karthik Shekhar, Irene E Whitney, Anne Jacobi, In-bal Benhar, Guosong Hong, Wenjun Yan, Xian Adiconis, McKinzie EArnold, Jung Min Lee, et al. Single-cell profiles of retinal ganglion cellsdiffering in resilience to injury reveal neuroprotective genes. Neuron,104(6):1039â1055, 2019.[105] E Madissoon, A Wilbrey-Clark, RJ Miragaia, K Saeb-Parsy, K Mah-bubani, N Georgakopoulos, P Harding, K Polanski, K Nowicki-Osuch,RC Fitzgerald, et al. Lung, spleen and oesophagus tissue remainsstable for scrnaseq in cold preservation. BioRxiv, page 741405, 2019.[106] C Domı´nguez Conde, C Xu, LB Jarvis, DB Rainbow, SB Wells,T Gomes, SK Howlett, O Suchanek, K Polanski, HW King, et al.Cross-tissue immune cell analysis reveals tissue-specific features in hu-mans. Science, 376(6594):eabl5197, 2022.[107] Jiarui Ding, John J Garber, Amiko Uchida, Ariel Lefkovith, Grace TCarter, Praveen Vimalathas, Lauren Canha, Michael Dougan, KyleStaller, Joseph Yarze, et al. An esophagus cell atlas reveals dynamicrewiring during active eosinophilic esophagitis and remission. NatureCommunications, 15(1):3344, 2024.71[108] Ruben Chazarra-Gil, Stijn van Dongen, Vladimir Yu Kiselev, andMartin Hemberg. Flexible comparison of batch correction methods forsingle-cell rna-seq using batchbench. Nucleic acids research, 49(7):e42âe42, 2021.[109] Kim Tuyen Nguyen-Ba-Charvet and Alexandra Rebsam. Neurogenesisand specification of retinal ganglion cells. International journal ofmolecular sciences, 21(2):451, 2020.[110] Margarette H Clevenger, Adam L Karami, Dustin A Carlson, Peter JKahrilas, Nirmala Gonsalves, John E Pandolfino, Deborah R Winter,Kelly A Whelan, and Marie-Pier Te´treault. Suprabasal cells retainprogenitor cell identity programs in eosinophilic esophagitisâdrivenbasal cell hyperplasia. JCI insight, 8(19), 2023.[111] Taylor A Doherty, Rachel Baum, Robert O Newbury, Tom Yang,Ranjan Dohil, Melissa Aquino, Ashmi Doshi, Hannah H Walford,Richard C Kurten, David H Broide, et al. Group 2 innate lympho-cytes (ilc2) are enriched in active eosinophilic esophagitis. Journal ofAllergy and Clinical Immunology, 136(3):792â794, 2015.[112] Duncan M Morgan, Bert Ruiter, Neal P Smith, Ang A Tu, Brinda Mo-nian, Brandon E Stone, Navneet Virk-Hundal, Qian Yuan, Wayne GShreffler, and J Christopher Love. Clonally expanded, gpr15-expressing pathogenic effector th2 cells are associated with eosinophilicesophagitis. Science immunology, 6(62):eabi5586, 2021.[113] Tetsuro Kobayashi and Kazuyo Moro. Tissue-specific diversity ofgroup 2 innate lymphoid cells in the skin. Frontiers in Immunology,13:885642, 2022.[114] Long-hao Chen, Jin-Fu Liu, Yan Lu, Xin-yu He, Chi Zhang, and Hong-hai Zhou. Complement c1q (c1qa, c1qb, and c1qc) may be a potentialprognostic factor and an index of tumor microenvironment remodelingin osteosarcoma. Frontiers in oncology, 11:642144, 2021.72Appendix AExperiment SettingHyper-parameterIn our experiments, we configured the model with the following hyper-parameters: a learning rate of 1 Ã 10â3, encoder of 3 hidden layers withdimensions of 1024, 512, and 256, and decoder of 2 hidden layers with di-mensions of 512 and 1024. The latent dimension was set to 20, and thenumber of prototypes per class was 6. Although ProtoCloud can still havesatisfying performance on fewer prototypes per class, we find that more pro-totypes help to capture the inner variations within each cell type and givea more reasonable PRP explanation. The model is trained for 100 epochs,and the two-step training process was initiated after 30 epochs. For trainingdata less than 100,000 cells, the batch size is 128; for any size larger thanthis, the batch size is set to 1024. The above hyper-parameters are set bydefault and are consistent for all experiments across the datasets (exceptfor demonstrations with specific targets specified). When dealing with rawUMI counts, the NB dispersion uses the target cell type dispersion. For nor-malized data or scATAC-seq input, the standard VAE setting with normaldistribution can be used.DatasetsThis section will list all the datasets used in this thesis. For data prepro-cessing, the mitochondrial and ribosomal coding genes are removed from alldatasets. The default filtering includes removing cells with counts less than1,000 and filtering genes with less than 500 total counts. All datasets gothrough unified filtering processes unless specified.The datasets used for benchmarking and experiment in Section 5 are listedbelow:â¢ PBMC10K The preprocessed PBMC dataset is downloaded from73Datasets[102] with 12,039 cells and 3,346 sampled genes. We retained theSeurat-generated labels containing 9 cell types. No filtering is appliedexcept for removing the unknown cells.â¢ PBMC30K This PBMC dataset is collected from two experiments,each with multiple scRNA-seq methods [29]. The original datasetconsists of 30,495 cells across 10 cell types. We removed the unknowncells in experiments and filtering, then used 22996 cells and top 3000HVGs for analysis.â¢ AtlasRGC The comprehensive atlas of mouse retina cells (RGC) inthe adult retina [104]. A total of 45 RGC subtypes are identified,ranging from 8.4% (C1) to 0.15% (C45). Unknown cells were excludedfrom the analysis, and the analysis focused on the 3,000 HVGs.â¢ TSCA Datasets - Lung, esophagus, and spleen It focuses onthree human primary tissues, each expected to exhibit different levelsof sensitivity to ischemia. The spleen, considered the most stable,contains 94,257 immune cells spanning 30 cell types. The esophagusmucosa includes 87,947 cells from 19 cell types, and the lung, identifiedas the least stable, has 57,020 cells across 28 cell types. For all threetissues, the 3,000 most variable genes were selected for analysis.â¢ ICA Cross-tissue Immune Cell Atlas dataset is a part of the HumanCell Atlas project [106]. In this study, researchers profiled more than300,000 individual immune cells. These cells were extracted from 16different tissues obtained from 12 deceased adult organ donors. Theoriginal cell identities were assigned using CellTypist into 43 cell sub-types. We selected the top 3,000 HVGs for experiments.â¢ AtlasEoE The esophageal cell atlas [107] consists of 421,312 scRNA-seq profiles. These samples were taken from 15 EoE patients (8 in ac-tive disease state and 7 in remission) and 7 healthy participants. Thedataset includes 60 prevalent cell types grouped into 6 categories (ep-ithelial cells, stromal cells, Monocytes/Mac/DC cells, plasma B cells,T/NK/ILC cells, and the others). Additionally, the atlas identifies 12rare cell subsets, comprising 1,215 cells in total. For this dataset, weuse the marker genes (G = 2956) provided by the author.For application, the ONC datasets are from the same publication as theAtlasRGC [104]: For prediction or continued training, only the shared genesfrom the pretrained model were utilized. Missing genes from the model arefilled with zeros. No further preprocessing is done for application data.74DatasetsDPC Num. of cells Unknown (%)0 (ctrl) 12062 00.5 13619 1.2%1 12478 3%2 10695 4.7%4 10599 23.8%7 8700 25.0%14 8456 17.2%Table A.1: RGC ONC datasets by 7 time points post-crush75Appendix BSupplementary Figures andTablesFigure B.1: The gene score of known marker genes for PBMC30K76Appendix B. Supplementary Figures and TablesCell type Prototype Cells0 6221 683B cell 2 6183 6184 5565 6630 6501 603CD14+ 2 490monocyte 3 5544 6065 6000 761 90CD16+ 2 109monocyte 3 1074 1375 113Cell type Prototype Cells0 9391 1016CD4+ 2 1085T cell 3 11134 10635 9360 11571 1140Cytotoxic 2 1181T cell 3 12324 11345 11490 641 52Dendritic 2 59cell 3 844 465 50Table B.1: Number of cells assigned to each prototype by similarity ofPBMC30K. Each prototype represents a similar number of cells in eachclass.Figure B.2: The prediction certainty on PBMC10K with end-to-end train-ing. Epoch 30 and 100 correspond to the end of Stages 1 and 2 in two-steptraining.77AppendixB.SupplementaryFiguresandTablesFigure B.3: PBMC10K HRG dotplot. a) The HRG relevance score dot plot. b) The HRG expression dot plot.78AppendixB.SupplementaryFiguresandTablesFigure B.4: AtlasEoE known marker genes PRP relevance score dot plot79Appendix B. Supplementary Figures and TablesCell Type Health Condition PercentageTh2 EoE 98.76%Plasma cell (IgA+) EoE 85.83%Mast cell EoE 81.71%Macrophage (ALOX15) EoE 75.89%NK (XCL1/CD160) EoE 60.34%Pericyte HC 58.97%Apical cell HC 56.63%MAIT (NCR3) GERD 56.45%CD4+ Trm (CCL5) GERD 56.03%B cell (memory) HC 55.91%Fibroblast HC 53.18%Langerhans cell HC 52.89%Arterial (GJA4) HC 51.97%Macrophage (SEPP1) EoE 51.72%Th17 GERD 51.06%cDC2B (CD1C) HC 50.57%Table B.2: Cell type distribution across different health conditions of EoEapplied on Clevenger et al. dataset80Appendix B. Supplementary Figures and TablesPredicted Cell Type CountT/NK/ILC 4619Eosinophil cells 4189Basal cells 3624Monocytes/Mac/DC 942Fibroblast cells 669Plasma cells 180Rare cells 19Table B.3: Cell type prediction count of EoE applied on Morgan et al.dataset by cell subsets81
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Ding, Jiarui
ProtoCloud : a generative prototypical self-explanatory model for cell type annotation
Text