UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessing a hyperspectral image analysis system to study tumor and immune cell spatial organizations… Kung, Sonia H.Y. 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2018_may_kung_sonia.pdf [ 5.84MB ]
Metadata
JSON: 24-1.0363386.json
JSON-LD: 24-1.0363386-ld.json
RDF/XML (Pretty): 24-1.0363386-rdf.xml
RDF/JSON: 24-1.0363386-rdf.json
Turtle: 24-1.0363386-turtle.txt
N-Triples: 24-1.0363386-rdf-ntriples.txt
Original Record: 24-1.0363386-source.json
Full Text
24-1.0363386-fulltext.txt
Citation
24-1.0363386.ris

Full Text

ASSESSING A HYPERSPECTRAL IMAGE ANALYSIS SYSTEM TO STUDY  TUMOR AND IMMUNE CELL SPATIAL ORGANIZATIONS  WITHIN LUNG TUMOR MICROENVIRONMENTS  by  Sonia H.Y. Kung  B.Sc. Honours, The University of British Columbia, 2014  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Interdisciplinary Oncology)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)   January 2018 © Sonia H.Y. Kung, 2018 ii  Abstract  Lung cancer is the leading cause of cancer-associated death worldwide. To improve clinical management, biological discovery underlying disease pathogenesis and improved sensitivity of early detection imaging modalities are required. Presently, therapeutic targeting immune components of the tumor microenvironment led to immunotherapies, potent therapeutic agents revolutionizing clinical practice. Increasingly recognized, the native spatial organization of tumor microenvironment immune cells could potentially serve as a surrogate for predicting prognosis and stratifying tumor immunogenicity. However, conventional methods, including histopathology and genomic profiling, are limited in measuring such parameters. Image analysis systems are being tested for these applications, but further development and optimization is required before clinical adoption. Dr. Calum MacAulay and Dr. Martial Guillaud’s research team at the British Columbia Cancer Research Centre (BCCRC) developed a novel hyperspectral image analysis system capable of spatially profiling tumor microenvironments in situ. Herein, this thesis reports the workflow optimization of this system prototyped with the lung adenocarcinoma (AC) microenvironment. This study was conducted with 21 primary lung AC. Adjacent tumor sections were stained for nuclei with hematoxylin and markers of tumor-infiltrating lymphocytes (CD3, CD79a, CD8) or adaptive immune resistance (PD1, PDL1, CD8) by multiplex immunohistochemistry. After a lung pathologist identified areas of interest on tissue sections, five representative regions within each area were imaged. Image stacks of sixteen different illumination wavelengths (from 420 to 720 nm) were then processed by spectral unmixing, segmented, and immunohistochemical marker positivity thresholds applied. Processed two-dimensional images were quantified for cell types and neighboring cell type spatial iii  correlations. Additionally, three lung AC within this cohort were selected to test workflow of image registration software to interpolate cell spatial distributions between sections. The system was able to determine immune cell-to-cell spatial correlations and distribution within and across sections. Furthermore, technical factors were identified that affected the workflow of this system, including staining, image acquisition, segmentation, specialized equipment, and sampling strategy. Here we describe a valuable platform to quantitatively and spatially profile tumor heterogeneity that could then be used to correlate with lung cancer prognosis and treatment outcomes, although further optimization of this method is required.    iv  Lay Summary  Lung cancer is the leading cause of cancer-related death worldwide. To improve patient survival, understanding lung cancer biology and developing new technologies are essential. Awareness that immune cells exist in tumors has enabled remarkably effective drugs, and their spatial locations better predict cancer survival and treatment response. However, conventional methods in pathology and biology do not precisely measure these spatial features. Image analysis software is primarily being explored, but requires further development. Dr. Calum MacAulay and Dr. Martial Guillaud’s research team developed a new image analysis system to measure cell spatial locations in tumors. This thesis determined that this system can identify different levels of immune cell spatial organization and distribution in tumors and discovered technical factors affecting the workflow of this system. These results reveal that the platform is valuable, although additional testing and optimization are required.   v  Preface  Research from this thesis was approved by the University of British Columbia (UBC) Research Ethics Boards, Certificate Numbers: H13-00389 and H16-03060.  Portions of Chapter 1 have been published.   Marshall EA, Ng KW, [Kung SHY†], Conway EM, Martinez VD, Halvorsen EC, Rowbotham DA, Vucic EA, Plumb AW, Becker-Santos DD, Enfield KSS, Kennett JY, Bennewith KL, Lockwood WW, Lam S, English JC, Abraham N, Lam WL†. (2016). Emerging roles of T helper 17 and regulatory T cells in lung cancer progression and metastasis. Molecular Cancer. 15(1), 67. [PMID: 27784305]. Underline indicates co-first authors. Obelus (†) indicates corresponding authors. This is a review article. I am a co-first author and corresponding author of this manuscript. I was involved in concept formation, wrote the manuscript, and illustrated the figures.      Conway EM, Pikor LA, [Kung SHY], Hamilton M, Lam S, Lam WL*, Bennewith KL*. (2016). Macrophages, inflammation and lung cancer. American Journal of Respiratory and Critical Care Medicine. 193(2):116-30. [PMID: 26583808]. Asterisk (*) indicates authors contributed equally. This is a review article. I am a co-author of this manuscript. I helped revise the manuscript and provide figures.            vi  Portions of Chapter 2 have been published in the following selected conference proceedings.  Enfield KSS, [Kung SHY], Gallagher P, Milne K, Chen Z, Piga D, Lam S, English JC, Guillaud M, MacAulay CE, Lam WL. (2017). P2.01-065 Quantification of tumour-immune cell spatial relationships in the lung tumour microenvironment using single cell profiling. Journal of Thoracic Oncology. 12(1), Supplement, S826–S827. [DOI: http://dx.doi.org/10.1016/j.jtho.2016. 11.1117]. This work was presented at the International Association for the Study of Lung Cancer (IASLC) 17th World Conference on Lung Cancer at Vienna, Austria. I am a co-author of this abstract. I was involved in concept formation, conducted image acquisition and processing, and helped revise the abstract.   Enfield KSS, [Kung SHY], Gallagher P, Milne K, Chen ZY, Lam S, English JC, Guillaud M, MacAulay CE, Lam WL. (2016). Single cell profiling and cell sociology of the lung tumor microenvironment. Cell-VIB Symposium: Hallmarks of Cancer, Ghent, Belgium. I am a co-author of this abstract. I was involved in concept formation, conducted image acquisition and processing, and helped revise the abstract.   Enfield KSS, Martin SD, Martinez VD, [Kung SHY], Gallagher P, Milne K, Chen Z, Lam S, English JC, MacAulay CE, Guillaud M, Lam WL. (2017). P2.02-038 Imaging platform for the quantification of cell-cell spatial organization within the tumour-immune microenvironment. Journal of Thoracic Oncology. 12(11), Supplement 2, S2112–S2113. [DOI: http://dx.doi.org/ 10.1016 / j.jtho.2017.09.1216]. I am a co-author of this abstract. I was involved in concept formation, conducted image acquisition and processing. This abstract was presented at the IASLC 18th World Conference on Lung Cancer at Yokohama, Japan as a late-breaking abstract by KSS Enfield. A related abstract will be presented as a plenary by KSS Enfield at the 2018 Fifth American Association for Cancer Research (AACR)-IASLC International Joint Conference: Lung Cancer Translational Science from the Bench to the Clinic at San Diego,USA.  Portions of Chapter 2 will also be included in a manuscript in preparation as a methods paper describing this hyperspectral image analysis system. Appendix C reports additional citations regarding Chapter 2 of this thesis. vii  The hypothesis, aims, and experimental design of this study were developed under the guidance of Dr. Wan Lam and my supervisory committee, Dr. Stephen Lam, Dr. Kevin Bennewith, and Dr. William Lockwood. All software programs of the hyperspectral image analysis system that were assessed in this study were developed and programmed by Dr. Calum MacAulay, Dr. Martial Guillaud, and members of their research teams, including Paul Gallagher. Under the guidance of Paul Gallagher (a member of Dr. Calum MacAulay’s research team), I implemented fixes to the scripts of his MATLAB programs in Sub-Section 3.5.4 and recommended functions for software program design in Section 3.1 and Sub-Section 3.5.4. Tissue sectioning, multiplex immunohistochemistry (IHC) staining of tissue, and whole slide image scans in Section 2.3 was performed with assistance from Daniela Piga, Katy Milne, and Anita Carraro respectively. Area selection of tumors to image in Sub-Section 2.3.1 was performed with the assistance of Dr. John English, a pulmonary pathologist. I subsequently sampled regions within these areas and acquired all images of regions of interest (ROI) for this study. I performed digital image acquisition, image processing, image experiments, and image analyses presented in this study with assistance for the following sub-sections: digital image stitching and spectral unmixing in Sub-Section 2.4.1 from Paul Gallagher, segmentation and cell classification analysis from Dr. Calum MacAulay and Dr. Katey Enfield in Sub-Sections 2.4.2 and 2.4.3 to assess inter-observer variability of immunohistochemical stain positivity for threshold establishment, and Asterics software execution in Sub-Section 2.5.1 from Dr. Martial Guillaud. Unless otherwise specified in figure captions, I designed and illustrated figures in this thesis, which have not been previously published in research articles (Figure 1.2, Figure 1.4, Figure 1.6, Figure 2.2A, Figure 3.10, and Figure 4.1).  viii  Table of Contents  Abstract .......................................................................................................................................... ii Lay Summary ............................................................................................................................... iv Preface .............................................................................................................................................v Table of Contents ....................................................................................................................... viii List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiv  List of Symbols and Abbreviations .......................................................................................... xvi Acknowledgements .................................................................................................................... xix Dedication .....................................................................................................................................xx Chapter 1: Introduction ................................................................................................................1 1.1 Background on lung cancers ........................................................................................... 1 1.1.1 Classification of lung cancers ..................................................................................... 2 1.2 Lung cancer detection and diagnostics ........................................................................... 5 1.2.1 Cancer imaging and clinical features of lung cancer diagnostics ............................... 5 1.2.2 Molecular indicators for lung cancer diagnostics ....................................................... 9 1.3 Lung cancer prognosis methods.................................................................................... 10 1.3.1 Gold standard for lung cancer prognostics ............................................................... 10 1.3.2 Discovering molecular features for lung cancer prognostics .................................... 11 1.3.3 Lung tumor microenvironment, cancer immunology, and immunotherapies........... 13 1.4 Digital pathology .......................................................................................................... 18 1.5 Thesis theme ................................................................................................................. 23 ix  1.5.1 Rationale ................................................................................................................... 23 1.5.2 Objective and hypothesis .......................................................................................... 25 1.5.3 Specific aims and thesis outline ................................................................................ 25 Chapter 2: Materials and Methods ............................................................................................27 2.1 Samples ......................................................................................................................... 27 2.2 Multiplex immunohistochemistry (IHC) ...................................................................... 28 2.3 Hyperspectral imaging .................................................................................................. 30 2.3.1 Imaging selection criteria .......................................................................................... 30 2.3.2 Image acquisition ...................................................................................................... 32 2.4 Digital image processing............................................................................................... 35 2.4.1 Image stitching and spectral unmixing ..................................................................... 35 2.4.2 Segmentation............................................................................................................. 36 2.4.3 Classification............................................................................................................. 37 2.5 Spatial analyses ............................................................................................................. 39 2.5.1 Cell quantity and cell-to-cell spatial analyses ........................................................... 39 2.5.2 Image registration ..................................................................................................... 40 Chapter 3: Results and Discussions............................................................................................42 3.1 Image acquisition and spectral unmixing ..................................................................... 42 3.2 Segmentation................................................................................................................. 56 3.3 Classification................................................................................................................. 60    x  3.4   Cell quantity and cell-to-cell spatial relations:     potential factors related to data interpretation analysis................................................. 69   3.4.1   Parameters measured by Asterics:              Cell quantity and cell-to-cell spatial relations ........................................................... 70  3.4.2   Quantity of general cell types ..................................................................................... 77  3.4.3   Cell-to-cell spatial relations of same neighbor general cell types .............................. 81  3.4.4   Quantity of specific cell types .................................................................................... 89  3.4.5   Cell-to-cell spatial relations of same neighbor specific cell types .............................. 92  3.4.6   Cell-to-cell spatial relations of different neighbor general cell types ......................... 98  3.4.7   Cell-to-cell spatial relations of general and specific cell type neighbors ................. 102  3.4.8  Summary of quantity and cell-to-cell spatial relations with Asterics ........................ 107  3.5 Image registration ....................................................................................................... 111  3.5.1  Different stains ........................................................................................................... 117  3.5.2  Local landmarks ......................................................................................................... 119  3.5.3  Tissue artifacts ........................................................................................................... 121  3.5.4  Image preprocessing with image cropping ................................................................ 123  Chapter 4: Conclusions .............................................................................................................131  4.1 General conclusions and significance ......................................................................... 131  4.2 Future directions ......................................................................................................... 140  Bibliography ...............................................................................................................................141     xi  Appendices ..................................................................................................................................152 Appendix A - Manual graphs of cell-to-cell spatial analyses (raw data without exclusion of data indicating not enough cells for analysis, value = -999) .................................................. 152  A.1 General cell type cell quantity and spatial relations of same neighbors in adaptive resistance panel (raw data plot). .............................................................................. 152  A.2  Specific cell type identified by double markers cell quantity and spatial relations of same neighbors (raw data plot). .............................................................................. 154  A.3 Specific cell type identified by triple markers cell quantity and spatial relations of same neighbors (raw data plot) ............................................................................... 155  A.4 General cell type next to different general cell type neighbor cell quantity and spatial relations and vice versa (raw data plot) .................................................................. 157  A.5 General cell type next to specific cell type neighbor and vice versa cell quantity and   spatial relations (raw data plot) ............................................................................... 159   Appendix B - Autoscaled graphs of cell-to-cell spatial analyses  .......................................... 161  B.1 General cell type cell quantity and spatial relations of same neighbors in T&B cell panel with graph autoscaling .................................................................................. 161  B.2 General cell type cell quantity and spatial relations of same neighbors in adaptive resistance with graph autoscaling ........................................................................... 163  B.3 Specific cell type identified by double markers cell quantity and spatial relations of same neighbors with graph autoscaling .................................................................. 165  B.4 Specific cell type identified by triple markers cell quantity and spatial relations of same neighbors with graph autoscaling .................................................................. 166  B.5 General cell type next to different general cell type neighbor and vice versa cell   quantity and spatial relations .................................................................................. 168  B.6 General cell type next to specific cell type neighbor and vice versa cell quantity and spatial relations ....................................................................................................... 170   Appendix C - List of publications and selected conference proceedings ............................... 172 xii  List of Tables  Table 1.1 Several commercial software quantifying immune cell counts and  spatial analyses.............................................................................................................................. 21  Table 1.2 HALO software modules for oncology research .......................................................... 22  Table 2.1 Clinical features of lung AC cohort .............................................................................. 27  Table 2.2 Summary of immune cell marker antibody panels. ...................................................... 29  Table 3.1 Spectral components with six and sixteen illumination wavelengths........................... 46  Table 3.2 Random selection of cases for sixteen versus six illumination wavelength  experiment..................................................................................................................................... 48  Table 3.3 Averages of each spectral component of T&B Cell panel for sum of the  absolute difference, mean of the absolute difference, and performance time for  sixteen compared to six illumination wavelengths ....................................................................... 50  Table 3.4 Averages of each spectral component of AR panel for sum of the absolute  difference, mean of the absolute difference, and performance time for sixteen compared  to six illumination wavelengths .................................................................................................... 52  Table 3.5 Summary of thresholds for T&B Cell panel ................................................................. 61  Table 3.6 Summary of thresholds for adaptive immune resistance panel .................................... 63  Table 3.7 Summary of thresholds for adaptive immune resistance panel for PDL1high ............. 65  Table 3.8 Parameters measured by Asterics used to assess selection bias  analysis in this study ..................................................................................................................... 71  Table 3.9 Summary and significance of analyses performed with Asterics. ................................ 75  Table 3.10 Summary and future directions of analyses performed with Asterics. ..................... 109  Table 3.11 Sample selection for image registration analysis ...................................................... 113  Table 3.12 File sizes of preprocessed image classes .................................................................. 127  Table 3.13 Running time of Imreg.m with fixes on images without preprocessing  and with preprocessing ............................................................................................................... 129 xiii   Table 3.14 Single image preprocessed running time .................................................................. 129  Table 4.1 Proposed solutions for the BCCRC hyperspectral image analysis  system for performance and specialized equipment ................................................................... 134  Table 4.2 Proposed solutions for the BCCRC hyperspectral image analysis  system for cell-to-cell spatial analysis ........................................................................................ 136  Table 4.3 Proposed solutions for the BCCRC hyperspectral image analysis  system for image registration ...................................................................................................... 138  xiv  List of Figures  Figure 1.1 Classification of lung cancers ........................................................................................ 3  Figure 1.2 Cancer imaging in clinical work up of lung cancer ....................................................... 8  Figure 1.3 Heterogeneity of immune cell infiltration within lung adenocarcinoma (AC)  tumor microenvironment  ............................................................................................................. 14  Figure 1.4 Biology of immune checkpoint blockade and mechanism of action  of immune checkpoint inhibitors. ................................................................................................. 16  Figure 1.5 Overview of software modules for BCCRC hyperspectral  image analysis system ................................................................................................................... 24  Figure 1.6 Overview of methods of specific aims of this thesis ................................................... 26  Figure 2.1 Sampling of tumors for image acquisition .................................................................. 31  Figure 2.2 Image acquisition hardware and software of BCCRC  hyperspectral image analysis system ............................................................................................ 34  Figure 2.3 Graphical user interface of Hyperspectral_Processing.m ........................................... 35  Figure 2.4 Gallery for validation of thresholds ............................................................................. 38  Figure 2.5 Syntax modified in Imreg.m to perform image registration ........................................ 41  Figure 3.1 Spectral unmixing of acquired images. ....................................................................... 43  Figure 3.2 Spectral unmixing sixteen versus six illumination wavelengths  concentration maps ....................................................................................................................... 47  Figure 3.3 Sample images of absolute difference matrix between sixteen and  six illumination wavelengths with saturation observed. ............................................................... 54  Figure 3.4 Segmentation of images with low and high density cell nuclei. ................................. 57  Figure 3.5 Segmentation inside and outside of clusters of dense cell nuclei ................................ 59  Figure 3.6 Homogeneity and heterogeneity of antibody stains. ................................................... 64  Figure 3.7 Establishing multiple thresholds for PDL1 stain. ........................................................ 66  Figure 3.8 Classification trees of cell types based on threshold positivity ................................... 68  xv  Figure 3.9 Mean neighborhood group (MNG) by Asterics .......................................................... 72  Figure 3.10 Key for graph interpretations..................................................................................... 76  Figure 3.11 General cell type cell quantity and spatial relations of same  neighbors in T&B cell panel. ........................................................................................................ 83  Figure 3.12 General cell type cell quantity and spatial relations of same  neighbors in adaptive resistance panel.. ........................................................................................ 87  Figure 3.13 Specific cell type identified by double markers cell quantity  and spatial relations of same neighbors ........................................................................................ 93  Figure 3.14 Specific cell type identified by triple markers cell quantity  and spatial relations of same neighbors.. ...................................................................................... 96  Figure 3.15 General cell type next to different general cell type neighbor  and vice versa cell quantity and spatial relations ........................................................................ 100  Figure 3.16 General cell type next to specific cell type neighbor  and vice versa cell quantity and spatial relations ........................................................................ 105  Figure 3.17 Examples of well registered images ........................................................................ 114  Figure 3.18 Examples of differences in reproducibility for image registration .......................... 116  Figure 3.19 Examples of differences in stain of CD8. ................................................................ 118  Figure 3.20 Examples of small landmarks .................................................................................. 120  Figure 3.21 Examples of tissue artifacts ..................................................................................... 122  Figure 3.22 Image preprocessing with image cropping in Microsoft Office  Picture Manager .......................................................................................................................... 124  Figure 3.23 Preprocessing of images with Microsoft Office Picture Manager  is recognized as a three dimensional image in MATLAB workspace. ....................................... 125  Figure 3.24 Image registration with fixes implemented of preprocessed images ....................... 126  Figure 3.25 Classes of preprocessed image tested with Imreg.m with fixes .............................. 127  Figure 3.26 Image registration of preprocessed image with Imreg.m with fixes ....................... 130  Figure 4.1 Flowchart of workflow optimization of the BCCRC hyperspectral  image analysis system. ................................................................................................................ 133  xvi  List of Symbols and Abbreviations  #   number  %  percent, may refer to ‘Cell Percentage’ metric from Asterics if specified,     or may refer to MNG written as frequency  .jb0   file extension of Asterics .m   file extension of MATLAB program .TIFF or .TIF   tagged image file format .unit   file extension of DUnit /   when designated with MNG, refers to 'next to' <   less than  >   greater than ≃ or ~   approximately  AC   adenocarcinoma ALK   anaplastic lymphoma kinase AR   adaptive immune resistance ATS   American Thoracic Society  b  y-intercept, value of average of ROI (i.e. percentage, MNG, VAR) BCCRC   British Columbia Cancer Research Centre  BCG   Bacillus Calmette-Guérin bpp   bits per pixel CAD   computer aided diagnosis CD   cluster of differentiation CE   Conformité Européene CI   confidence interval CT   computed tomography CTLA4   cytotoxic T-lymphocyte-associated protein 4 cTNM   clinical Tumor Node Metastasis staging  CXR   chest X-ray DAB   3,3'-diaminobenzidine dH2O   distilled water DNA   deoxyribonucleic acid EBUS   endobronchial ultrasound EGFR   epidermal growth factor receptor EMCCD  electron-multiplying gain CCD ERS   European Respiratory Society  FDA   Food and Drug Administration FFPE   formalin-fixed paraffin-embedded FISH   fluorescent in situ hybridization  GB   gigabyte = 230 bytes (binary) = 1,073,741,824 bytes (binary) xvii  GE   General Electrics GPU   graphics processing unit GUI   graphical user interface HD   hard disk  Hx   hematoxylin I   Roman numeral one if next to 'Stage', 'TNM', or 'Type'  i.e.  id est, Latin for ‘that is’ IASLC   International Association for the Study of Lung Cancer IDP   Integrated Digital Pathology IHC   immunohistochemistry II  integrated intensity, or can also refer to  Roman numeral two if next to 'Stage', 'TNM', or 'Type'  III   Roman numeral three if next to 'Stage' or 'Phase' IL-2  interleukin-2 irAE   immune related adverse effects ISH   in situ hybridization  JPEG  Joint Photographic Experts Group KRT7   keratin 7  LCC   large cell carcinoma lncRNA   long non-coding RNA m  slope of line (average of ROI) (m ≃ 0, i.e. percentage, MNG, VAR) MB   megabyte = 220 (binary) = 1,048,576 bytes (binary) MIA   minimally invasive adenocarcinoma miRNA   microRNA MNG   mean neighborhood group ( 0 ≤ MNG ≤ 100%)  MRI   magnetic resonance imaging mRNA   messenger RNA NLST   National Lung Screening Trial nm   nanometer  NSCLC   non-small cell lung cancer oC  degrees Celsius ORR   objective response rate PC   personal computer PCR   polymerase chain reaction PD1   programmed death 1 PDL1   programmed death ligand 1 PET   positron emission tomography PIPS   Philips IntelliSite Pathology Solution  piRNA   piwi-interacting RNA pTNM   pathological Tumor Node Metastasis staging R   refers to red in relation to 'Primary' or 'matrix'  RAM   random-access memory xviii  RGB   red-green-blue RNA   ribonucleic acid ROI   region of interest SCLC   small cell lung cancer sCMOS   scientific complementary metal-oxide-semiconductor SD   standard deviation SqCC   squamous cell carcinoma TB   T&B Cell  TCGA   The Cancer Genome Atlas  TM   trademark TMA   tissue microarray TNM   Tumor Node Metastasis Tregs   regulatory T cells TTF1   thyroid transcription factor 1 USB  Universal Serial Bus VAR   variance of MNG (VAR ≥ 0) WSI   whole slide image X+   positive staining (+) for protein of interest (X)  X++   higher positive staining (+) for protein of interest (X)  μm    micrometer   xix  Acknowledgements  This work was supported by the following granting agencies: the National Sanitarium Association (NSA) and the Canadian Institutes of Health Research (CIHR) Foundation. This work was supported by the following scholarships: CIHR Frederick Banting and Charles Best Canada Graduate Scholarship - Master's Award, Interdisciplinary Oncology Program Travel Award, University of British Columbia (UBC) Graduate Student Travel Fund, and UBC Faculty of Medicine Graduate Awards.   I thank my supervisor Dr. Wan Lam, and my supervisory committee members Dr. Stephen Lam, Dr. Kevin Bennewith, and Dr. William Lockwood for their support, guidance, and opportunities to develop my training in scientific research. I specially thank Dr. Calum MacAulay, Dr. Martial Guillaud, and their research team, including Paul Gallagher, for the opportunities to be involved in assessing and optimizing the workflow of their hyperspectral image analysis system. I am grateful for my colleagues in the Wan Lam Laboratory past and present, particularly Dr. Katey Enfield and Daiana Becker-Santos for their mentorship. I also thank Dr. John English, Katy Milne, Daniela Piga, and Anita Carraro for their expertise and technical assistance during the implementation of this project. I would also like to acknowledge the cancer patients and their caregivers, families, and friends.  Lastly, I thank my father, a lung cancer survivor, and my mother, also a cancer survivor, who inspired me to contribute to cancer research.  xx  Dedication  This thesis is dedicated to my parents and to those affected by cancer.    1  Chapter 1: Introduction  1.1 Background on lung cancers  Lung cancers are the leading cause of cancer-associated death worldwide. The five-year survival rate for this disease remains only 17% in Canada [1]. Underpinning the poor prognosis of this disease include the following two challenges. Firstly, current lung cancer treatment modalities are not curative to all patients. Despite primary treatment by surgical resection with curative intent, recurrence occurs in over 30% of patients with early stage lung cancer [2, 3].  And secondly, existing methods for early detection of lung cancers are limited. Approximately 70% of patients are presented with locally advanced or metastatic disease at the time of diagnosis [4]. Antithetically, lung cancers detected at early stage can have five year survival rates of over 70% [5]. Solving these two fundamental problems in lung cancer clinical management requires multidisciplinary research. The former challenge can be approached through biological discovery – a deeper understanding of the mechanisms driving lung cancer pathogenesis is required to identify therapeutic intervention points to inform the development and implementation of new treatment strategies. The latter can be tackled by technological advancement – improved sensitivity of imaging methods to detect early-staged lung cancers will be imperative, as these localized cancers have a higher probability to respond to treatment [6-8]. Identification of populations to screen for pre-existing and new technologies will also be a relevant strategy. Integration of this multidisciplinary scientific knowledge will inform clinical practice to improve prognosis of lung cancer patients.   2  1.1.1 Classification of lung cancers  Lung cancers are heterogenous and are classified into two major groups: small cell lung cancer (SCLC), accounting for 15% of lung cancer cases, and non-small cell lung cancer (NSCLC), which are the remaining 85% of cases. The dichotomization of these two major groups of carcinoma is based on the morphology of lung cancer cells, where “small cell” in SCLC nomenclature refers to tumor cells with scant cytoplasm and small nuclei, and where NSCLC do not exhibit such phenotypes [9]. As the most aggressive type of lung cancer, SCLC are highly malignant, are thought to originate from neuroendocrine cells present central airways, and is typically diagnosed in smokers. Representing the majority of lung cancer cases, NSCLC is further subdivided into three major histological subtypes: adenocarcinoma (AC), squamous cell carcinoma (SqCC), and large cell carcinoma (LCC). Lung AC is the predominant NSCLC diagnosed, accounting for approximately half of all NSCLC cases and thus, will be the focus of this thesis [10]. Lung AC tends to arise from glandular epithelium of the lung periphery from bronchioalveolar stem cells, club cells, or type II pneumocytes and unlike other subtypes, occurs in greater percentage of never smokers [11]. Conversely, SqCC develops mostly in the central airways and segmental bronchi and is predominantly diagnosed in smokers. Lung tumors that do not exhibit hallmark features of SCLC, SqCC, and AC are diagnosed as LCC. These tumors may arise anywhere in the lung. The diverse pathologies of lung cancers described here display the extensive biological complexities of this disease (Figure 1.1). 3   Figure 1.1   Classification of lung cancers. Left: Lung cancer classification and percent incidence of American populations. Lung cancer is dichotomized into two major classes, small cell lung cancer (SCLC) and non small cell lung cancer (NSCLC). Percent incidence of NSCLC histologies sum to NSCLC percent incidence (83.4%). Remaining histologies (non-small cell carcinoma and other specified carcinomas) are not shown. Right: Representative sites of origin typical of lung cancer. SCLC originates from the central airway, and is posited to arise from neuroendocrine cells (middle panel). As the predominant class, NSCLC is further classified into three major subtypes (SqCC), large cell carcinoma (LCC), and adenocarcinoma (AC). SqCC typically arises from central airways and segmental bronchi from basal cells (upper and middle panel). Tumors without features of other NSCLC subtypes are classified as LCC, and may arise from anywhere in the lung. AC is the most frequent NSCLC and is postulated to arise from type II pneumocytes and club cells (upper and lower panel). I produced the illustrated figures in this publication. Figure reprinted is original from the original publisher BioMed Central with proper attribution of authorship and citation from Marshall, Ng, and Kung et al. [12].  BioMed Central does not require formal written permission to reprint figures. 4  Classification of lung cancers is essential to enable best clinical practice. Classification is mandatory for treatment, as drugs are only approved for specific lung cancer subtypes or are contraindicated (inadvisable) in certain cases, as illustrated by pemetrexed, one of the first drugs rationalizing the distinction between squamous and nonsquamous histologies for treatment decisions [13]. To identify lung cancer subtypes, histopathology and molecular testing of lung tumors are essential techniques implemented in the clinic. Histopathology refers to the study of disease states microscopically characterized in tissue. The heterogeneous pathogenesis of lung cancer subtypes previously described are attributed by the plethora of different genetic backgrounds expressed in lung cancer subtypes, resulting in assorted morphologies and protein marker expression patterns. Unique, common physical features or markers (both molecular and histopathological) within each group can be exploited as a means to categorize different lung cancer subtypes [11]. For instance, AC can be differentiated from SqCC using thyroid transcription factor 1 (TTF1, alternatively called NKX2-1) and keratin 7 (KRT7), as the latter do not highly express these markers [14]. Lung AC is further classified based on invasiveness (pre-invasive lesions, minimally invasive AC (MIA), invasive AC, or variants of invasive AC). These AC classifications can be further characterized based on predominant histological growth patterns, a feature significantly correlating with patient prognosis: predominant lepidic lymphovascular or pleural invasion) with better prognosis, predominant acinar (glandular formation and predominant papillary) fibrovascular cores with tumor cells replacing alveolar lining) with intermediate prognosis, predominant micropapillary (tufting without fibrovascular cores) and predominant solid (sheets of tumor cells) with poorer survival, and predominant invasive mucinous (columnar cells with mucin) [15, 16]. Lung AC can also be further subdivided based on expression of specific proteins, referred to as molecular subtypes. Molecular subtyping 5  of lung cancers enables systematic stratification of patients for treatment with small molecule inhibitors. For instance, epidermal growth factor receptor (EGFR) positivity prescribes the usage of EGFR tyrosine kinase inhibitors (erlotonib, geftinib, and aftinib), and anaplastic lymphoma kinase (ALK) positivity with ALK inhibitors (crizotinib and ceritinib) [17]. Lung cancers are genetically heterogeneous, and the complex diversity of clinically relevant phenotypes emphasizes the need for appropriate classification. With new biological discoveries and technological advances expected, the lung cancer classification system undergoes revisions to reflect updated knowledge about this disease with recommendations by the International Association for Study of Lung Cancer (IASLC) [18, 19]. Indeed, the presentation of distinct physical features in tumors is a wealth of information that if precisely understood, can be strategically mined for appropriate clinical decision making.  1.2 Lung cancer detection and diagnostics  1.2.1 Cancer imaging and clinical features of lung cancer diagnostics  Medical imaging has an essential role in lung cancer detection and is present in all steps of the complete medical examination used to diagnose suspected lung cancer, referred to as the clinical work up. Lung cancer diagnosis relies on both sophisticated macroscopic and microscopic imaging to comprehensively provide evidence of disease presentation for effective clinical decision making. These conventional imaging modalities tend to be initially non-invasive and more invasive downstream. The imaging modalities implemented in the clinical work up include chest radiograph or chest X-ray (CXR, as commonly referred to), computed tomography (CT), 6  bronchoscopy, microscopy, positron emission tomography (PET), and magnetic resonance imaging (MRI) (Figure 1.2) [20]. The workup is sequentially divided in the following major steps: clinical presentation, diagnosis, and staging. In clinical presentation, macroscopic, non-invasive cancer imaging is primarily implemented. If a patient is suspected to present with lung cancer to a health care provider (family physician, walk-in clinic, emergency department), the patient will first be referred for a CXR, and the radiograph will be reviewed by a primary care provider. CXR provides initial evidence of lung cancer, however is not ideal for cancer characterization and staging [21]. If results are suspicious, the patient will be referred to a diagnostic assessment program or specialist to perform a chest CT, an imaging method found to be more effective than CXR at determining early stage lung cancer as determined by the seminal study known as the National Lung Screening Trial (NLST)  [22]. A chest CT scan facilitates the identification of preliminary staging, tumor type, and to assess patient fitness for prospective treatment strategies. Although CT is a core imaging modality for lung cancer screening, this technique presents many challenges, including interpretation of positive CT results, such as high false positivity rates from low dose CT, and possible negative effects after repeated CT exposure, such as potential risks for cancer development [23-29]. Based on initial cancer imaging, the patient will undergo diagnostic procedures if a  suspected mass is identified from initial staging, involving invasive procedures. Image-guided biopsies using real-time imaging are performed by a lung oncologist to collect primary specimens for analysis, including bronchoscopy (endoscopic imaging of the airways) to collect bronchial fluid specimens and bronchial/transbronchial biopsies and endobronchial ultrasound (EBUS) biopsy of lymph nodes [30, 31]. Anatomical location of the tumor informs the type of 7  diagnostic technique used. For instance, bronchoscopy is appropriate for central/hilar lung tumors (located in airways), and is not as accurate for peripheral nodules (small airways or lung parenchyma) [32]. Anatomical location of tumor also informs lung cancer diagnosis, as different subtypes of lung tumors tend to originate in different sites, described earlier in Sub-Section 1.1.1.  In staging, the definitive diagnosis of the extent of anatomical disease is made. Cancer diagnosis is performed using lung tumor biopsies by the pathologist, a trained observer, to classify and describe disease severity. Specifically, the pathologist evaluates an analog image of the tumor directly viewed through brightfield microscopy to determine histopathology. The microscope is the principle optical instrument used by the pathologist for imaging, and this imaging modality will be the focus of this thesis. Key histopathological features of lung tumors are evaluated, as described in Sub-Section 1.1.1, where tumor tissues are stained to enhance the visualization and perception of histological features (detailed in Sub-Section 1.2.2). In addition to tumor tissue, histopathology of lymph nodes present in the mediastinum is also assessed if invasion is evident from initial cancer imaging. These organs of the immune system are one of the first site of metastasis for the majority of solid cancers and therefore, provide valuable clinical information that informs staging, prognosis, patient management, and decision making of treatment modalities such as surgery and/or chemoradiation [33-36].  Cancer staging also relies on PET/CT and MRI or CT brain scans (if MRI is not available or is contraindicated) to determine if extrathoracic spread has occurred [21, 37]. This procedure may be optional for patients that have clinical stage I or II disease and are asymptomatic. If the tumor is non-resectable, additional treatment algorithms are implemented to decide potential treatment strategies. In summation, the lung cancer diagnostic work up highlights that modern  8                    Figure 1.2   Cancer imaging in clinical work up of lung cancer.  Microscope illustration adapted from photo courtesy of Paul Gallagher. 9  imaging technologies and optical devices are essential infrastructure for appropriate clinical decision making and timely intervention of lung cancer treatment strategies [38].  1.2.2 Molecular indicators for lung cancer diagnostics  The detection of proteins in preserved tumor tissue is routinely performed in the clinic. Specifically, immunohistochemistry (IHC) of formalin-fixed paraffin-embedded (FFPE) lung tumor tissue is primarily used for definitive diagnoses due to practicalities, including greater convenience for storage and sectioning of tissue and improved preservation of tissue morphology compared to frozen tissue specimens [39, 40]. Chromogen detection of IHC is primarily used in the clinic. By contrast, fluorescence detection methods can simultaneously assess more markers, but are confounded by autofluorescence present in tissues, signal degradation leading to fading, and the cost of image acquisition hardware [39]. As previously mentioned, brightfield IHC of singly stained sections of tissue is primarily used in clinical settings. Multiplex IHC approaches (chromogen or fluorescence), where more than one molecular target is simultaneously stained in one tissue section, is not routinely performed in clinical settings, but rather are implemented in research settings. Multiplex IHC staining of tissue is advantageous, whereby simultaneous analysis of markers conserves limited tumor biopsy specimens and more accurately phenotypes specific cell types, such as immune cells that require multiple markers for identification [41].  In addition to protein, other molecular indicators for lung cancer are used for diagnosis, including nucleic acids. Nucleic acid assessment is primarily used to determine lung cancer molecular subtypes. Clinical assays include polymerase chain reaction (PCR) for molecular subtyping of lung cancer to assess mutation of EGFR to stratify tyrosine kinase inhibitors. However, such 10  methods do not describe localization of specific cell types in tissue. IHC is used to identify ALK translocations [42]. Staining of nucleic acid using in situ hybridization (ISH) is also implemented in clinical practice, including fluorescent in situ hybridization (FISH). The United States Food Drug Administration (FDA) approved the Vysis ALK Break-Apart FISH Probe Kit as a companion diagnostic test for the ALK inhibitor crizotinib for lung cancer [43]. However, ISH methods are relatively poorly standardized method across centers, and where false negatives are more prevalent compared to IHC [44, 45]. In anticipation of auxiliary technologies to improve standardization of in situ nucleic acid detection, IHC and brightfield imaging of stained antigens in specimens remain the mainstay methodology implemented in routine clinical practice for economy and convenience [39, 40].        1.3 Lung cancer prognosis methods  1.3.1 Gold standard for lung cancer prognostics  The 8th Edition of the Tumor Node Metastasis (TNM) classification system established  by the IASLC and Prognostic Factors Committee (enacted January 1, 2017) is at present the gold standard for lung cancer diagnosis to describe the anatomic extent of this disease [19]. Collective observations derived from cancer imaging modalities from the clinical work up are summarized by the TNM descriptor, where T descriptor refers to primary tumor site, N descriptor represents the regional lymph nodes in the lung, and M is the presence or absence of distant metastasis. T and N staging primarily rely on histopathology results (as described in Sub-section 1.2.1). The TNM staging system is subdivided into clinical TNM (cTNM or TNM) where diagnoses are 11  made from imaging studies and biopsies, and pathological TNM (pTNM), where diagnosis is determined post-surgery of resectable tumors to validate features previously identified in the biopsy and extent of local involvement. Although the gold standard, this classification scheme is prone to inaccuracies, primarily from the differences in survival observed within TNM staging groups, which may be attributed to a lack of personalized prognosis that accounts for genetics and environmental factors of patients [5, 46, 47]. Thus, identification of additional prognostic markers is needed to further refine this classification scheme.   1.3.2 Discovering molecular features for lung cancer prognostics  Elucidation of the molecular features of lung tumors carries significant potential to improve lung cancer prognostics. Genomic profiling is the current state of knowledge approach to capture new insights into lung cancer molecular biology, including clinical applications for precision oncology [48]. Gene expression profiles of cancers is spearheaded by microarray and ribonucleic acid (RNA)-sequencing technologies, enabling multidimensional genomic databases, including The Cancer Genome Atlas (TCGA), a repository for high throughput analyses of cancer genetic alterations [49, 50]. Comparison of genetic profiles of non-malignant and malignant tissue enables determination of candidate genes specific to tumors, encapsulating global cellular functions specific to cancer pathogenesis. Gene sets derived from gene expression profiling can be combined and are referred to as gene signatures. Gene signatures, when correlated with clinical features and treatment outcomes, are a powerful device to predict cancer prognosis and improve treatment stratification. Expression signatures include different types of RNA, including microRNA (miRNA), piwi-interacting RNA (piRNA), messenger RNA (mRNA), and long non-12  coding RNA (lncRNA), or a combination of these RNA species [51-55]. Several studies have demonstrated that gene expression profiles of lung cancers derived from microarrays are able to predict lung cancer survival [53, 56, 57]. This includes analyzing non-tumor cell gene expression profiles in bulk tumor derived from microarray [58]. Other recently developed methods include gene expression profile of immune cells in tumors and peripheral blood, and such methods are also useful for monitoring therapeutic responses in clinical trials [59, 60]. Thus, this in silico approach has been implemented to improve lung cancer prognostics.  Despite the utility of gene expression profiling for large scale molecular data analyses, this method is limited by the use of dissociated bulk tumor and not the assessment of single cells. Furthermore, RNA levels do not always correlate with protein levels, which may not necessarily directly predict tumor phenotypes [61]. Consequently, inaccurate gene expression profiling confounds interpretations of cancer biology. In addition to analyzing genetic components intrinsic to tumor cells, the context of tumors, referred as the tumor microenvironment, is presently recognized as a significant component of tumors that requires profound elucidation, given that the targeting of this specific biology has enabled dramatic therapeutic responses [60]. Present methods for genomic profiling do not fully capture sufficient information regarding tumor cellularity, including characterizing the native cellular spatial locations and organizations within the tumor microenvironment [62]. Thus, new methods are needed to provide more immediate insights into this expanding field of research. These innovative methods that investigate tumor microenvironments will be discussed in further detail in Section 1.4.   13  1.3.3 Lung tumor microenvironment, cancer immunology, and immunotherapies  The lung tumor microenvironment is comprised of many different cell types, including  stromal, endothelial, and immune cells that coexist amongst one another, in addition to non-cellular components such as extracellular matrix [14]. Thus, the tumor entity can be appreciated from the perspective of ecology as an ecosystem, where diverse cell types interact with one another in various ecological relationships, including synergistic relationships such as mutualism or others that are oppositional, including predator-prey interactions [63, 64]. The cellular composition of the tumor microenvironment can be thought of as an equilibrium between pro-tumor or anti-tumor responses, and it is the balance of these biological processes that significantly contribute to cancer progression and patient prognosis [65]. For instance, immunosuppressive cells, such as regulatory T cells (Tregs), are known to promote immune evasion of tumors, progression, and metastasis [66]. In addition, the different levels and types of cytokines produced by tumor and immune cells is a primary mechanism that modulates the tumor microenvironment to favor progression of tumors [14]. As such, these mechanisms are affected by cellularity, including heterogeneity in infiltrating immune cell composition (Figure 1.3). The paradigm of the tumor microenvironment has presently revolutionized clinical practice of lung cancers, as epitomized by the innovation of immune checkpoint inhibitors.  Immune evasion is a hallmark of cancer [67]. Such mechanisms enable cancers to  circumvent anti-tumor immune surveillance. Therapies that enhance the anti-tumor immune response are referred to as cancer immunotherapies. Presently, several are becoming first-line treatments for cancers, including lung cancers. For instance, the understanding of immune checkpoint molecules and realization of their potential as targets for therapeutic blockade of  14                   Figure 1.3    Heterogeneity of immune cell infiltration within lung adenocarcinoma (AC) tumor microenvironment.   (A) Lung tumors can exhibit high immune cell infiltration. For instance, wide expansion (star) of alveolar interstitium by a diffuse population of inflammatory cells, mostly lymphocytes, can be presented in tumors with lepidic growth pattern. Arrow indicates cancer cells. (B) By contrast, a similar tumor can present with less immune infiltration, with minimal interstitial expansion (star) and few infiltrating inflammatory cells. Arrow indicates cancer cells. (C) AC with lepidic growth can have focal expansion of interstitium by lymphoid follicular hyperplasia (star). Arrow indicates neoplastic cells. (D) Fibroblastic scarring (star) with few infiltrating lymphocytes may also be observed, as in this AC with acinar pattern. Arrow indicates neoplastic cells. (E) AC can exhibit no fibrous expansion nor contain infiltrating lymphocytes as exhibited in this tumor with a papillary pattern, as highlighted by the alveolar septae (arrow heads). Arrow indicates cancer cells. Original magnification of images 100x. Annotations of histology courtesy of Dr. John English. Figure reprinted is original from the original publisher BioMed Central with proper attribution of authorship and citation from [12]. BioMed Central does not require formal written permission to reprint figures. 15  tumor immune evasion was an important contribution to the development of cancer immunotherapies [68]. Several monoclonal antibody cancer immunotherapies are approved by the FDA, including for lung cancers. Despite the observation of dramatic responses, only a fraction of patients respond to these immunotherapies. Therefore, a greater understanding of how cancer cells evade the immune system is essential for: (a) identifying and stratifying patients who may benefit from existing immunotherapies, and (b) informing the development and application of novel immunotherapies targeting the biology underlying specific tumor immune evasion mechanisms observed in different individuals. Lung cancers were historically thought to be non-immunogenic as patients displayed limited response when treated with non-specific immunotherapies expected to stimulate an immune response, such as Bacillus Calmette-Guérin (BCG) vaccination and cytokine therapies, including interleukin-2 (IL-2) and interferon treatments [69-71]. However, based on results from recent immunotherapy trials targeting immune checkpoint molecules, including cytotoxic T-lymphocyte-associated 4 (CTLA4), programmed cell death 1 (PD1), and programmed cell death ligand 1 (PDL1), it appears that lung cancers are sensitive to specific cancer immunotherapies (Figure 1.4). FDA-approval of immune checkpoint inhibitors, including first line treatment of lung cancers, has been a significant milestone in lung cancer treatment. However, like most therapies, there is a range of responses observed following immunotherapy, as illustrated by across and within different kinds of immune checkpoint inhibitors. For instance, anti-PD1/PDL1 immune checkpoint inhibitors compared to anti-CTLA4 inhibitors have less toxicities, known as immune related adverse events (irAE) [72].Within PD1 and PDL1 immune checkpoint inhibitors, objective response rates (ORR) are observed to be variable. In the KEYNOTE 024 Phase III clinical trial of pembrolizumab (anti-PD1 humanized monoclonal antibody), ORR was 44.8% for patients16         Figure 1.4    Biology of immune checkpoint blockade and mechanism of action of immune checkpoint inhibitors. (A) T cell and cancer cell immune checkpoint blockade. T cell receptor (TCR) binds to tumor associated antigen (TAA) and bound to cognate major histocompatability complex (MHC) leading to anti-tumor T cell activation and cancer cell death. Immune checkpoint receptor and ligand interactions dampen the TCR-MHC axis. Administration of immune checkpoint inhibitors restores T cell activation and induces cancer cell death (B) T cell and antigen presenting cell (APC) immune checkpoint blockade. Similarly, T cell activation from antigen presenting cells, such as dendritic cells requires the binding of the TCR to TAA peptide/MHC complex in addition to the binding of CD28 and B7 costimulatory molecules. Interactions of these molecules promote T cell activation and cancer cell death. Immune checkpoint receptors and ligands inhibit activation of these interactions. Immune checkpoint inhibitors restore APC-mediated T cell activation. (C) FDA approved cancer immune checkpoint inhibitors targeting the PD1/PDL1 axis for lung cancers. Generic drug name is listed first, and corresponding trade drug name is included in brackets. Drugs in bold and with asterisks are FDA approved for lung cancer. (D) FDA approved cancer immune checkpoint inhibitors targeting the CTLA4/B7 axis for lung cancers. Generic drug name is listed first, and corresponding trade drug name is included in brackets.17  that qualified (including criteria of having at least PDL1 tumor proportion score of 50%), while in CHECKMATE 057 Phase III clinical trial of nivolumab (another anti-PD1 humanized monoclonal antibody), only 19% of patients had objective responses [73, 74]. The initial Phase III OAK trial of atezolizumab (anti-PDL1 humanized monoclonal antibody) revealed that ORR was observed in only 14% of patients, where the median duration of response was significantly longer [16.3 months (95% CI 10.0–not evaluable) versus 6.2 months (4.9–7.6) with docetaxel; p value < 0.0001] [75]. Despite recent accelerated approval by the FDA, combination of pembrolizumab with cisplatin in the treatment arm was observed to have ORR of 55% [76]. Evidently, improved stratification of these drugs will be necessary. Certainly, the identification of definitive cancer immunotherapy diagnostic markers remains a challenge. Although PDL1 companion diagnostic tests are FDA-approved for NSCLC, the test is imperfect for PD1 immune checkpoint inhibitors, as patients negative for PDL1 can still benefit from these immunotherapies, and some PDL1 positive patients do not benefit.  In order to improve the efficacy of immunotherapies, including immune checkpoint inhibitors, to treat lung cancer patients, it is imperative to (1) further characterize the mechanisms contributing to immune evasion/suppression phenotypes in lung cancer patients and (2) determine the variability in these phenotypes across and within individual patients. Resolution of these questions will enable improved stratification of immunotherapeutic drugs and/or repositioning of immunotherapies used in other cancers to treat lung cancer patients.    18  1.4 Digital pathology  At present, traditional histopathology remains the gold standard for definitive diagnosis, whereby immunohistochemically stained tissue biopsies are viewed and carefully assessed under brightfield microscopy by a pathologist for suspected carcinoma [39, 77, 78]. Converging with the revolution of computerization and technological development, pathology has accelerated towards automation, such as in immunohistochemical staining systems that have been integrated in the routine workflow of pathologists. However, histopathological evaluation of tumor tissue slides remains manual and is problematic due to subjectivity, involving intra-observer and inter-observer variability, and limited standardization of  pre-fixation and fixation methodologies [79]. Histopathology remains at most semi-quantitative, as the metrics for positive stain level quantification carries few descriptors, with the highest resolved description being the semi-quantitative scoring system that is primarily nominal or ordinal and with discrete 3- or 4-point scoring methods for stain intensity levels [39, 61, 77]. These limitations are associated with imprecise cut off values, heterogenous staining patterns, in addition to details that the human eye are not sensitive to detect [39, 78, 80].  Digital pathology presents as a promising method to more objectively assess tumors when incorporated within the pathologist’s workflow and provides numerous advantages [81]. Digital images enable the minimization of inter-batch variation from IHC staining, such as  images of multiplex IHC staining systems that enable analysis of multiple parameters in tissue to understand complex biological systems related to pathology in limited, precious tissue specimens from biopsies and to further classify tumors that appear similar in morphology [39, 82-84]. Digital pathology is versatile and can be used to measure multiple parameters, including 19  immunohistochemical stain of proteins, estimation of cell proliferation, and multiplex FISH of deoxyribonucleic acid (DNA) sequences as highlighted in Bengtsson et al. 2017 [85].  In addition, image registration (or alignment) of adjacent tissue sections would enable the integration of image data sets, an image processing method used in tomography, such as CT for higher level spatial analysis of tissues and anatomy, in addition to image correction, such as for tracking lung motion from respiration [39, 86-89]. Additional advantages of digital pathology include transmission of pathology image data to facilitate telepathology (the practice of pathology from a distance), which could enable outsourcing, consultation, and diagnostic confirmation for remote areas with uneven distribution of pathologists, including in developing countries [90-92]. Nationally, telepathology was identified to be safe, accurate, and reliable for frozen sections by the University Health Network Canada [93-95]. Digital pathology is a modality that can be used to improve education of future pathologists, including enhancing learning and collaboration among medical students [96]. In addition, digital pathology enables the generation of digital tissue banking as a repository for research settings that can be continually reassessed for whole slide or virtual biopsy analysis to interrogate novel biological questions [39, 97]. Digital pathology can improve quantitative assessment of pathological specimens, including quantitative measures of native spatial organizations of immune cells in tumors. In situ spatial analyses of tumor infiltrating immune cells presents as a valuable prognostic and theranostic biomarker. Spatial patterns of immune cell infiltration in tumors significantly correlates with cancer prognosis, as illustrated by the Immunoscore, an immune parameter that is significantly better in predicting survival than standard TNM staging of colon cancers [98]. The Immunoscore method has been promising in lung cancer with significant correlation between 20  stromal PDL1+ immune cells and PD1+ intraepithelial tumor infiltrating lymphocytes with improved disease-specific survival, though additional validation is needed in multicentric studies [46]. In addition, spatial organization of immune cells, such as tumor infiltrating T cells, significantly correlates with immunotherapy, including immune checkpoint inhibitors in melanoma [99]. Thus, spatial profiling of immune cell infiltration in tumors holds significant potential to improve prognostic accuracy of lung tumor subclassification. Certainly, further investigations are warranted to refine this strategy for lung cancer, as many different types of immune cells correlate with survival and it will be critical to ascertain those that are most clinically relevant in a succinct manner  [100, 101]. With the advancement of computer vision and digital pathology, the notion to integrate computer aided diagnosis (CAD) within the pathologist’s workflow is crystallizing, as CAD has valuably improved accuracy, reproducibility, and efficiency of diagnostic tests, such as the automated screen of Pap smears [97]. Digital pathology imaging systems have been approved by Health Canada, including OmnyxTM Integrated Digital Pathology (IDP) system by General Electrics (GE) Healthcare [102, 103]. The first whole slide imaging (WSI) system, Philips IntelliSite Pathology Solution (PIPS), has been approved by the FDA for marketing in 2017, a significant landmark for the adoption of digital pathology into clinical practice [104]. Both digital pathology imaging systems have received Conformité Européene (CE) Mark status in Europe [105, 106]. Through globalization, digital pathology is poised to transform clinical practice, and implementation of guidelines for these technologies will need to be revisited [107]. Aforementioned in Section 1.3 and in this section, conventional methods of histopathology carry several limitations, including accounting for spatial architecture of tumors. Image analysis software for in situ study of tissue presents as viable and robust method for the 21  integration of CAD in clinical pathology [108-110]. Several commercial software that enumerate cellular composition and analyze spatial architecture of tumor microenvironments exist for non-clinical settings, summarized in Table 1.1 and Table 1.2. The workflow of such commercial systems still carries several limitations before routine implementation in clinical decision making is possible [111]. Challenges in workflow are a significant aspect that needs to be optimized and then standardized for effective clinical translation, including for applications for digital pathology, such as telepathology [84].  Table 1.1 Several commercial non-clinical software for quantifying immune cell counts and spatial analyses   Company Commercial software Description PerkinElmer [112] inForm Tissue Finder  Analyzes, quantifies, and phenotypes immune and other cells in tissue in situ  Can analyze immunofluorescence in images, and can separate weakly expressing and overlapping marker, and perform scoring (percent positivity, 0/1+/2+/3+, co-localization)  Quantifies biomarker from tissue regions (tumor/stroma), including locates tissue and structures by training  Batch processing of images and merges data from images into summary data files inForm Cell Analysis  Tissues understood at cellular level (cells, nuclei, membrane for biomarker expression) via cell segmentation   Visualize, analyze, quantifies, and phenotypes immune and other cells in situ  Immunofluorescence in images tumor/stroma  Weakly expressing or overlapping markers, batch processing, scoring (percent positivity, 0/1+/2+/3+, co-localization, and more) PhenoLOGIC Machine Learning  Performs machine learning of different cell populations and regions  Definiens [113, 114] Tissue Studio  Quantifies morphology and expression profile in relation to tissue architecture  Performs image registration of adjacent sections Developer XD  Quantifies relationships and morphologies at different levels of analysis (regions, populations, cell and cell components) Image Miner  Quantifies tumor heterogeneity of biomarker expression in individual cells Indica Labs [115] HALO  Singleplex and multiplex stain analysis, including dual staining  Quantifies various object features in tissues  Spatial analyses of stained markers   Performs image registration of adjacent sections  Refer to Table 1.2 for details on software modules       22  Table 1.2 HALO software modules for oncology research    *Description from HALO by Indica Labs [115]; CISH = chromogenic in situ hybridization; FISH = fluorescent in situ hybridization; IHC = immunohistochemistry; ISH = in situ hybridization; SISH = silver in situ hybridization     Software modules  Description* Double stain IHC  Analyzes single or double stained cells related to cytoplasmic or nuclear positivity Membrane IHC Quantification  Quantifies membranous stain in regions or individual cells Cell-Based Immunofluorescence  Quantifies immunofluorescence of up to three markers Fluorescent Membrane Quantification  Quantifies membranous fluorescent stain Brightfield Microvessel Quantification  Quantifies and measures markers of vessels (CD31, CD34, Factor VIII) Fluorescent Object Colocalization  Counts one or two object types based on size, shape, percentage of colocalizing areas, including intensity measurements Immune Cell Proximity  Counts immune cells and measures proximity in relation to other structures (number of immune cells, number of proximate immune cells, percentage of proximate immune cells, tissue area, and tumor area) Spatial Analysis  Performs spatial analyses at different levels in tissue  Nearest neighbor analysis: measures average distance and number of unique neighbors between two populations of cells or objects   Proximity analysis: measures number of objects or cells from another object or cell within given distance away   Infiltration analysis: measures number of objects or cells within a given distance of an annotated region High-plex Immunofluorescence  Analyzes up to eight immunofluorescently labeled marker  Can be overlaid with Spatial Analysis module Multiplex IHC  Separates up to five stains   Analyzes subcellular compartments (nucleus, cytoplasm or membrane) and measures size and staining intensity SISH & Dual CISH Quantification  Measures gene amplification by counting probe signals in individual cell nuclei (total number of cells, spots, average number of spots per cell, histogram of frequency of cells containing signals) Chromogenic RNA ISH  Brightfield RNA ISH quantification of RNA probes for each cell (cell counts, signal counts, average signals per cell, signal sizes and optical densities, and histograms ) Multiplex RNA FISH  Multi-channel fluorescent images analyzing up to four RNA ISH probes   Average expression of individual probe in region of analysis and also expression profile for each cell Amplification & Deletion FISH  Measures gene amplification and deletion by measuring two probes in cells (including average copy counts and ratios of all cells within region analyzed) Break-Apart & Fusion FISH  Multi-channel fluorescent images and measures break-apart and fusion (including copy counts and fusion counts) Multiplex DNA FISH  Measures up to four DNA FISH probes to measure single cell DNA expression profiles for each cell and measures signal count for individual probe in region analyzed  Tissue Classification  Machine learning to identify tissue types related to color, texture, and context (train to detect tumor regions) Serial Section Analysis  Creates classifier in a reference slide that can be applied to serial sections Tissue Microarray Circulating Tumor Cell (CTC)  Analysis of whole slide tissue microarray (TMA) image to identify tumor, quantify cell-based IHC, ISH, or immunofluorescence  Protein Foci Quantification  Spot counting tool to quantify CISH, SISH, RNA ISH signals and cellular protein foci (i.e. gamma-H2AX) to determine various parameters (stain intensity, total number of cells, total number of DNA repair foci, size of foci, and number of foci per cell) 23  1.1 Thesis theme   1.1.1 Rationale  Scientists, Dr. Calum MacAulay and Dr. Martial Guillaud’s research team at the British Columbia Cancer Research Centre (BCCRC) developed a hyperspectral image analysis system capable of measuring percentage of cell types, cell-to-cell spatial relations analysis, and image registration of hyperspectral image data derived from multiplex IHC stained tissue (Figure 1.5). The cell-to-cell spatial relations analysis of this hyperspectral image analysis systems is an extension of  previous software by Dr. MacAulay and Dr. Guillaud’s research team to analyze cell proliferation with singleplex IHC of Ki67 and a multiplex FISH analysis workstation to identify clonally related cells [116] [117]. The underlying motivation of this study is to determine if the spatial analysis metric correlates with prognosis and treatment outcome of lung cancer patients, as this metric has not been reported by existing commercial systems. Quantitative metrics of tumor histology with CAD systems have not yet been clinically adopted with the government-approved digital pathology systems (mentioned in Section 1.4) or non-clinical commercial systems (described in Table 1.1 and 1.2). Further studies will be required to rationalize the use of these metrics, which this system would contribute towards. However, this underlying motivation is beyond the scope of this thesis. In this proof of principle study, the workflow of this platform in a non-clinical setting will be first assessed and optimized in this pilot study.     24   Figure 1.5  Overview of software modules for BCCRC hyperspectral image analysis system. Spectral unmixing figure courtesy of Dr. Calum MacAulay and Paul Gallagher. Asterics figure courtesy of Dr. Calum MacAulay.          25  1.1.2 Objective and hypothesis  The overall goal of this study is to facilitate the implementation of the workflow of the BCCRC hyperspectral image analysis system by Dr. Calum MacAulay and Dr. Martial Guillaud’s research team for the use of improved prognostic assessment and clinical management of lung cancers. As a first step, we introduce this pilot study with a specific hypothesis. The specific hypothesis of this work is that the workflow of the BCCRC hyperspectral image analysis system will facilitate the determination of spatial relationships of cell populations within the tumor microenvironment. The objective of this work is to assess the workflow of this new BCCRC hyperspectral image analysis system using the tumor microenvironment of lung AC as a prototype. Given their significant clinical relevance in lung cancer pathogenesis and subsequent mechanistic role in immunotherapeutic response, immune cells were a sample feature of the tumor microenvironment used to verify the new BCCRC hyperspectral image analysis system.   1.1.3 Specific aims and thesis outline  The following specific aims addressed the objectives and hypothesis of this study, and are also visually summarized in Figure 1.6:   Aim 1: Assess the workflow of the imaging method in facilitating the determination of spatial relationships of cell populations using protein markers in a single plane   Aim 2: Assess the workflow of spatial alignment between tissue sections 26  Chapter 2 is a composite methodology chapter, specifying the samples, digital image acquisition, digital image processing, and spatial analyses performed to address these aims. Chapter 3 describes results of spatial analyses (first, cell-to-cell spatial analyses in a single plane of tissue and subsequently, cellular distribution in multiple planes of tissue), technical factors affecting workflow optimization of this method, and potential solutions for technical factors. Lastly, Chapter 4 describes the significance of this study for non-clinical and clinical settings.     Figure 1.6  Overview of methods of specific aims of this thesis. Methods addressing Aim 1 are demarcated in the red box. Methods addressing Aim 2 are demarcated in the blue box. For Aim 2, tumors were selected from Aim 1 to perform image registration analysis. AC: adenocarcinoma; FFPE: formalin-fixed paraffin-embedded   27  Chapter 2: Material and Methods  2.1 Samples  This study was performed with a cohort of 21 FFPE primary lung AC (Table 2.1). Specimens were accrued from the Tumor Tissue Repository of the British Columbia Cancer Agency or Vancouver General Hospital with informed, written patient consent and approval from the University of British Columbia – British Columbia Cancer Agency Research Ethics Board.   Table 2.1 Clinical features of lung AC cohort.    Case Ethnicity Age Stage Gender Smoking status Diagnosis TNM T N M Mutation Recurrence Chemo (Regimen) 05L9 Caucasian 71 IA F CS Unk T1bN0M0 1 0 0 KRAS n/a    No 06L3 Caucasian 67 IIA F CS ACI T2aN1M0 2 1 0 WT n/a Yes (CDDP/VNB) 06L14 Caucasian 57 IIA F CS ACI T2aN1M0 2 1 0 KRAS n/a Yes (CDDP/VNB) 06L50 Caucasian 78 IA F CS ACI T1aN0M0 1 0 0 KRAS Yes No 06L57 Caucasian 53 IIIA F CS ACI T2bN2M0 2 2 0 KRAS Yes No 05L4 Asian 82 IIB F NS ACI T3N0M0 3 0 0 EGFR n/a No 05L39 Asian 86 IB F NS N/A T2N0M0 2 0 0 KRAS n/a No 05L49 Asian 77 IB F NS N/A T2N0M0 2 0 0 WT Yes No 05L54 Asian 77 IA F NS N/A T1N0M0 1 0 0 KRAS Yes No 06L29 Asian 71 IB F NS ACI T2N0M0 2 0 0 WT n/a No 06L71 Caucasian 68 IA F NS N/A T1N0M0 1 0 0 WT n/a No 05L12 Asian 66 IIIA M NS Invasive  MIP T2aN2M0 2 2 0 EGFR Yes Yes (CDDP/VNB/ Erlotinib) 05L27 Asian 77 IIA F NS Unk T2aN1M0 2 1 0 EGFR Yes No 06L10 Caucasian 72 IB F CS Invasive AC mixed MUC and non MUC-  prod ACI T2aN0M0 2 0 0 WT n/a No 06L42 Caucasian 82 IA F EX ACI T1bN0M0 (IA) 2 0 0 Unk n/a No 05L46 Unk 90 IIA F EX PAP  (some ACI) IIB 2 1 0 Unk Yes No 06L30 Asian 58 IIA M EX ACI IIB 2 1 0 Unk Yes No 06L74 Asian 60 IIA F EX PAP IIB 2 1 0 Unk Yes No (CDDP/VNB) 05L14 Unk 73 IIA F CS Invasive ACI T1N1M0 4 1 0 Unk Yes No 06L19 Unk 45 IA M CS Solid AC T1N0M0 1 0 0 Unk Yes Yes 05L17 Unk 70 IA M CS Invasive ACI T1N0M0 1 0 0 Unk n/a No  Patients are radiation naïve. Stage refers to the 7th Edition of the Tumor Node Metastasis (TNM) classification system for lung cancer.  ACI: acinar, CDDP: cisplatin, MIP: micropapillary, MUC: mucinous, PAP: papillary, WT: wildtype, VNB: vinorelbine, Unk: Unknown 28  2.2 Multiplex immunohistochemistry (IHC)  FFPE tumors were sectioned at 4 μm and baked overnight at 37oC. Slides were manually deparaffinized with xylene to a 50% xylene to ethanol solution, then through a gradient of alcohol to water (100%, 90% 70%) and rinsed with distilled water (dH2O). Antigen was retrieved using the decloaking chamber plus with Diva decloaker (Biocare, United States) at 125oC for 30 seconds. Slides were added to Intellipath FLX rack, rinsed with dH2O and borders marked with super pap pen and covered with TBS buffer. Slides were loaded into the pre-programmed, reagent-loaded Intellipath FLX (Intellipath FLX protocol: DS1 for adaptive immune resistance panel; DS2 for T&B Cell Panel, excluding CAT hematoxylin counter stain for both protocols). Endogenous peroxidase was blocked with peroxidazed-1 dispensed by the Intellipath FLX for 5 min at room temperature. Non specific blocking was performed with background sniper 1 dispensed by the Intellipath FLX for 10 minutes at room temperature. Cocktail was manually applied to diluted primary antibodies to slides (Cocktail 1: Anti-CD3 + Anti-CD8 + DaVinci Green diluents; Cocktail 2: Anti-PD-L1+ Anti PD1+ DaVinci Green diluents, Table 2.2 and 2.3) for 30 minutes at room temperature. Mach2 Double stain (stain #1 for adaptive immune resistance and stain #2 for T&B cell) was dispensed by the Intellipath FLX for 30 minutes at room temperature. Ferangi blue chromogen was dispensed by Intellipath FLX for 8 minutes at room temperature. DAB chromogen was dispensed by Intellipath FLX for 5 minutes at room temperature. Slides were removed from Intellipath FLX, rinsed well with dH2O, and slides were added to dH2O at 50oC and transferred to pre-warmed SDS-glycine pH 2.0 [118]. Slides were incubated with periodic agitation in SDS-glycine pH 2.0 (Pirici et al.) and washed with dH2O for 45 minutes at 50oC [118]. Slides were placed into Intellipath FLX racks and covered with 1x 29  TBS Automation Wash Buffer and loaded into pre-programmed, reagent-loaded Intellipath FLX. The Intellipath FLX Protocol was followed for mouse primary, and the peroxidase blocking and non specific blocking steps were excluded, and mouse AP polymer, Warp Red chromogen, and CAT hematoxylin counterstain were included. Diluted primary antibody was manually applied for second round of staining for 30 minutes at room temperature (Primary Antibody 1: Anti-CD79a + DaVinci Green diluents; Primary Antibody 2: Anti CD8 + DaVinci Green diluent). Mouse-AP polymer was dispensed by the Intellipath FLX at 30 minutes at room temperature. Warp Red chromogen was dispensed by Intellipath FLX for 7 minutes at room temperature. The counterstain, CAT hematoxylin (1:5 dilution with dH2O), was dispensed by Intellipath FLX for 5 minutes at room temperature. Slides were removed from the Intellipath FLX and rinsed well with dH2O. Slides were air dried and coverslipped with Ecomount. Stained tissues were scanned as whole slide images with Pannoramic MIDI scanner and viewed with the companion digital pathology software Pannoramic Viewer (3D HISTTECH, Hungary).   Table 2.2 Summary of immune cell marker antibody panels. Panel Name Antibody* Company Antibody clone Localization Spectral properties (chromogen) Positive Cell Types T&B Cell (TB) CD3 Spring Biosciences Clone SP7 Membranous Blue  (Ferangi Blue) T-cells CD8 Cell Marque Clone C8/144b Membranous Brown  (DAB) Cytotoxic T-cells CD79a Spring Biosciences Clone SP18 Membranous/ Cytoplasmic Red  (Warp Red) Pre-B cells, B cells and plasma cells Adaptive Immune Resistance (AR) PDL1 Spring Biosciences Clone SP142 Membranous Brown  (DAB) T and B cells, tumor cells, macrophages PD1 Cell Marque Clone NAT105 Cytoplasmic Blue  (Ferangi Blue) T and B cells CD8 Spring Biosciences Clone SP7 Membranous Red  (Warp Red) Cytotoxic T cells *Control tissue was tonsil   30  2.3 Hyperspectral imaging  2.3.1 Imaging selection criteria  Regions of interest (ROI) in tumors were defined for image acquisition due to limited computer memory for storage of whole slide images (WSI), downstream digital image processing files. ROI were manually selected with high immune cell infiltration in order to obtain sufficient signal detection to test spatial analyses. First, a lung pathologist selected areas with high immune cell infiltration in 20 out of 21 whole slide images of lung AC stained with the T&B Cell panel by annotating WSI of tumor sections scanned with Pannoramic Viewer (3D HISTTECH, Hungary) (Figure 2.1). One tumor (case 06L19) was not annotated for area selection by the lung pathologist due to small tissue dimensions, and was thus, included for method establishment only and not for spatial analyses (Section 2.3 and 2.4). A total of five images were then manually sampled within pathologist selected areas of T&B Cell images that had high stain intensity in red–green–blue (RGB) WSI images (Figure 2.1). Approximate locations of adjacent areas in the tumors stained with adaptive immune resistance were identified by comparing with WSI stained for T&B Cells (Figure 2.1).   31   Figure 2.1 Sampling of tumors for image acquisition. (A) Whole scan image (WSI) section with multiplex IHC staining of T&B Cells. Red box denotes lung pathologist (Dr. John English) selected areas with high immune cell infiltration. Whole area was not imaged and analyzed due to limited computer memory for image storage and subsequent digital image processing. Black circle denotes ROI with high stain selected for image acquisition.  Yellow X indicates origin used for image acquisition. (B) WSI of adjacent section stained with adaptive immune resistance markers and corresponding ROI for image acquisition.     32  2.3.2 Image acquisition  The hardware for the hyperspectral image acquisition workstation was a Zeiss Axioscope 2 Mot Plus (Canada) connected to an Andor Neo sCMOS Camera (United Kingdom), CRI VariSpec Liquid Crystal Tunable Light Filter (Canada), and Marzhauser Wetzlar SCAN Series Stage (Germany) connected to a desktop personal computer (PC) (with a BitFlow Frame Grabber (United States) and installed with MATLAB R2014a) using two serial cables and serial to Universal Serial Bus (USB) converters (Figure 2.2A and 2.2B). The hyperspectral image capture software program, ImageCapture.fig, was developed and programmed in MATLAB by Paul Gallagher, a member of Dr. Calum MacAulay’s research team. This image acquisition software enables controls for lamp intensity, filter wavelength, microscope stage control, calibration (tiling), and spectra imaging range. To image the same region in adjacent sections, a common landmark was identified between tissues to orient coordinates, referred to as the origin (Figure 2.1). Then landmarks within the five ROI selected from Section 2.3 were identified. ImageCapture.fig has a control for region selection and can record coordinates to locate regions (Figure 2.2C). Indexes can be saved, and ‘Jump’ feature available enabling navigation between ROI.  Briefly, ImageCapture.fig was selected, and ‘Open end guide’ was selected, opening the graphical user interface (GUI). The ‘Run’ button was selected to execute the program. In the GUI, the program was connected to the microscope hardware equipment by selecting ‘Axioscope’, ‘CRI filter’, and ‘Andor Neo’ in the corresponding drop down. The microscope settings were maintained at 82/255 lamp voltage and 600 nanometer (nm) filter in order to acquire an image on live view with optimal exposure and contrast (Figure 2.2C). Origin of 33  interest of tissue sections were navigated on live view with the ocular lenses at a 10x magnification and coordinates of this location were saved. A single bound for imaging was also set for each landmark of five ROI to create a list of bounds. A directory was inputted for files to be saved. The microscope condenser was centered, and an image of a flat-field (an area of the cover slipped glass slide containing no tissue and with limited slide artifacts in the field of view) was acquired per image acquisition run for flat-field correction. For the adjacent slide, previous coordinates were reloaded, and the joy stick was used to fine tune location of ROI coordinates to select new centers and to save regions. Image stacks of sixteen different illumination wavelengths in steps of 20 nm (420, 440, 460, 480, 500, 520, 540, 560, 580, 600, 620, 640, 660, 680, 700, 720 nm) were acquired as preset three-by-three image tiles. All five ROI per slide of tumor were imaged sequentially across the whole slide of tumor. After hyperspectral imaging, a coarsely stitched image, a print screen of the three-by-three tiles, was assessed to determine whether the imaged region was accurately imaged. Output image file extensions of this hyperspectral imaging software were ‘.TIF’ and named with the following file name: ‘SlideID_RegionNumber.TIF’. Images from adjacent sections were confirmed for image quality (i.e. out of focused images) and compared for centering of landmarks. Out of focused images and images with uncentered landmarks were reimaged.        34                    Figure 2.2   Image acquisition hardware and software of BCCRC hyperspectral image analysis system. (A) System hardware. Manual image acquisition was performed with the setup. Microscope illustration adapted from image courtesy of Paul Gallagher. Wiring of hardware to power supply not shown. Dimensions of hardware drawn not to scale. (B) Dark room containing microscope. (C) GUI of ImageCapture.fig. Figures B and C courtesy of Paul Gallagher. A B C 35  2.4 Digital image processing  2.4.1 Image stitching and spectral unmixing  All three-by-three image tiles were stitched with the MATLAB image stitching software, Tile_processing.fig, developed by Paul Gallagher on a PC with the following minimal requirements: minimum 16 gigabyte (GB) of random-access memory (RAM) (32 GB of RAM used), Windows 7 64-bit operating system (OS), and sufficient hard disk (HD) space per run (varies with image size and number of spectral components, i.e. for a 5-spectral component tissue microarray (TMA) spot 600 MB of HD space per run is required), where a graphics processing unit (GPU) was unused, and MATLAB R2014a or later version installed (MATLAB R2014a was used in this study). Hyperspectral_Processing.m, also programmed by Paul Gallagher, was used to perform spectral unmixing of acquired images (Figure 2.3). Briefly,      Figure 2.3  Graphical user interface (GUI) of Hyperspectral_Processing.m. Directory indicates location to insert files. All indicates selection of spectral unmixing of internal files. Spectra file indicates file containing pure spectra measured at specified illumination wavelengths. Scale factor indicates value to normalize grey-level intensities of output image files. Figure courtesy of Paul Gallagher. 36  under ‘Stitching’ the directory of stitched files was specified in the child window of ‘Directory’. ‘All’ was also selected in order to iterate spectral unmixing in all files in directory. Under ‘Unmixing’, an Excel file (referred to as the spectra file) containing sixteen illumination wavelengths of five pure spectra (Hematoxylin (Hx), Red, Blue, Brown, Black) was loaded into the child window under ‘Spectra File’. Black spectrum was used to spectrally unmix anthracotic pigmentation that may be present in the lung. The counterstain illumination wavelengths were listed first in the spectra file, required for downstream segmentation software. Under ‘Constraints’, ‘Positivity’, and under ‘Spectra Evolution’, ‘Fixed’ was selected. The unmixing program was run by selecting ‘Un-mix ALL’. The spectral unmixing was first run with a scale factor equal to zero (autoscaling). The spectral unmixing was then repeated with the highest sampled scale factor to normalize grey-level intensities of all images. Output files included a folder entitled ‘imageInverted’ of grayscale images and inverted spectrally unmixed images of each channel.    2.4.2 Segmentation  To detect single cells in images, segmentation (process to partition a digital image to identify objects of interest) was performed to identify nuclei of cells. Segmentation was manually performed with DUnit, a digital image processing software developed by Dr. Calum MacAulay and Dr. Martial Guillaud’s research team. To increase throughput of manual segmentation, three independent observers (Dr. Calum MacAulay, Dr. Katey Enfield, and I) segmented ROI images processed by image stitching and spectral unmixing. Briefly, the software DUnit was implemented to segment images. Grayscale images of stitched and spectrally unmixed image 37  files of all ROI were individually imported to DUnit where all indexes of images were selected. Files were then saved as with a ‘.unit’ file extension. Files were segmented with the following settings selected (‘Cluster A’, ‘Cluster B’, ‘Inflections’ were checked, and primary red (R)). Composite image settings were not selected. Thresholds were then selected for nuclei detection, where autothreshold was first tested and then optimized with manual threshold adjustment if the autothreshold was not sufficient to segment all cells. Efficacy of segmentation was visually confirmed by assessing Voronoi tessellation, and vertices (seed points) recognizing nuclei at various magnifications of the monochromatic image of hematoxylin, viewed by selecting RGB color channels for counterstain only (‘Filter 0’). Images were re-segmented if necessary by optimizing the threshold value. Individual image files of segmented cells were then saved.   2.4.3 Classification  In addition to segmentation, DUnit was also implemented to establish staining thresholds for cell classification. Three independent observers (Dr. Calum MacAulay, Dr. Katey Enfield, and I) established thresholds for each stain of images for both differently stained sections. Each observer established thresholds for each panel on one set of ROIs in both different sections. Briefly, DUnit was opened and ‘Classification’ selected to use the feature ‘test.tree’. In the GUI, corresponding integrated intensity (II) was selected for each channel and the program executed. The RGB map was modified to view the stain of interest. Thresholds were optimized by testing different threshold value in the threshold ‘Menu’, ‘Options’, and ‘Edit Record’ control. Threshold values were saved, and the program was run. Thresholds were evaluated with the scatter plot in ‘Features’ and ‘Gallery’ visualizing individual positive cells that can be sorted  38                                 Figure  2.4   Gallery for validation of thresholds. (A) Image 05L27_5 analyzed with Gallery for CD3+ cells in the T&B Cell Panel. Teal dots indicate areas DUnit recognizes segmented CD3+ cells. (B) GUI of gallery in DUnit. The gallery lists all of the identified positive cells,  Each single array (red box) of the matrix in Gallery indicates image objects recognized with integrated intensity (II) matching threshold highlighted corresponding to individual cells (for blue = II3). Teal letters indicates integrated intensity (II). (C) Cells in gallery with low, intermediate, and high intensity staining for CD3+. Cells correspond to magnified array in Gallery matrix. Red contour indicates mask of cell nuclei indicating area with intensity measured by established threshold in DUnit. Number below indicates value of II.  Array  39  based on positivity threshold and toggled to view segmentation (Figure 2.4). Groups of single stains of interest were assessed by visual confirmation in the gallery of single positive cells to    Gallery for validation of thresholds evaluate whether positivity threshold selected was appropriate. Thresholds in ‘test.tree’ were optimized based on this assessment. Groups of other cell classifications were hidden. All thresholds were manually recorded. The average of the three optimized thresholds was calculated to account for intra-observer variability for the T&B Cell. The average of two optimized thresholds was calculated for AR panel. The optimized tree, entitled ‘immuno.tree’, for each panel was then applied to all images (105 images per panel).  2.5 Spatial analyses  Spatial analyses using the BCCRC hyperspectral image analysis platform were performed in a single plane and in more than one plane. Section 2.5.1 describes cell-to-cell spatial analyses in a single plane with custom software (Asterics), and Section 2.5.2 for cell distribution spatial analyses in more than one section with custom image registration software (Imreg.m). For these spatial analyses, case 06L19 was excluded, as areas of this tumor were not annotated by a pathologist.  2.5.1 Cell quantity and cell-to-cell spatial analyses  The cell sociology software, Asterics, developed by Dr. Calum MacAulay and Dr. Martial Guillaud’s research team, was implemented to analyze cell quantities and cell-to-cell spatial analyses. Asterics uses input files with ‘.jb0’ file extension. First, all ‘.jb0’ images of processed 40  images were moved into one single directory in one file, and loaded into Asterics to prime the program. In the ‘Sociology’ features window, ‘Options’ was selected, and all settings were deselected, except for ‘Morphology’ features, ‘Architectural’ features, and ‘QImmuno’ features (all groups, 4 moments, layer types). In ‘Batch Processor’ window, ‘Options’ were selected and searched for ‘.jb0’ file type and selected for architectural features and all corresponding ROIs. ‘Input’ and then ‘Calculate’ were selected. Microsoft Excel was opened to search directory of ‘.jb0’ files as Asterics output files. Output files of Asterics containing cell quantity and cell-to-cell spatial analyses was entitled ‘Arch.txt’ and were in the same location as the main directory. ‘Arch.txt’ was modified into a Microsoft Excel file and was formatted by text delimited and space. Graphs of average and standard deviation (SD) of all ROI of tumors in regards to cell percentages, mean neighborhood groups (MNG), and variance (VAR) of neighborhood groups were individually plotted with commercial statistical software package, Statistica Academic. Line of best fit across averages value of all five ROI (Percentage, MNG, or VAR) and bands of 95% confidence intervals (CI) were also plotted in Statistica Academic. Values equating to -999 indicate that not enough cells are present for accurate analysis and were excluded in the data set. Analyses are further described in Section 3.4 of this thesis.   2.5.2 Image registration  Three lung AC were selected out of the cohort (from Section 2.1) with cell count data available for all five ROI with the highest average of percentage of any CD8+ cells in both adjacent sections for image registration analysis. The image registration software programmed by Paul Gallagher, Imreg.m, applies geometric transformation to digital images of adjacent sections 41  based on similar intensities in images. Hardware used was a desktop PC (Cooler Master Co., Ltd) with MATLAB 2016a installation. Syntax that was modified and MATLAB functions of Imreg.m are summarized in Figure 2.5 and Table 2.3.  Digital images were assigned either as the moving image (undergoes geometric transformation) or fixed image (remains unchanged) with ‘Imread’ command (Figure 2.3). Image file names were created and directory to save registered images were specified with ‘Imwrite’ command. File name of output files of registered images were as follows: ‘Stain_FixedImage_CaseID_RegionNumber.TIFF’, where ‘FixedImage’ denotes if T&B Cell or Adaptive Resistance stained section was assigned as fixed. Modifications to script were saved each time, and then the program was executed to issue image registration commands. ‘Command Window’ and ‘Workspace’ were not manually cleared after each run, as the script contains these internal commands. The reciprocal assignment of fixed and moving images for each image pair (T&B Cell and corresponding Adaptive Resistance image) was also performed with Imreg.m to test reproducibility of image registration.   Moving = Imread(‘directory\filename.TIF’); Fixed = Imread(‘directory\filename.TIF’);   Imwrite (img, ‘directory\filename.TIFF’)   Figure 2.5   Syntax modified in Imreg.m to perform image registration. (A) Syntax for assigning moving and fixed images for image registration with Imreg.m in MATLAB. Directory refers to directory of grayscale image for image registration. (B) Syntax for assigning output file containing registered image. Directory refers to directory to save output file. Single quotations operator (‘’) indicates character string. Semicolon operator (;) indicates end of command and suppresses display of command in Command Window.   A B 42  Chapter 3: Results and Discussions  3.1 Image acquisition and spectral unmixing  We acquired images by sampling areas of tumor sections with the custom image acquisition system (virtual biopsy). Spectral unmixing was required to analyze specific wavelengths of light that contain different information, including stains of markers that correspond to different immune cells. Computer memory for image storage and management and image acquisition time are important factors considered in digital pathology workflow, and these factors are as important for other imaging modalities, such as CT screening [6]. File sizes present as a unique challenge for digital pathology images, as they are dramatically larger, even when compared to CT scans [89]. In this study, a total of 210 images were imaged, 105 images for each panel (5 ROI imaged per lung AC × 21 lung AC = 105 images) with the described hyperspectral imaging system. Sample regions were imaged, as file size for hyperspectral imaging of whole slide images with this system would exceed space for file storage. The project file size with all image processing performed was 456 GB, where average file size of individual ‘Scaled_InvertedConcentrations.TIF’ image (T&B and AR) was ~148 megabyte (MB) per image, with additional image data output files acquired from spectral unmixing (Figure 3.1) This file size exceeds the capacity for rapid transfer through an existing BCCRC computer network, limiting file accessibility in our research setting. File size represents an important issue with implications for telepathology, which rely on placing image datasets on a server (or cloud) for tissue evaluation [81]. Commercial systems that use remote image management system and  43    Figure 3.1  Spectral unmixing of acquired images. (A) Spectral unmixing of different spectral components of sample ROI of T&B cell hyperspectral image data (B) Spectral unmixing of different spectral components of sample ROI of adaptive immune resistance hyperspectral image data.    A B 44  cloud server for various electronic devices for efficient access, though data security will need to be addressed [119]. Two potential approaches could be proposed in the future to address this challenge: (1) acquiring a larger computer network or (2) integrating image compression methods to reduce file size [120]. Regarding the former approach, though conspicuous, this potential solution may be more uneconomical. Continual studies using this system would require frequent acquisition of new networks to perform new studies when memory in networks becomes saturated from large file size. Prospectively, this approach would not be sustainable in a clinical setting where a large repository of data would be collected from a frequently expanding cohort. The latter approach is inclined to be more sustainable by conserving existing memory. Furthermore, this approach has shown to be effective. Previously, investigators have shown that lossy image compression produces similar image analysis results as lossless image compression where these results did not affect interpretation of pathologists [120-122]. Compatibility of digital image processing software with diverse image file extensions, including compressed images, will enable more versatility of programs, as illustrated by commercial systems [123, 124].  In addition to economical conservation of computer memory, approaches to more efficiently maintain image quality was another aspect observed in this study. With the BCCRC image acquisition system, approximately seven days were required to manually acquire all 210 images. Out of focus scans were observed in these imaged slides, which are also an existing challenge for current digital pathology platforms [61]. Out of focus or blurred images may be caused by panning artifacts including variations in tissue thickness, air bubbles, tissue folds or dust, as reported for whole slide scanners [125, 126]. Rescanning of images is a method to troubleshoot images containing artifacts [127]. In this study, an additional six days were required 45  to reimage ROI to account for image quality, including images that were out of focus or uncentered. A total of 32% of all ROI images were reimaged (for skew or out of focus together) [67 reimaged / 210 total images] with the hyperspectral image acquisition system, indicating that the majority of images were imaged in focus, though certain images needed to be reimaged. Collectively, a grand total of thirteen days were required for hyperspectral image acquisition with the BCCRC hyperspectral image acquisition system. Currently, out of focus images were assessed manually. It would be interesting to integrate an automated method to assess image quality as in Ameisen et al. 2014 for a quantitative and more efficient way to determine image quality [128]. Presently, the workflow of this system will need to take into account additional time for reimaging for sufficient image quality for downstream analysis if a larger cohort is to be used.   Alternatively, imaging slides in batches (including a method used by commercial systems) or where microscope stage can be moved in larger step sizes can also improve image acquisition performance [114, 123, 124, 127, 129]. Reducing the number of illumination wavelengths to perform image acquisition and spectral unmixing is anticipated to also decrease the amount of running time required for image acquisition  [130]. This last approach was tested to assess whether the current image acquisition system, independent of additional auxiliary electronic units could have improved performance.   To determine whether fewer illumination wavelengths could improve performance while maintain image quality, the sixteen illumination wavelengths was reduced to six illumination wavelengths (420, 460, 520, 600, 660, 720 nm) (Table 3.1). These six illumination wavelengths were selected due to a similar concentration map to the sixteen illumination wavelengths that were initially used (Figure 3.2). Confirmation is required to determine if reducing the number of 46  illumination wavelengths to this fewer amount would produce similar image data to imaging all sixteen illumination wavelengths for hyperspectral images. Thus, a pilot experiment was performed to test whether reducing the number of illumination wavelengths to these selected six illumination wavelengths would produce similar image data as spectral unmixing with all sixteen illumination wavelengths. The experiment was performed for spectral unmixing of the initially acquired images. Results from this experiment will provide evidence as to: (a) whether these selected six illumination wavelengths will provide similar image data in a more efficient manner, and (b) initial evidence regarding whether these fewer number of wavelengths are appropriate for building a multispectral image acquisition system that images only with these fewer steps of wavelengths.  Table 3.1 Spectral components with six and sixteen illumination wavelengths.     Components (optical density, unitless) Wavelengths (nm) Hematoxylin Red Blue Brown Black 420 0.101999 0.286282 0.219142 0.494995 1.074562 440 0.09631 0.317762 0.160342 0.544291 1.077952 460 0.093373 0.407146 0.124488 0.542725 1.115249 480 0.115964 0.577312 0.144825 0.476438 1.15619 500 0.158563 0.772731 0.236577 0.383639 1.20467 520 0.204072 0.932973 0.36889 0.29274 1.230545 540 0.242716 0.858718 0.557132 0.217443 1.255227 560 0.272961 0.834685 0.743612 0.167585 1.267742 580 0.302181 0.551552 0.892058 0.13098 1.28088 600 0.313563 0.075651 1.050625 0.098693 1.296845 620 0.28804 0.015417 1.036513 0.081003 1.289892 640 0.226668 0.008166 0.976663 0.067658 1.288713 660 0.139926 0.007791 0.925634 0.055718 1.271939 680 0.077127 0.005662 0.631702 0.046867 1.252295 700 0.05005 0.004395 0.367524 0.040852 1.21398 720 0.034088 0.003482 0.195295 0.036276 1.172802  Yellow cell: Selected six illumination wavelengths spectral components with similar concentration map to all sixteen illumination wavelengths 47    Figure 3.2 Spectral unmixing sixteen versus six illumination wavelengths concentration maps. (A) Hypercube of sixteen illumination wavelengths in 20 nm steps between 420 to 720 nm. (B) Concentration maps of six selected illumination wavelengths and sixteen selected illumination wavelengths. Legend indicates spectral components. Figure adapted from Dr. Calum MacAulay and Paul Gallagher.  A B (nm) 48  A subset of tumors was selected for this imaging experiment. Random sampling [Microsoft Excel = RAND() formula] was performed to select three tumors from the lung AC cohort from this study (the unannotated tumor, case 06L19, was excluded). Briefly, a number between zero and one was randomly assigned to cases, and the three cases with the smallest values were selected for spectral unmixing (Table 3.2). For these selected tumors, the five ROI images per tumor were tested for each panel (TB = 15 images total from three tumors, five per tumor; AR = 15 images total from three tumors, five per tumor). This randomly sampled dataset will be referred to as the ‘randomly sampled image data’. An independent MATLAB program named Compare16v6.m, developed and programmed by Paul Gallagher, was written. This    Table 3.2   Random selection of cases for sixteen versus six illumination wavelength experiment.                Random number Case 0.041242 05L54 0.09972 06L14 0.105602 05L12 0.169646 06L29 0.20203 06L42 0.204751 05L9 0.273944 06L30 0.416507 05L17 0.496195 06L74 0.513502 05L49 0.524725 05L4 0.561008 05L46 0.625822 06L57 0.704934 06L3 0.707257 05L27 0.726324 05L14 0.789525 06L10 0.839762 05L39 0.949695 06L50 0.984409 06L71 Turquoise cell = cases selected for analysis. 49  program is capable of spectral unmixing functions, using the same code as Hyperspectral-_Unmixing.m. The program is also able to iterate image subtraction between sixteen and six illumination wavelengths for each spectral component and print the absolute difference matrix as an image output file (subtracted image file). To assess image quality, Compare16v6.m calculates the sum of the pixels (to determine if pixel intensities differ between images spectrally unmixed with sixteen or six illumination wavelengths) and mean of the absolute difference matrix (to determine the variability of array elements between sixteen or six illumination wavelengths). To assess performance time, Compare16v6.m prints the performance time of spectral unmixing of sixteen and six illumination wavelength (the time the script takes to perform spectral unmixing), and of total program run time in a Microsoft Excel file (the entire software, including sixteen and six illumination wavelengths spectral unmixing and other functions).  Compare16v6.m was executed on a desktop PC with MATLAB R2016a installation, as this computer had sufficient computer memory to perform the analysis. Several test runs with single test image and multiple test images were performed to test program execution prior to running with the randomly sampled image data. Image data directory path were independently specified in each run. Spectra file path and spectra Excel files were also specified. Changes to code were saved in the script, and ‘Run’ was selected to execute the program. ‘Clear Workspace’ and ‘Clear Command Window’ was performed in between runs.  It was observed that quantitatively the image data were similar even with reduction of illumination wavelengths. The mean of the absolute difference was close to zero in all cases in both panels. For the T&B Cell cases, the range of mean absolute difference matrix ranged was close to zero (0.000696 to 0.019987) (Table 3.3). The range of mean absolute difference matrix was also close to zero for the AR panel (0.000788 to 0.020319), indicating reproducibility of  50    Table 3.3 Averages of each spectral component of T&B Cell panel for sum of the absolute difference, mean of the absolute difference, and performance time for sixteen compared to six illumination wavelengths.  Case_ROI Sum of the absolute difference matrix  (number of pixels) Mean of the absolute difference matrix  (unitless) Time  (seconds) Hx Red Blue Brown Black Hx Red Blue Brown Black Sixteen Bands Six Bands TB_05L54_1 477035.7 84930.88 101665.4 171446.7 35835.36 0.013233 0.002356 0.00282 0.004756 0.000994 59.51947 35.805 TB_05L54_2 584774.2 150081.5 135334.1 163348.5 31362.73 0.016176 0.004152 0.003744 0.004519 0.000868 60.2355 36.17703 TB_05L54_3 773528 182734.4 168629.7 199053.1 18839.48 0.021434 0.005063 0.004673 0.005516 0.000522 59.43266 34.70936 TB_05L54_4 748245.8 187768.1 167647.7 198960.4 21129.06 0.020736 0.005204 0.004646 0.005514 0.000586 59.6508 34.66419 TB_05L54_5 496068.3 80758.04 94717.89 157657 21566.08 0.013732 0.002235 0.002622 0.004364 0.000597 61.5027 36.1281 TB_06L14_1 730979.2 165173.3 122685.5 196704.6 18921.03 0.02019 0.004562 0.003389 0.005433 0.000523 60.87164 35.22379 TB_06L14_2 717237.5 151701.4 140943.1 189418.9 23593 0.01989 0.004207 0.003909 0.005253 0.000654 60.19385 35.08114 TB_06L14_3 741096.6 152873.6 147442.4 198234.1 25030.74 0.020583 0.004246 0.004095 0.005506 0.000695 60.09994 34.91044 TB_06L14_4 759549.5 151195.8 145258 204337.3 19559.81 0.021025 0.004185 0.004021 0.005656 0.000541 60.2182 34.99122 TB_06L14_5 778430.3 138029.8 167050.7 277801.9 22361.68 0.02163 0.003835 0.004642 0.007719 0.000621 60.11466 35.03641 TB_05L12_1 854932.9 178919.5 196918 232547 45216.28 0.023693 0.004959 0.005457 0.006445 0.001253 60.36455 35.01754 TB_05L12_2 854932.9 178919.5 196918 232547 45216.28 0.023693 0.004959 0.005457 0.006445 0.001253 60.39611 35.05055 TB_05L12_3 778551.6 186278.9 187599.5 208531.2 17838.9 0.021598 0.005168 0.005204 0.005785 0.000495 59.77609 34.55337 TB_05L12_4 725232.4 147326.2 148064.1 192906.9 18010.82 0.020105 0.004084 0.004105 0.005348 0.000499 60.39761 35.21993 TB_05L12_5 797446.5 128715.3 149711.5 211808.3 12403.65 0.022085 0.003565 0.004146 0.005866 0.000344 60.4923 35.23833 Average of cases 721202.8 151027.1 151372.4 202353.5 25125.66 0.019987 0.004185 0.004195 0.005608 0.000696 60.21774 35.18709 SD of cases 114181.4 33107.73 30712.61 29820.27 9906.943 0.003169 0.000917 0.000853 0.000831 0.000275 0.526824 0.4893     51  results with different markers (Table 3.4). These results indicate that spectral unmixed images with only six illumination wavelengths have modest difference in array elements as sixteen illumination wavelengths. These results suggest that quantitatively, image data would be similar despite wavelength number reduction when performing spectral unmixing (Table 3.3). Mean absolute difference was also close to zero in all cases for AR panel, illustrating the reproducibility of this result, that image quality is similar between the spectrally unmixed image with fewer wavelengths (Table 3.4). It was noted that the sum of the absolute difference for hematoxylin was greatest and least for black spectral components in both panels, although values for all spectral components are close to zero. These results suggest that different stains may have different image qualities when reducing number of illumination wavelengths for spectral unmixing. However, all mean absolute differences remain close to zero, suggesting negligible differences between sixteen and six illumination wavelengths in image quality in this experiment. Thus, these results reveal images spectrally unmixed with fewer illumination wavelengths generally have similar image quality from a quantitative perspective. In addition to quantitative assessment, the absolute difference matrix output files were also qualitatively observed to confirm if there are differences in image quality between sixteen versus the selected six illumination wavelengths for spectral unmixing. The latter analysis took into account the whole image. This qualitative analysis would confirm if there are areas in images that tend to differ by providing spatial context to differences. The majority of absolute difference matrix image files were observed to be black for each spectral component, suggesting that overall, few intensity differences between sixteen and six illumination wavelength spectrally unmixed files are presenting, suggesting similar image quality from qualitative analysis. However, a few images had observable qualitative differences in areas of images, such as  52    Table 3.4 Averages of each spectral component of AR panel for sum of the absolute difference, mean of the absolute difference, and performance time for sixteen compared to six illumination wavelengths.  Case_ROI Sum of the absolute difference matrix  (number of pixels) Mean of the absolute difference matrix  (unitless) Time  (seconds) Hx Red Blue Brown Black Hx Red Blue Brown Black Sixteen Bands Six Bands AR_05L54_1 577067.7 86209.94 131928.2 254833.7 43321.96 0.016033 0.002395 0.003665 0.00708 0.001204 61.29854 36.27254 AR_05L54_2 671566.8 118944.1 137125 228782.3 35780.82 0.018615 0.003297 0.003801 0.006341 0.000992 61.84023 36.20765 AR_05L54_3 755079.7 124275 129823.8 210138 22317.13 0.020912 0.003442 0.003595 0.00582 0.000618 61.12668 35.54924 AR_05L54_4 752795.9 126661 135514.6 205286.9 23552.11 0.020914 0.003519 0.003765 0.005703 0.000654 60.88083 35.53811 AR_05L54_5 597651.7 88025.71 90243.95 212725.7 30906.97 0.016579 0.002442 0.002503 0.005901 0.000857 61.77709 36.37333 AR_06L14_1 788080.2 108766.2 132268.6 192845.7 26247.7 0.021822 0.003012 0.003663 0.00534 0.000727 61.27095 35.80502 AR_06L14_2 758844.6 107974.5 120709.8 189323 28443.24 0.021048 0.002995 0.003348 0.005251 0.000789 61.26159 35.69767 AR_06L14_3 754828.7 98470.13 116336.3 194723.2 27719.77 0.020981 0.002737 0.003234 0.005413 0.000771 61.03862 35.78427 AR_06L14_4 765714.5 100870 119944.9 199847.5 21411.09 0.02121 0.002794 0.003322 0.005536 0.000593 61.34608 35.68836 AR_06L14_5 755642.6 112551.3 101050.4 278384.7 25794.89 0.020945 0.00312 0.002801 0.007716 0.000715 61.01091 35.66472 AR_05L12_1 769549.5 121107.4 124339.2 215136.9 45872.4 0.021341 0.003359 0.003448 0.005966 0.001272 60.96933 35.51549 AR_05L12_2 761335.9 111549.8 129397.2 185513.2 25054.4 0.021022 0.00308 0.003573 0.005122 0.000692 61.32784 35.69278 AR_05L12_3 742075.9 113936.7 127543.9 188628.2 22052.68 0.020483 0.003145 0.003521 0.005207 0.000609 61.1215 35.66012 AR_05L12_4 745933.3 117749.7 131671.9 187426.7 26365.39 0.020717 0.00327 0.003657 0.005205 0.000732 61.13218 35.57954 AR_05L12_5 799492.4 129563.7 139695.3 191171.3 21541.26 0.022168 0.003592 0.003873 0.005301 0.000597 61.10332 35.46461 Average of cases 733044 111110.3 124506.2 208984.5 28425.45 0.020319 0.00308 0.003451 0.005794 0.000788 61.23371 35.76623 SD of cases 65388.02 13104.53 13536.48 26804.29 7618.571 0.001803 0.000363 0.000375 0.000746 0.000212 0.270511 0.286249 53  saturation (bright pixels) in output images from image subtraction, though these images comprised the minority of all images tested (Figure 3.3). These differences as observed were multiplied by a factor of 255. We then observed that images with differences in image subtraction (points of saturation) appear to correspond to areas containing tissue fold artifacts in images, such as in Figure 3.3A and Figure 3.3B, or some kinds of dark pigmentation in the tissue, as in Figure 3.3C, potentially due to anthracotic pigmentation in tumors. These results suggest that avoiding tissue artifacts may improve image quality of the selected six illumination wavelength to sixteen illumination wavelength data, though further investigation is required, such as the intensity of image stain caused by artifacts related to nonrigid deformation in tissue or endogenous tissue pigmentation. These qualitative results corroborate the quantitative results of spectral unmixing, that the selected six illumination wavelengths generally will produce little difference in image data to sixteen wavelengths, though certain areas in tissue may be prone to slight differences in image quality than others. Since we verified that image quality was similar with reduced image quality, differences in efficiency were then observed between both methods. Total running time of Compare16v6.m to spectrally unmix with six and sixteen illumination wavelengths of all fifteen images from T&B Cell panel was 34.8 minutes and 35.6 minutes respectively for all fifteen images of AR panel data, indicating stability and reproducibility of Compare16v6.m in analyzing datasets. The average file size for absolute difference matrix output files was ~140 MB (4,427,035,520 bytes total images ÷ 30 images ÷ 1,048,576  bytes per MB = 140.7 MB). These image subtraction image files are comparable in size to the grayscale acquired images in Section 3.1. Performance time of sixteen illumination wavelength spectral unmixing was nearly twice as long to run as six 54                  ABababccC defde f55  Figure 3.3   Sample images of absolute difference matrix between sixteen and six illumination wavelengths with saturation observed. Majority of images for all components were all black. The images shown in this figure are outliers with strong saturation that was visually detected, indicating a difference in image subtraction between spectral unmixing images with the sixteen and selected six illumination wavelength. Differences shown were multiplied by a factor of 255. Lower case letter annotations indicate magnified areas magnified with strong saturation. White arrowheads indicate areas of saturation (bright pixels) in images, (A) Third ROI of case 05L12 in the T&B Cell panel (TB_05L12 _3) of hematoxylin spectral component. a refers to enlargement of region a in A, b refers to enlargement of region b in A (B) Second ROI of case 05L54 in the T&B Cell panel (05L54_2) of red spectral component. c refers to enlargement of region c in B. (C) Second ROI of case 06L14 in the AR panel (AR_06L14_2) of hematoxylin spectral component.   illumination wavelengths for the T&B Cell panel for each image (60.21774 vs. 35.18709 seconds) (Table 3.3). The sixteen illumination wavelengths were also observed to be twice as long for each image in AR (61.23371 vs. 35.76623 seconds) indicating the reproducibility of results (Table 3.4). These results indicate that in addition to having similar image quality, spectral unmixing with six illumination wavelengths is more efficient than sixteen illumination wavelengths. Image file size was not assessed with Compare6v16.m because files were kept as variables local in workspace in order to accurately perform image subtraction. Future studies may want to confirm if significant differences in file sizes are present when using these selected fewer illumination wavelengths. Collectively, these results indicate that using these selected six illumination wavelengths may be valuable to improve the efficacy and workflow of spectral unmixing. These results also serve as initial evidence revealing that development of a multispectral image acquisition system with only these wavelengths may be justified. However, testing with such an image acquisition system warrants further investigation.    Together, the results in this thesis identified that the image acquisition system is feasible, technical factors that affected image acquisition and spectral unmixing, and that optimizing some of these technical factors could improve efficiency of the current system. For image acquisition, these include computer memory and file size, image quality, and efficiency of spectral unmixing. Potential solutions for these systems have been speculated for some of these factors, including 56  testing image compression and automated methods to detect image quality. We tested six illumination wavelengths produce similar image quality as the sixteen used in this study, which enabled more efficient spectral unmixing. These results rationalize the use of these six wavelengths and potentially in the development of a multispectral image acquisition system with these selected wavelengths.      3.2 Segmentation  Segmentation is a digital image processing method used to extract objects of interest in images in order to simplify the image for analysis [131]. For this study, the objects of interest are cells in the tumor microenvironment. Thus, segmentation of cell nuclei (stained with hematoxylin) in hyperspectral images of lung tumors was performed to delineate cells. Briefly, the DUnit segmentation software uses Voronoi tessellation, a type of geometric partitioning that involves determination of neighboring cells of a given identified cell [117]. The accuracy of segmentation would contribute to the interpretation of results. If segmentation were inaccurate, this could translate to undersegmentation – loss of area containing cells missed in analysis or alternatively – or oversegmentation – leading to false positive results by analyzing background areas containing no cells [132-134]. Multiple software users could improve the efficiency of image analysis, as the concept of incorporating a multi-user platform to enable simultaneous collaboration of image datasets, such as Cytomine, so that multiuser segmentation could be concurrently done [135, 136].  To increase efficiency of manual segmentation, three individuals were used to segment images. In assessing the workflow of image segmentation of this hyperspectral image analysis system, it was observed that, while the majority of images could be segmented, the quality of  57   Figure 3.4    Segmentation of images with low and high density cell nuclei. (A) Low nuclei density sample image (B) High nuclei density sample image. Grey areas in image indicate hematoxylin stain. Blue circle indicates high density area. Red indicates object recognition of nuclei. In Voronoi tessellation, red outline indicates edge detection of cellular membrane.  segmentation varied. Approximately 95.7% of the images were segmented with DUnit (201 images segmented/total of 210 images) indicating that the software could segment the majority of images. We observed that within segmented images, segmentation of nuclei present in images was variable across all images. It was observed cell nuclei density affected the efficacy of image segmentation. Images with high density of cell nuclei in clusters were unable to be segmented, while images that had lower densities of cell nuclei were able to be segmented (Figure 3.4). Areas with high density of cell nuclei in tumors include immune cell clusters, such as the presence of tertiary lymphoid structures in lung tumors, highly organized ectopic lymph node 58  structures containing large abundance of tumor infiltrating T and B cells [137]. The size of these clusters may also vary depending on the axis the tumor specimen was cross sectioned in consideration of the three dimensional spatial configurations of bulk tumor. A number of investigators have used various approaches to overcome the challenge of segmenting overlapping cells, including deep learning approaches and various algorithms using approximation, image preprocessing, and including a priori information regarding nuclear morphology [123, 138-147].  Improved segmentation was observed by selecting one of these areas for images with high density of cell nuclei (Figure 3.5). Thus, an immediate potential solution to this segmentation problem could be to frame a biological research question requiring to interpret immune cell clusters or segment outside of the clusters. This would enable separate segmentation of areas with high or low nuclei density. Additional solutions remain to be determined, including image preprocessing steps implementing image sharpening to enhance region and edge detection of nuclei or image smoothing to denoise images for segmentation. Other systems, including commercial systems use pattern recognition to increase performance [123]. Various methods have been used for segmentation, that have been highlighted in Zarei et al. 2016 [148]. Notably, accuracy of cell segmentation in images remains a challenge for current commercial software. Segmentation may be limited by partial cells exposed on tissue from sectioning or multinucleated cells, as nuclear segmentation algorithms interpret nuclei as one cell [41]. Other studies have tried building algorithms to better partition ‘immune cell conglomerates’ [149]. An automated method with pattern recognition to detect immune clusters would be useful to inform users if the area may be difficult to segment [150].  59   Figure 3.5    Segmentation inside and outside of clusters of dense cell nuclei. Seed detection and Voronoi tessellation of image for nuclei object recognition. Red indicates object recognition of nuclei. In Voronoi tessellation, red outline indicates edge detection of cellular membrane Seed detection 60  Collectively, these results indicate segmentation with the BCCRC hyperspectral image analysis system is possible for the majority of images. However, the quality of segmentation is primarily affected by the nuclei density. These results suggest that additional optimization especially for images containing large cell clusters are to be investigated in the BCCRC  hyperspectral image analysis system. In summary, at present, study design, such as whether a biological research question is to be focused on highly dense immune cell clusters or less dense areas that may influence criteria for ROI selection during image acquisition stage and segmentation, would be a feasible solution to account for technical factors effecting segmentation efficacy. Future studies may investigate how to better segment images with both high and low nuclei density areas.    3.3 Classification  Thresholds or brightness constants can be used to distinguish object from background,  and thus, can dichotomize whether an object is negative or positive for a particular stain [148, 151]. Thresholds can be established by classifying grey level or intensity of pixels in an image, which is one of the simplest segmentation methods to separate objects from background [148, 151]. Establishing thresholds by this method is possible as different objects can display different constant surface reflectivity or light absorption [151]. These differences in turn can then be isolated from images as objects.  We considered that inter-observer variability may be present in establishing thresholds. Consequently, thresholds were determined by three observers for each index of ROI (Table 3.5). Thresholds could range from 0 (black, high threshold) to 255 (white, low threshold) at 8 bits per 61   Table 3.5   Summary of thresholds for T&B Cell panel.        Case Index 1 (grey level intensities) Index 2 (grey level intensities) Index 3 (grey level intensities) Average of 3 indexes per case (grey level intensities) SD of 3 indexes per case (grey level intensities) II2 (Red) CD79a II3 (Blue) CD3 II4 (Brown) CD8 II2 (Red) CD79a II3 (Blue) CD3 II4 (Brown) CD8 II2 (Red) CD79a II3 (Blue) CD3 II4 (Brown) CD8 II2 (Red)  CD79a II3  (Blue)  CD3 II4 (Brown) CD8 II2 (Red) CD79a II3  (Blue)  CD3 II4  (Brown)  CD8 05L14 229 229 228 218 225 210 210 220 195 219 224.6667 211 9.539392 4.50925 16.52271 05L49 232 230 226.7 233 221 219 230 210 195 231.6667 220.3333 213.5667 1.527525 10.01665 16.5337 05L46 242.6 239 229.9 225 223 215 225 220 200 230.8667 227.3333 214.9667 10.16136 10.21437 14.95003 05L54 225 220 216.3 222 205 210 225 225 195 224 216.6667 207.1 1.732051 10.40833 10.94212 06L42 242 218.6 225.5 235 225 225 235 225 230 237.3333 222.8667 226.8333 4.041452 3.695042 2.753785 06L57 238.6 237.9 235 230 220 215 225 225 195 231.2 227.6333 215 6.878953 9.23598 20 06L71 237.5 230 229 230 220 211 215 200 210 227.5 216.6667 216.6667 11.45644 15.27525 10.69268 06L74 240 233 234.8 232 225 205 225 200 195 232.3333 219.3333 211.6 7.505553 17.21434 20.70459 06L10 242 238 220 230 210 210 200 190 210 224 212.6667 213.3333 21.63331 24.11086 5.773503 06L14 239.2 240 237 217 225 215 230 210 200 228.7333 225 217.3333 11.15407 15 18.61003 06L19 N/A N/A N/A 210 225 205 200 220 195 205 222.5 200 7.071068 3.535534 7.071068 005L4 237.5 228.5 224.9 230 225 225 245 238 230 237.5 230.5 226.6333 7.5 6.726812 2.916048 06L3 230 229.1 220.8 230 225 225 235 235 230 231.6667 229.7 225.2667 2.886751 5.026927 4.605793 06L29 238.6 236.5 230 218 225 210 210 220 175 222.2 227.1667 205 14.75534 8.460693 27.83882 06L30 235.7 224 227.3 230 225 210 225 225 195 230.2333 224.6667 210.7667 5.353815 0.57735 16.16364 05L9 237.2 227.9 224.3 230 225 228 215 225 200 227.4 225.9667 217.4333 11.32608 1.674316 15.21063 05L12 237.2 227.9 232 234 215 222 215 225 200 228.7333 222.6333 218 12.00056 6.767816 16.37071 05L17 236 235.1 230.6 225 227 220 215 215 215 225.3333 225.7 221.8667 10.50397 10.11286 7.96576 05L27 237 225 221.4 225 215 220 220 215 220 227.3333 218.3333 220.4667 8.736895 5.773503 0.80829 05L39 240 230 230.9 225 229 217 220 215 195 228.3333 224.6667 214.3 10.40833 8.386497 18.10166 06L50 N/A N/A N/A N/A N/A N/A 200 200 190 200 200 190 #DIV/0! #DIV/0! #DIV/0! Average (grey level intensities) 237 231 228 226 222 216 220 217 203 226 222 214 8.81 8.84 12.7 Average SD (grey level intensities) 4.67 6.16 5.48 6.58 6.09 6.94 12.0 12.0 14.5 N/A:  no data due to no segmentation; #DIV/0!: unable to calculate 62  pixel (bpp) for grey level intensity for monochromatic images of each spectral component of images processed by spectral unmixing. Calculation of threshold, as measured by average integrated intensity (II), was dependent on whether segmentation was possible, which was the case for the majority of images. However, a few images were not able to be segmented well and thresholds were not able to be established for these images (indicated by N/A in Table 3.5). These images without thresholds were not included in the calculations of average threshold. These observations highlight the importance of upstream processes effecting downstream procedures in the workflow of this image analysis system.   As predicted, inter-observer variability for immunohistochemical stain thresholds were observed. Average of averages for T&B Cells is red, blue, and brown threshold are 226 (SD = 8.81), 222 (SD = 8.84), and 214 (SD = 12.7) grey level intensity respectively (Table 3.5). The intensities for a positive stain are similar in range when analyzed with monochromatic spectrally unmixed images. The SD is greater than zero, indicating that there is some variability involved in visually assessing stain levels. Thresholds recorded by two observers (Dr. Calum MacAulay and I) were used to calculate the threshold for the AR panel. Averages were calculated independently of cases. Average of averages for AR in red, blue, and brown thresholds were 231 (SD = 6.20), 212 (SD = 15.0), and 221 (SD = 9.88) grey level intensities (Table 3.6). These results were similar to using three observers and averaging individual cases. In summary, these results demonstrate that inter- observer variability is present in the establishment of threshold, suggesting that there is variation in the subjective interpretation of positivity and also biologically within images. Currently, this study has used a manual approach to determine thresholds for stain positivity. It would be interesting to fine tune thresholding with deep learning approaches as investigated by other studies to minimize this interobserver variability [152, 153].  63   Table 3.6   Summary of thresholds for adaptive immune resistance panel.   Index II2 (Red) CD8 (grey level intensities) II3 (Blue) PD1 (grey level intensities) II4 (Brown) PDL1 (grey level intensities) Index 3 225 195 220 225 195 225 225 195 225 230 195 235 230 195 235 230 195 230 230 195 225 240 215 235 235 210 233 235 210 225 235 210 230 235 210 230 230 205 220 230 205 235 230 210 230 230 210 220 240 210 230 230 210 230 230 200 225 230 200 230 230 200 220 Index 1 232 216 223 219 189.5 204 235 231.2 200 235 196 218 230 200 210 230 228.7 208 237.3 203 220 235 234.4 222.5 235.6 244 215 222 224 208 234.6 235 220 234.8 231 217 234 220 225 209.45 224 220.7 242 225 225.8 226.4 237.9 209.65 230 218.6 204.2 241.1 230.6 200 241.7 230 229 230 190 210 Average thresholds (grey level intensities) 231 212 221 Average SD (grey level intensities) 6.20 15.0 9.88  64  Another potential explanation of inter-observer variability is biological variation in staining patterns. We noticed that staining of antibodies within images was observed to be heterogeneous, in other words, some cell types had greater variability in stain intensities than others. It was observed that PDL1 had heterogenous stain patterns with more variable grey level intensities, while others were more homogenous, such as CD8 (Figure 3.6). A potential reason for this variability is the different expression levels of the protein in different cell types. For instance, PDL1 is known to be highly expressed in macrophages and moderately expressed in tumors [154]. A potential solution is to establish and apply additional positivity thresholds for IHC marker; as currently, the development of companion diagnostic tests of PDL1 implement expression threshold levels of this protein to stratify the administration of PD1 drugs. In this study, the PDL1 antibody clone used was SP142. This is the same clone used in the FDA-     Figure 3.6    Homogeneity and heterogeneity of antibody stains. Hematoxylin (Filter 0) stained in image. CD8 in image homogenous stain. PDL1 in image stained with heterogenous stain.   65  approved complementary diagnostic test for the stratification of the FDA approved immune checkpoint inhibitor atezolizumab, and is known to stain immune cells more than tumor cells [154-157]. This approach of a scoring method has also been applied in commercial systems [124, 158]. Using this approach, we established and implemented multiple PDL1 thresholds to account for the variation of the expression of PDL1 within tissues and to be able to measure these differences (Table 3.7). To test the addition of higher threshold, two observers (Dr. Katey Enfield and I) established thresholds for high intensity values of PDL1 thresholds for two different indexes of ROI by reoptimizing thresholds on ‘test.tree’ on DUnit (Table 3.7, Figure 3.7). This new higher   Table 3.7   Summary of thresholds for adaptive immune resistance panel for PDL1high.   Case Index 2 II4  (Brown)  PDL1 (grey level intensities) Index 3 II4  (Brown)  PDL1 (grey level intensities) Average of Index 2&3 (grey level intensities) SD (grey level intensities) 05L14 200 192 196 5.656854 05L49 180 201 190.5 14.84924 05L46 190 178 184 8.485281 05L54 180 182 181 1.414214 06L42 190 185 187.5 3.535534 06L57 180 200 190 14.14214 06L74 180 210 195 21.2132 06L71 180 170 175 7.071068 06L10 200 216 208 11.31371 06L14 200 187 193.5 9.192388 06L19 190 179 184.5 7.778175 06L3 160 190 175 21.2132 06L29 200 199 199.5 0.707107 06L30 180 185 182.5 3.535534 05L9 185 N/A 185 #DIV/0! 05L12 200 180 190 14.14214 05L17 200 170 185 21.2132 05L27 180 194 187 9.899495 05L39 170 205 187.5 24.74874 06L50 170 223 196.5 37.47666 05L4 200 190 195 7.071068 Average of Averages 189 12.2 N/A: no data due to no segmentation 66                 Figure 3.7   Establishing multiple thresholds for PDL1 stain. Spectral unmixed image of PDL1 stain. Image of total segmentation of Voronoi tessellation (red). PDL1+ threshold established by the average of three observers (blue). PDL1++ threshold established by the average of three observers (orange).  67  threshold that was established was referred to as PDL1++. The average threshold of PDL1++ had an II value 15% less than the initial PDL1+ II, creating a higher threshold (Table 3.7). Again the threshold observed had a SD greater than zero, suggesting the presence of inter-observer variability and heterogeneity of stain distribution present in sampled regions. This assessment corroborates that inter-observer variability in establishing thresholds is present, though some of the variability may be biologically related, such as a gradient of positivity present in tissue, rather than a binary assessment.  Accordingly, we were able to construct a classification tree based on grey level thresholds of spectrally unmixed images (Figure 3.8). With this supervised approach, it is important to account for inter-observer variability when establishing these thresholds that may introduce potential biases to subsequent interpretation of images. Furthermore, application of immunohistochemical stain thresholds will depend on the variability of grey level intensities present in spectrally unmixed images or the gradient of stain that may be observed, which may be biologically relevant, such as different expression levels of proteins in different cell types.  Due to the limited subset of markers that can be interrogated using light absorbance from chroma-based systems, it would be interesting to further differentiate between different immune cell subsets, such as neutrophils or natural killer cells, with the integration of a classifier for machine learning to differentiate these cell types with pattern recognition of additional cellular features.     68      Figure 3.8    Classification trees of cell types based on threshold positivity. Figure courtesy of Dr. Katey Enfield.   69  3.4 Cell quantity and cell-to-cell spatial relations:  potential factors related to data interpretation  Selection bias refers to systematic error in selecting samples that do not represent the population of interest, where false significant relationships may be identified (false positive or type I error) and true significant relationships potentially missed (false negative or type II error) [159-161]. Thus, selection bias is problematic in limiting the generalizability or external validity of a study, and is of particular concern for limiting clinical error [162].  In routine clinical practice, selection bias of tissue sections for histopathology remains a problematic issue, as typically only one tissue section is selected for assessment. Estimates will be biased if every structure in tissue has an unequal probability of being assessed [161, 163]. A solution for unbiased sampling relies on equal probability of all structures in a three dimensional tissue being assessed [161, 163]. In the case of digital pathology clinical workflow optimization and in this study, limited memory in servers is an expected constraint. Thus, analysis of entire WSI was not performed, and ROI sampling (virtual biopsy) was conducted as a sensible and practical means to conserve image file size and computer memory.  It is important to distinguish the difference between precision and bias. Precision determines how reproducible an estimate is and is related to the rigor of sampling [161]. The more precise an estimate is, the lower the variability will be present for a particular estimate  [161]. Bias refers to how accurate an estimate is to the true population mean and is difficult to determine, and unbiased data can be either precise or imprecise [161]. The presence of selection bias of imaged areas would contribute to misinterpretation of cell-to-cell spatial organizations measurements and clinically, for potential false predictions of cancer prognosis and pathological 70  features of tumors. Thus, evaluation of ROI sampling strategy in this study for image acquisition and subsequent downstream digital image processing, data analysis, and interpretation is essential for potential future clinical application of this image analysis system. Due to the small number of biological replicates in this cohort in this proof-of-concept study, we were neither able to determine accurate population mean to assess selection bias nor generate any biologically relevant conclusions from these results. However, the following section of this thesis demonstrates the feasibility of Asterics for cell-to-cell spatial analysis of the tumor microenvironment and evaluates if the ROI sampling strategy and related effects deriving these metrics associate with data precision and effect data interpretation.   3.4.1 Parameters measured by Asterics: Cell quantity and cell-to-cell spatial relations  Asterics is the cell-to-cell spatial analysis software of the BCCRC hyperspectral image analysis system developed by Dr. Calum MacAulay and Dr. Martial Guillaud’s research team. Asterics performs analysis of various tissue architecture parameters, including the following three metrics to quantify cells and their spatial contexts: percentage of positive cells (Percentage), mean neighborhood group (MNG), and variance of neighborhood group (VAR). Definitions, significance, and specific examples of these parameters assessed by Asterics are summarized in Table 3.8, and will be further described in detail in this section.           71  Table 3.8   Parameters measured by Asterics used to assess selection bias analysis in this study.   Parameter Definition Significance Example Percentage of positive cells (Percentage) Proportion of cells with positive stain relative to all segmentable cells in image Measures quantity of a given cell population  A conventional metric used by other commercial system  Collects baseline information of cell abundance to inform tumor microenvironment spatial model related to MNG Proportion of PD1+ cells relative to all detected cells in image Mean neighborhood group (MNG) Average proportion of cell population of interest next to a given cell population across an entire image Profiles cell sociology or ‘tissue level organization’  Emerging, novel metric to quantify spatial organization in tissue  Average proportion of PD1+ cells next to PDL1+ cells across image Variance of neighborhood group (VAR) Variance of proportion of cell population of interest next to a given cell population (MNG) across an entire image Assesses whether the pattern of spatial relationships (MNG) is consistently present or not in an image   Novel metric to quantify spatial organization in tissue Variance of proportion of PD1+ cells next to PDL1+ cells across image   Percentage of positive cells, herein referred to as just ‘percentage (%)’, measures the proportion of cells positive for a stain relative to all segmented cells that could be recognized by the segmentation software DUnit (Table 3.8). This parameter provides a global proportion of the number of a particular cell population is present, and is a metric more classically used in the literature for quantifying the abundance of a cell population of interest [82, 164, 165]. Such metrics are conventionally measured by commercial systems (refer to Table 1.1).  For Asterics, percentage would provide baseline information on the relative presence or absence of a particular cell type in relation to more complex spatial modelling of the tumor microenvironment that is capable by this software. For instance, few percentage of cells identified from this metric could inform that few initial cells are available for spatial modelling; highlighting that this basic information is essential for interpretation of higher dimensional analyses if performed.  MNG is a unique parameter measured by the BCCRC hyperspectral image analysis software, Asterics (Table 3.8). MNG refers to the average proportion of cell population of interest ‘next to’ or ‘neighboring’ (symbolized as forward slash (/) in this Chapter of this thesis) a given cell population (referred to as ‘neighborhood group’, i.e. CD3+ cell next to CD79a+ cell) 72  across the entire image (Figure 3.9). For instance, a high MNG (MNG = 1) would refer to a high frequency that a reference cell type is next to a given cell type, while a low MNG (MNG = 0) would be indicative of a low frequency. As colloquially understood, a neighborhood, such as within a city, is a physical area implying both geographic and social constructs [166]. An extension of this concept can be applied to understand single cells in the tumor microenvironment, which is uniquely performed by Asterics. MNG enables an even higher resolution, local description of cell geography, specifically the configuration of the immediate  local spatial domain or micro-habitat of a single tumor microenvironment cell, whether one classified cell type is more or less likely to be in the vicinity – appears to physically touch – another given classified cell type. Relative to percentage, the concept of cell-to-cell spatial relation analysis is an emerging metric with few developments currently being made. MNG is  especially valuable by profiling tissue level organization in a quantitative manner, contributing to     Figure 3.9    Mean neighborhood group (MNG) by Asterics. A frequency of neighboring cells from reference cell is calculated for all cells in images. The mean of these frequencies reference to neighboring cell pairs is calculated for all cells in images. Pink indicates adjacent, neighboring cells. Red seed point indicates cell classified as group 1 (i.e. cell with no stain). Blue seed point indicates cell classified as group 2 (i.e. PDL1+ cell). Green seed point indicates cell classified as group 3 (i.e PD1+ cell). Yellow seed point indicates cell classified as group 4 (i.e. CD8+ cell). Turquoise indicates DUnit edge detection of cell membrane. Figure courtesy of Dr. Calum MacAulay. 73  developing a method for mapping the spatial ecology of cells in tumors. Frequent geographic proximity of specific cell types of interest in images implies potential cell-to-cell interactions or cell sociology. However, such physical interactions are inferred and thus, correlative. Therefore, MNG derived from this software is a valuable tool to posit hypotheses regarding novel cell-to-cell interactions in the tumor microenvironment for further validation. Furthermore with this approach, MNG could then be correlated with treatment or survival outcomes as a CAD tool, contributing to more advanced spatial analyses of tissue for prognostic and theranostic predictions.  In conjunction with MNG, VAR indicates the variance of MNG – how variable the proportion of cell population of interest is next to a given cell population is across the entire image (Table 3.8). Classically, variance assumes a non-negative value, where the minimum value is zero, indicative that no variation is present. For instance, across a given ROI image, a reference cell may be consistently next to two cells of a given population in different sites of the ROI image, or observed to be more variable, next to four different cells, or sometimes one different cell of a given population strewn across different sites within an ROI. Thus, VAR measures the spread or dispersion of neighborhood group within an ROI – a high variance (VAR > 0) would indicate that there is large variation of a reference cell type next to a given cell type in the whole image, and a small variance (VAR = 0) would be indicative that little variation is present between a reference cell type and a given cell type in the whole image. In the case that not enough cells are present to accurately analyze variance, Asterics value can output a special value (-999) to alert the user (raw data plots with inclusion of these large negative values are in Appendix A of this thesis). The following graphs in this thesis have this large value excluded in plots. In conclusion, VAR would additionally inform whether the cell sociology pattern of 74  specific neighborhood groups observed tends to be relatively constant or is more sporadic in a given snapshot of the tumor microenvironment.  Thus far, HALO is one of the only known commercial, non-clinical image analysis system with a similar cell-to-cell spatial metric as Asterics. However, the clinical value of its proprietary algorithm for prognostic or theranostic predictions remains to be tested. There remains a paucity of published studies with the HALO nearest neighborhood analysis metric currently available. The different neighborhood analysis algorithms by Asterics could present as an alternative method with regards to consilience (the concept that convergence of evidence derived from different methods when synthesized, generates strong conclusions). Moreover, technology transfer and access to deluxe commercial quantitative image analysis software remains a challenge from an economical perspective, including for developing countries. The image analysis system described in this thesis may provide a potential alternative, cost-effective approach if efficacy is demonstrated in the workflow of this system.  In this section of this thesis, the following six analyses were conducted on Asterics with the processed image data. The table that follows, Table 3.9, outlines and describes the types of analyses performed to assess ROI sampling strategy in this study, and potential challenges that arose during data interpretation. The analyses presented here are organized from least to most complex scenarios when quantitatively modeling the tumor microenvironment. The graphs presented in this thesis have y-axes scaled to the same dimensions to facilitate data comparisons. To better represent observations made from some analyses, the same graphs also had their axes autoscaled to have a larger domain to view all coordinates, which are presented in the Appendices (B.1 to B.6) at the end of this thesis. A graphical key to facilitate data interpretation of all graphs in this Sub-section are presented in Figure 3.10.75  Table 3.9   Summary and significance of analyses performed with Asterics.   Sub-section # Cell population Examples from panels Parameters  assessed Asterics metric Significance Neighbor type  (# of cell subsets) Population properties  (# of markers) T&B cell  Adaptive immune resistance  3.4.2 Same neighbor (1) General (1) CD3+ CD79a+ CD8+ CD8+ PD1+ PDL1+ PDL1++ Cell quantity Percentage Assesses baseline level of general subsets in ROI  Least complex scenario with the expected largest available pool of cells (no spatial modeling) to assess:   data precision of individual tumor sample (intratumoral)  extent of variability across cohort (intertumoral) 3.4.3 Same neighbor (1) General A (1) / General A (1) CD3+ / CD3+ CD79a+ / CD79a+ CD8+ / CD8+ CD8+ / CD8+ PD1+ / PD1+ PDL1+ / PDL1+ PDL1++ / PDL1++ Cell-to-cell spatial relations MNG  VAR More complex scenario with the expected largest available pool of cells (with spatial modeling) to assess:   data precision of individual tumor sample (intratumoral)  extent of variability across cohort (intertumoral) 3.4.4 Same neighbor (1) Specific (2) CD3+CD8+ CD8+PD1+ Cell quantity Percentage Assesses if Asterics may be able to detect more specific cell subsets (preliminarily assess sensitivity)  Least complex scenario with the expected smaller available pool of cells (no spatial modeling) to assess:   data precision of individual tumor sample (intratumoral)  extent of variability across cohort (intertumoral) Specific (3) CD3+CD8+CD79a+ CD8+PD1+PDL1+ CD8+PD1+PDL1++ Cell quantity  Percentage Negative control: Determine if non-conventional cell types are detected in images 3.4.5 Same neighbor (1) Specific A (2) / Specific A (2) CD3+CD8+ / CD3+CD8+ CD8+PD1+ / CD8+PD1+ Cell-to-cell spatial relations MNG  VAR More complex scenario with the expected smaller available pool of cells (with spatial modeling) to assess:   data precision of individual tumor sample (intratumoral)   extent of variability across cohort (intertumoral) Specific A (3) / Specific A (3) CD3+CD8+CD79a+ / CD3+CD8+CD79a+ CD8+PD1+PDL1+ / CD8+PD1+PDL1+ CD8+PD1+PDL1++ / CD8+PD1+PDL1++ Cell-to-cell spatial relations MNG  VAR Negative control: Determine if non-conventional cell types are detected in images using spatial relations analysis 3.4.6 Different neighbor  (2) General A (1) / General B (1) CD3+ / CD79a+ PD1+ / PDL1+ PD1+ / PDL1++ Cell-to-cell spatial relations MNG  VAR Most complex scenario (with spatial modeling) with expected larger available pool of cells to assess:   data precision of individual tumor sample (intratumoral)  extent of variability across cohort (intertumoral) General B (1) / General A (1) CD79a+ / CD3+ PDL1+ / PD1+ PDL1++ / PD1+ Cell-to-cell spatial relations MNG  VAR Assess if cell-to-cell spatial analysis is dependent on reference cell matters, in other words if Asterics can potentially detect variation in microhabitats of cells 3.4.7 Different neighbor  (2) General (1) /  Specific (2) CD3+CD8+ / CD79a+ CD8+PD1+ / PDL1+ CD8+PD1+ / PDL1++ Cell-to-cell spatial relations MNG  VAR Most complex scenario (with spatial modeling) with expected least available pool of cells to assess:   potential sensitivity relative to general / general neighborhood cell-to-cell spatial correlations   data precision of individual tumor sample (intratumoral)  extent of variability across cohort (intertumoral)  Assess if cell-to-cell spatial relations is dependent on reference cell, in other words can potentially detect changes in cell microhabitats  Specific (2) /  General (1) CD79a+ / CD3+CD8+ PDL1+ / CD8+PD1+ PDL1++ / CD8+PD1+ Cell-to-cell spatial relations MNG  VAR  Dashed line = divides analysis within same sub-section; General = refers to cells defined by a single marker; Heterogenous = refers to two different cell populations or neighborhood groups being assessed; Homogenous = refers to two same cell populations or neighborhood groups being assessed; MNG = mean neighborhood group; Slash (/) = ‘next to’; Specific = refers to cells defined by multiple markers; VAR = variance of MNG  76                                                                                                                                                                                                                                             ≃                                                                                                                                                                                                     ≃                                                     Figure 3.10    Key for graph interpretations. (A) Components of graph representation of tumor. Texture in ROI was performed with texture image provided in Microsoft PowerPoint. (B) Graph representation of tumor (Case). Red box indicates pathologist annotation of area selection. Black circles indicate ROI containing high immune cell infiltration (blue blob). (C) Illustration of high and low precision. High precision indicated by small magnitude of SD and tight values of ROI (points). Low precision is indicated by high dispersion of values derived from ROI (points) and large magnitude of SD. (D) Red line indicates line of best fit of mean of all ROI. Blue bands indicate confidence interval (CI) of mean of ROI. A large spread of CI indicates that there is large variation across different tumors (Cases); whereas a small band indicates small variation across different tumors. (E) Formula of line of best fit of mean of all ROI. Vel ( ) in subscript refers to ‘OR’. Since (   slope of the line in these graphs approximates zero, therefore ( ), the line of best fit approximates the value of the y-intercept. A B C D E Five ROI 77  3.4.2 Quantity of general cell types   Effects of sampling strategy were first assessed in a single cell type that was part of a general cell class (as defined by a single molecular marker). Heuristically, analysis with only one general cell subset is expected to have greater confidence, as a larger pool of cells would be sampled (larger cell percentage or alternatively, larger number of expected cells to be positive. Thus, using this more conventional quantitative cell metric, the following analyses was conducted as a toy problem prior to more complex conditions, such as with more specific cell subsets (where fewer cells would be anticipated for assessment) or masked by additional layers of complexities related to spatial modeling of cells.  To determine the precision of ROI selection strategy for image acquisition for this hyperspectral image acquisition system, percentages of classified general cell types for five ROI (if available from adequate segmentation) from Section 3.3 were plotted along with mean and SD among the five ROI. Precision was assessed within individual tumors (technical replicates or ROI) in terms of how close points were in relation to one another. Variability across different tumors (biological replicates) among the 20 lung AC in this study was observed by the line of best fit across the mean of ROI (                       ≃                       ; Refer to Figure 3.10E for deriving this approximation) and spread of 95% CI bands plotted by Statistica (Refer to Figure 3.10D for graphical illustration). Data precision was first assessed within tumors (intratumoral) and across tumors (intertumoral) in cell types next to the same cell type, where a similar neighborhood group referred to as ‘same neighbor’ is present. Single markers used to identify general cell subsets that were used in this analysis include CD3+, CD8+, and CD79a+ in 78  the T&B Cell Panel, and CD8+, PD1+, PDL1+, and PDL1++ in the ARPDL1high panel. The following general observations were made: Sufficiency of sampling for cell quantity analysis: We observed that precision of sampling was variable among the cohort assessed. High precision when measuring cell percentages of general cell types with same neighbors was observed within several tumors for each stain. This was observed as close overlap of points within SD bars that were of a lower value (illustrated ‘low precision’ in Figure 3.10C) as observed in the T&B Cell panel (for example, Percentage(CD3+): case 06L3, 06L30; Percentage(CD79a+): case 05L14, 05L4, and 05L9; Percentage(CD8+TB): majority of cases; Figure 3.11 and for view of points refer to Appendix B, first column of B.1). These results suggest that for some tumors, selection of five ROI to assess cell percentages may be sufficient to produce precise data as observed by low variability of cell percentage.  High precision for several cases may potentially suggest that five ROI may be oversufficient for data precision within these cases. Low precision of this sampling strategy was also observed within few tumors for each stain, as indicated by dispersion of ROI data points (illustrated ‘high precision’ in Figure 3.10C), some of which are outside high SD whiskers, (examples include Percentage(CD3+): case 06L57, Percentage(CD79a+): case 05L17, Percentage(CD8+TB): case 05L54, first column of Figure 3.11 and for view of points refer to Appendix B, first column of B.1). In summary, the variability of precision in percentage values per case suggests that the ROI sampling strategy cannot be generalized for precise data values. Biologically, some tumors may have more variability in areas with high immune cell densities, while others may be more consistent with this ROI sampling strategy, highlighting the relevance of tumor heterogeneity. Presently, the accuracy of these results is not known as a small cohort 79  was analyzed. Determination of the accuracy of these results is required to verify if the sampling strategy is sufficient for future studies.  Reproducibility of sampling methods for cell quantity analysis: To assess if the precision observations of percentage in Asterics are generalizable, percentage of ARPDL1 high was assessed. Similar results were observed in the AR panel, where close overlap of points within SD (illustrated in Figure 3.10C) suggesting high precision were observed (Percentage(CD8+AR): 05L46, 06L14, 06L30; Percentage(PD1+): the majority of cases; Percentage(PDL1+): 05L46; Percentage(PDL1++): the majority of cases; first column of Figure 3.12 and for view of points refer to Appendix B, first column of B.2) and high scatter of points with high SD (illustrated in Figure 3.10C) were also observed suggesting the method of ROI selection exhibits low precision for some tumors (Percentage(CD8+AR): case 06L57; Percentage(PD1+): case 06L42, Percentage(PDL1+): majority of cases, Percentage(PDL1++): case 05L4, 05L54, 06L3, 06L30, 06L50; first column of Figure 3.12, and for raw data plot refer to Appendix A, first column of A.1, and view of points refer to Appendix B, first column of B.2). These results in a different panel suggest the reproducibility of this effect – that not all cases had high precision of cell quantity with five ROI. This effect is most prominently demonstrated in PDL1+ percentages, where intertumoral variation is present in the majority of cases, exhibiting high data scatter, and intratumorally, where high 95% CI bands are observed across the cohort (first column of Figure 3.12).When assessing precision in both panels, the 95% CI bands of several stains or markers are close to one another, indicating that different tumors exhibit similar baseline levels of cell composition from one another (95% CIPercentage(CD8+TB), first column of Figure 3.11; 95% CIPercentage(PD1+), CIPercentage(PDL1++), first column of Figure 3.12), while other stains or markers exhibit large spread of 95% CI bands (95% CIPercentage(CD3+), 95% CIPercentage(CD79a+), first column 80  of Figure 3.11; 95% CIPercentage(CD8+AR), 95% CIPercentage(PDL1+), 95% CIPercentage(PDL1++) first column of Figure 3.12), suggesting a larger range of immune cells may be present, though the accuracy of these measures needs to be verified in a larger population.  Interestingly, for CD8+ percentages in the T&B Cell panel and PD1+ percentages in the AR panel, few cells are present in any of the lung AC in this cohort, suggesting that the lung AC within this cohort exhibit similarly low levels of these immune cell populations as measured by Asterics (where bpercentage(CD8+TB) = 1.7119%, first column of Figure 3.11; bpercentage(PD1+) = 1.1279%, first column of Figure 3.12). However, CD8+ percentages in the AR panel exhibit more spread in the upper and lower 95% CI (blue dotted band) and have a higher percentage compared to CD8+ in the T&B Cell panel (bpercentage(CD8+TB) = 1.7119% first column of Figure 3.11;  bpercentage(CD8+AR) = 9.7947%, first column of Figure 3.12). Speculating on these results, interpretation of Asterics may also be dependent on biological variation, such as potentially the section of tumor selected for analysis or technical variation, including the type of stain used for protein detection. Potentially, section selection (i.e. number of sections assessed) and stain type selection may be important factors to consider in study design using this method and further investigation is warranted to validate their effects in Asterics data interpretation.     Threshold selection for cell quantity analysis: Threshold selection appeared to effect the interpretation of Asterics percentage results. The addition of a more stringent threshold with PDL1++, exhibits different precision of the same ROI compared to a less stringent threshold with PDL1+, in some tumors. For instance, intertumorally, PDL1+  has approximately four fold greater percentage of cells present compared to PDL1++ (bpercentage(PDL1+) = 8.0289% versus bpercentage(PDL1++) = 1.8786%, first column of Figure 3.12). Furthermore, PDL1+ has a larger spread of 95% CI bands of average mean of ROI compared to PDL1++ (first column of Figure 81  3.12). These results suggest that the stringency of threshold may bias interpretation of results for this image analysis system, including the precision of data collection. From this pilot study, future studies will need to consider the biases of threshold selection in study design if a manual approach as described in this thesis is used.   3.4.3 Cell-to-cell spatial relations of same neighbor general cell types   Since precision was evaluated by assessing a less complex context with general cell types of the same kind and baseline assessment of cell abundance was performed using a more conventional method, the sampled image dataset was then assessed using a more complex scenario in regards to cell-to-cell spatial relations. This was performed by conducting a similar analysis to the percentage analysis in the previous analyses described in Sub-Section 3.4.2, but the percentage metric was replaced with the novel spatial cell metric measured by Asterics, MNG and VAR. The same cell types were assessed as in the previous analyses in Sub-Section 3.4.2, where general cell types were assessed that had the same kind of neighbors.  To determine the precision of ROI selection strategy for studying spatial relations, MNG and VAR of classified general cell types for five ROI (if available from adequate segmentation) from Section 3.3 were plotted along with mean of MNG or VAR and SD of MNG or VAR. Precision was assessed within individual tumors (technical replicates or ROI) in terms of how close MNG or VAR values were (Refer to Figure 3.10E for illustrated schematic of high and low precision). Variability across different tumors (biological replicates) among the 20 lung AC cohort in this study was also evaluated by observing the line of best fit across the mean of MNG or VAR plotted with Statistica (                     ≃                      ; Refer to Figure 3.10E 82  for deriving this approximation) and spread of 95% CI (Refer to Figure 3.10D for graphical illustration). For this spatial relations analysis, single markers aforementioned in previous analysis of cell quantities in Sub-section 3.4.2 were used: CD3+, CD8+, and CD79a+ in the T&B Cell Panel and CD8+, PD1+, PDL1+, and PDL1++ in the ARPDL1high panel. Unlike the previous analysis, context was analyzed, where data precision of cell types next to the same cell type was assessed (same neighbors). For instance, data precision of a CD3+ cell next to another CD3+ cell was evaluated. The following general observations were made, which were similar to the previous analysis performed on cell percentages: Sufficiency of sampling for cell-to-cell spatial relations analysis: Similar observations were made as with the conventional quantitative measurement, cell percentage, using this cell-to-cell spatial relations metrics regarding sampling and data precision. Precision of spatial relations was first assessed in the T&B Cell panel. High precision, as defined by close overlap of data points within SD bars that were of a lower value (illustrated in Figure 3.10E), was observed when measuring spatial relations with Asterics within several tumors for each stain (MNG(CD3+/ CD3+): majority of cases; MNG(CD79a+/ CD79a+): cases 05L14, 05L4, 05L9, 06L50; MNG(CD8+TB/ CD8+TB): case 05L39; 06L10; 06L42, second column of Figure 3.11). Low precision was also observed within several tumors for each stain, as indicated by spread of ROI data points (illustrated in Figure 3.10E), some of which are outside high SD bars (this spread is most dramatically observed in MNG(CD79a+ / CD79a+): case 06L57 and MNG(CD8+TB/ CD8+TB): case 05L54; second column of Figure 3.11). These spatial analyses results corroborate with results from the percentage analysis, that for some tumors five ROI has precise data of spatial relations across tumors, while others may require sampling greater than five ROI. For cases with high overlap, potentially fewer than five ROI could be potentially used to gain precise data of spatial relations.83                   Figure 3.11    General cell type cell quantity and spatial relations of same neighbors in T&B cell panel. X-axis indicates 20 cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains large negative variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.   Percentage                                                  MNG                                                               VAR CD3+            CD79a+             CD8+    Percentage (CD3+) Percentage (CD79a+) Percentage (CD8+) 84  Intratumorally, low precision of MNG tended to be higher in different stains. For instance, CD3+ next to CD3+ (bMNG(CD3+/ CD3+) = 0.2102 MNG) and CD8+ next to CD8+ (bMNG(CD8+TB/ CD8+TB) = 0.0782 MNG) had lower precision across all tumors compared CD79a+ next to CD79a+ (bMNG(CD79a+/ CD79a+) = 0.2905 MNG) as demonstrated by closer proximity between line of best fit and 95% CI bands (second column of Figure 3.11). These results suggest that data precision of spatial relation metrics with Asterics may vary with cell type or stain type when analyzing same neighbors. In summary, this analysis indicates that analysis of spatial relations with the current sampling strategy has variations in data precision, as previously shown with cell percentage. Whether this precision may be affected by other biological variations, and whether this data precision is accurate requires assessment of a larger population of lung tumors than the cohort used in this study.  Reproducibility of sampling method for cell-to-cell spatial relations analysis: To assess if this effect of precision of spatial relations is dependent on staining panels and is generalizable, cell-to-cell spatial relations of same neighbors in a different panel (the AR panel) were analyzed. Similar observations regarding differences in data precision was also observed in a different panel, where some stains had higher precision than others (MNG(CD8+AR/ CD8+AR): majority of cases high precision; MNG(PD1+ / PD1+): majority of cases high precision) and other stains, such as PDL1 had lower precision (MNG(PDL1+ / PDL1+): majority of cases low precision; MNG(PDL1++ / PDL1++): majority of cases low precision, second column of Figure 3.12). These results suggest that variation in data precision is present when analyzing spatial relations with Asterics.  Potentially, stain selection may again be an important factor effecting data interpretation of spatial relations as with cell percentages. It was expected that magnitude of spatial relations as measured by MNG for CD8+ in the AR and TB panel may be similar in adjacent sections, as 85  although different stain colors were used, the same antibody was used between both panels to detect the same protein (refer to Table 2.2). However, it was observed that CD8+ spatial relations were more abundant in the AR panel (bMNG(CD8+AR / CD8+AR) = 0.174 MNG; second column of Figure 3.12) than the T&B Cell panel (bMNG(CD8+TB / CD8+TB) = 0.0782 MNG; second column of Figure 3.11), which was approximately over a twofold greater magnitude of this spatial correlation and also more precise across tumors, as indicated by the closer 95% CI observed in the AR panel to the CD8+ line of best fit (second column of Figure 3.11). Additional investigations into this observation are required, including if this variation are indeed due to technical variation that may implicate that stain type may bias accuracy of spatial relation analyses with Asterics across images in addition to cell quantity.  We highlight the importance of combining different cellular metrics, such as percentage data, for interpreting results from cell-to-cell spatial analyses. For instance, very few spatial correlations were observed between PD1+ next to PD1+ cells, as exhibited by a low MNG value for all tumors in this cohort (bMNG(PD1+/ PD1+) = 0.0254 MNG, second column of Figure 3.12). Potentially, this observation may be explained by the very few initial PD1+ cells identified in the ROI as observed in percentage data (bpercentage(PD1+) = 1.1279%, first column of Figure 3.12). These results illustrate the importance of pairing different quantitative metrics, such as percentage and MNG, to provide baseline information regarding the abundance of a particular cell type in tumors when interpreting spatial relations, particularly for biological studies. A larger cohort of lung AC would be further required to determine whether this low number of same neighbor PD1+ cells represents the whole population of lung AC or if bias is present from the ROI sampling method. Collectively, these results highlight that variation is present in data 86  precision and that validation of potential technical factors that may effect this data interpretation is needed.   Variation of spatial relations within ROI for cell-to-cell spatial relations analysis: To determine if spatial relations within ROI were variable, precision of VAR was assessed. With this sampling strategy, we observed that some molecular markers and staining conditions had more variation of spatial relations within an image. In the T&B Cell panel, some stains such as VAR of CD3+ next to CD3+ exhibited similar VAR across the lung AC cohort (bVAR(CD3+/ CD3+) = 0.1944 VAR), while other stains such as VAR of CD79a+ next to CD79a+ (bVAR(CD79a+/ CD79a+) = 0.23 VAR; VAR(CD79a+/ CD79a+): including case 05L54, 06L10) and especially VAR of CD8+ next to CD8+ (bVAR(CD8+TB/ CD8+TB) = 0.1218 VAR; VAR(CD8+TB/ CD8+TB): case 05L27, 05L54, 06L71, 06L74) exhibited larger spread of variance across different ROI and 95% CI, suggesting that these cell types or stains do not consistently exhibit the same cell-to-cell spatial relations (MNG) within different parts of the ROI (third column of Figure 3.11). However, in a different panel (AR panel), CD8+ exhibited precise variance value, where the majority of cases had closer overlap in SD (refer to Figure 3.10E for illustration), (bVAR(CD8+AR/ CD8+AR) = 0.1818 VAR, third column of Figure 3.12). This variance value in the AR panel is very similar to that of the adjacent tissue in the other panel. However, variance was observed to be more consistent across tumors in this cohort, as observed from closer 95% CI with the line of best fit compared to the T&B Cell panel. As the same antibody was used in these two different panels (refer to Table 2.2), these results further suggest that potential biological and technical factors (i.e. different sections and different stains in tissues) may bias interpretation of consistency of intertumoral variation of spatial relations within images of ROI, although further investigation to validate this correlation are required.   87              Percentage                                                  MNG                                                      VAR CD8+            PD1+           PDL1+            PDL1++       * Percentage (CD8+) Percentage (PDL1++) Percentage (PDL1+) Percentage (PD1+) 88  Figure 3.12    General cell type cell quantity and spatial relations of same neighbors in adaptive resistance panel. X-axis indicates 20 cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains ROI with large negative value (VAR = -999 indicating case has not enough cells to assess variance), which was excluded from analysis.  Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with no line of best fit and 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.   Lastly, we observed that significant negative value of SD was present when measuring VAR. Three cases out of the 20 cases in the PD1+ next to PD1+ cell-to-cell spatial relations analysis exhibited negative VAR values, resulting in a SD of large magnitude contributing to why the line of best fit and 95% CI bars were not shown in the graph as they were out of bounds of the graph (VAR(PD1+ / PD1+) = -999 VAR for case 05L54  indicated by asterisks (*) in third column of Figure 3.12 and seen in full range in A.1B). These results for negative VAR indicate that for some tumors, not enough cells are present to accurately determine variance. This observation regarding negative VAR will be further discussed below in further detail in the triple positive cell analysis in Sub-section 3.4.5. Deletion of the negative VAR value enabled the line of best fit to be plotted (third column of Figure 3.12). Collectively, these results indicate that variability of spatial relations in an image is present at many different levels, suggesting that the presence of specific cells next to the same kind of each other is not a constant value, but is highly variable within and across some tumors. However as previously illustrated with percentages, a different MNG and VAR value is observed in adjacent sections, suggesting that potential technical or biological variation, such as with different stains or sections assessed for CD8+. In terms of potential technical factors, determining whether stain type biases data and if one stain is more accurate to the true population value 89  would require validation in a larger dataset. In conclusion, these results further suggest that presently some bias is present in the interpretation of spatial relations across all samples.   3.4.4 Quantity of specific cell types   Precision was then observed in more specific subsets, defined by cell types with combined positivity of more than one marker. By analyzing cell types positive for more than one marker, a smaller pool of cells sampled (lower cell percentage) was expected to be analyzed. Thus, confidence of these results is expected to be less compared to the scenario tested within the previous analyses in Sub-section 3.4.2, as fewer quantity of cells were expected to be assessed than the general cell type analysis. Specific cell types were analyzed with two cell markers and also tested with the maximum of all three cell markers in each panel in this study. Double-marker cell combinations plotted were selected due to their biological relevance and establishment as immune cell subsets in literature: CD3+CD8+ cells as cytotoxic T cells in the T&B Cell panel and CD8+PD1+ as exhausted T cells in the AR panel. Triple marker cells in this study, CD3+CD8+CD79a+ and CD8+PD1+PDL1+, are expected to be present at a very low quantity as they are not presently recognized as a known specific cell type. As in previous analyses with general cell subsets, specific cell subset analysis reported in this sub-section will be first analyzed with a conventional cell quantity metric, percentage, to acquire baseline information of cell abundance and potential technical factors related to data interpretation. Double positive cell class analysis: We expected that specific cell types nested in a general cell type to be present at lower quantities than their broader counterpart. We could not completely validate if a smaller population of cells is present for double positive cells as the 90  general cell types (i.e. CD3+ refers to CD3+CD8-CD79a-) and specific cell types (i.e. CD3+CD8+) were classified as different groups. To observe the relative cell quantities of these groups, line of best fits of the percentage of double marker cell types were compared to line of best fits of percentages of single marker cell types. Smaller quantity of cells was present with the more specific cell types compared to general cell types for one of the panels (bpercentage(CD8+AR) =  9.7947%; bpercentage(PD1+) = 1.1279%, first column of Figure 3.12; bpercentage(CD8+PD1+) = 0.9355%, first column of Figure 3.13). In the T&B Cell panel, one of the more specific cell type (CD3+CD8+) had a greater percentage than the general subset (CD8+ ) (bpercentage(CD3+) = 10.8268%, bpercentage(CD8+TB) = 1.7119%, first column of Figure 3.11; bpercentage(CD3+CD8+) = 7.9835%, first column of Figure 3.13). These results indicate that very low spatial relationship may be indicated in the AR panel and more spatial relationships may be detected in the T&B cell panel.  Since specific cell types appeared to be of a less percentage for some general subsets as expected, we then determined if data for cell percentage was precise with the ROI sampling strategy using specific cell types. Precision was again evaluated by proximity of points and spread of SD (as illustrated in Figure 3.10C). For cell percentage, double positive marker cell types selected exhibited high precision as observed by close data points within low SD (SDpercentage(CD3+CD8+): case 06L3 and 06L30 and SDpercentage(CD8+PD1+): majority of cases), while some tumors exhibit low precision with double markers indicated by scatter of points, and points outside high SD (SDpercentage(CD3+CD8+): case 06L29, 06L57 and (SDpercentage(CD8+PD1+): case 05L17, 05L54, and 06L42, first column of Figure 3.13). Again, these results suggest that with the given ROI sampling strategy, five ROI may be sufficient to provide precise data, while for some tumors five ROI may be insufficient for precise data. Given that the majority of CD8+PD1+ cells 91  demonstrated precise data the majority of cases in the cohort, five ROI may be over sufficient to represent certain types of marker combinations. These results suggest that double positive cells can be detected by the system in tissues.  Triple positive cell class analysis: Conventionally, the triple positive cells in this study, CD3+CD8+CD79a+ and CD8+PD1+PDL1+, are not presently considered as any known major subset of immune cells. Therefore, we expected that triple marker cell abundance in images would be a rare event. Corroborating this prediction, a low percentage of CD8+PD1+PDL1+ and CD8+PD1+PDL1++ in the AR panel (bpercentage(CD8+PD1+PDL1+) = 0.4232%, bpercentage(CD8+PD1+PDL1++) = 0.192%) was observed, though one case (05L54), had slightly higher levels of CD8+PD1+PDL1+ compared to other lung AC in this cohort (first column of Figure 3.14B). However, percentage of triple positive cells was identified for all tumors in the T&B cell panel, where the value was greater than zero (bpercentage(CD3+CD8+CD79a+) = 3.2966%), first column of Figure 3.14A). These values were fairly precise for tumors, as the 95% CI and line of best fit were relatively close together for CD3+CD8+CD79a+ cells (first column of Figure 3.14A). To verify whether these cells exist in images, we visually inspected images for triple positive cells with DUnit. Triple positive cells were observed in images, which may be subject to different interpretations – that these may be a bonafide cell type or more likely, represent a co-localization of two different cell types, for instance, close proximity of CD3+CD8+ and CD79a+ cells or CD8+PD1+ and PDL1+ cells. If the latter is accurate, this would suggest that the segmentation method by this system at present cannot distinguish the different cell types, potentially due to close proximity from occlusion or limitations in segmenting areas with high nuclei density. This observation substantiates additional optimization of upstream steps of the image analysis system procedure (i.e. further optimize cell segmentation or establish the threshold as described in 92  Section 3.2), particularly if a future study entails analysis of cell types identified by triple markers using this platform.    3.4.5 Cell-to-cell spatial relations of same neighbor specific cell types  We then investigated a more complex scenario by assessing more specific cell subsets in the context of spatial modeling. Given that if a smaller cell population of cells was to be assessed, it is anticipated that fewer spatial relations may be assessed. This analysis could serve as a preliminary indicator of the sensitivity of Asterics to detect cell-to-cell spatial relations. Consistent with the previous cell quantity analysis (in Section 3.4.4), the same cell types were assessed with the added complexity of their spatial context in a adjacent to the same neighbors: double marker positive cells that are established cell types used in literature (CD3+CD8+ next to CD3+CD8+, CD8+PD1+ next to CD8+PD1+), and triple positive cells that are expected to be low in abundance (as verified in the previous analysis), as they are not considered any known cell type, including (CD3+CD8+CD79a+ next to CD3+CD8+CD79a+, CD8+PD1+PDL1+ next to CD8+PD1+PDL1+, and CD8+PD1+PDL1++ next to CD8+PD1+PDL1++).       Double positive cell class analysis: Reiterating from the previous cell quantity analysis (in Sub-section 3.4.4) we observed two scenarios regarding cell percentage: (1) a result where specific cell types had a very low percentage and (2) a result where specific cell types had a very high percentage, comparable to the general cell types. The results derived from these cell quantity analyses would impact data interpretation for cell-to-cell spatial relations discussed in this analysis.  93                          Figure 3.13    Specific cell type identified by double markers cell quantity and spatial relations of same neighbors. (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains ROI with large negative value (VAR = -999 indicating case has not enough cells to assess variance), which was excluded from analysis. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.  B A *  Percentage                                           MNG                                                VAR Percentage (CD3+CD8+) CD3+CD8+             CD8+PD1+  Percentage (CD8+PD1+) 94  (1) Double positive cell class low percentage: We observed high precision of MNG of CD8+PD1+ next to CD8+PD1+ cells for tumor, where MNG line of best fit was close to zero (bMNG(CD8+PD1+ /  CD8+PD1+) = 0.0326%), suggesting very few interactions between CD8+PD1+ with their own cell type (second column of Figure 3.13B). This is to be expected as the percentage of  CD8+PD1+ cells was initially very low to begin within the ROI sampled across all tumors in this cohort (bpercentage(CD8+PD1+)  = 0.9355%, first column of Figure 3.13B). Large variation of CD8+PD1+ cells within ROI was identified for the majority of tumors across the cohort (as defined by low precision of VAR values, refer to Figure 3.10C for illustration of low precision). This variation may be attributed to the few cells present in ROI. Large negative VAR was observed in one case of CD8+PD1+ (case 06L30), suggesting that not enough cells were present to perform the analysis (third column of Figure 3.13B, for full scale view refer to Appendix B, third column of B.3B). As previously mentioned, observations regarding negative VAR will be further discussed in detail in the triple positive cell class analysis in this sub-section. Removal of negative VAR enabled plotting of line of best fit within the field of view (third column of Figure 3.13B and refer to Appendix A, third column of A.2B).  (2) Double positive cell class high percentage: Since the average percentage of CD3+CD8+ cells was greater than zero for the majority of the lung AC cohort (bpercentage(CD3+CD8+ / CD3+CD8+) = 7.9835%, first column of Figure 3.13A), we observed that the cell-to-cell spatial relations of these cells were relatively frequent (bMNG(CD3+CD8+ / CD3+CD8+) = 0.2004 MNG, second column of Figure 3.13A). High variation of CD3+CD8+ cells across ROI was identified for some tumors within the cohort as defined by low precision of MNG values (refer to Figure 3.10C for illustration of low precision, MNG(CD3+CD8+): case 06L57, second column of Figure 3.13A). Low variation of CD3+CD8+ cells across ROI was identified for some tumors within the cohort (as  95  defined by low precision of MNG values, (refer to Figure 3.10C for illustration of low precision) (MNG(CD3+CD8+): case 05L9, case 06L74, second column of Figure 3.13A). VAR was also observed to vary across tumors as observed in the spread of 95% CI (bVAR(CD3+CD8+) = 0.1997). These results suggest that variation is present in the cell-to-cell spatial analysis within tumors and across tumors with this sampling strategy. Whether these results are accurate of the true population mean require a larger cohort for validation.  Together, these results provide preliminary indication that suggests the sensitivity of Asterics measuring spatial relations of specific cell types, including double positive cells. Specifically, these results suggest that the ROI sampling strategy may require a baseline level of percentage of a specific cell type of interest in order to detect cell-to-cell spatial interactions and that accurate interpretation requires sufficient cell quantity data. Additional validation of this finding may be useful for larger biological studies using this method. Triple positive cell class analysis: As with analysis of quantity of triple positive cells, we expected that MNG of triple positive cells would be low as the triple positive cells assessed in this study (CD3+CD8+CD79a+ / CD3+CD8+CD79a+, CD8+PD1+PDL1+ / CD8+PD1+PDL1+, and CD8+PD1+PDL1++ / CD8+PD1+PDL1++) are not considered as any known major cell type. Corroborating this hypothesis, we observed that cell-to-cell spatial relations of CD8+PD1+PDL1+ cells next to CD8+PD1+PDL1+ cells was close to zero across all tumors in the cohort (bMNG(CD8+PD1+PDL1+ / CD8+PD1+PDL1+) = 0.0461 MNG), where the line of best fit was relatively close 95% CI bands, suggesting very few interactions that are precise across tumors as expected (second column of Figure 3.14B). These results suggest that separation of cell groups, such as with segmentation, may have been possible for this cell combination (for instance, few cells CD8+PD1+ cells overlapping with PDL1+ or PDL1++ were detected). However, we observed an 96                          Figure 3.14   Specific cell type identified by triple markers cell quantity and spatial relations of same neighbors. (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains ROI with large negative value (MNG or VAR = -999 indicating case has not enough cells to assess variance), which was excluded from analysis. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit or no 95% CI or lines not shown is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.  A B * * * *   * * * * * * * Percentage (CD3+CD8+CD79a+) CD3+CD8+CD79a+           CD8+PD1+PDL1+           CD8+PD1+PDL1++             CD8+PD1+  Percentage (CD8+PD1+PDL1+) Percentage (CD8+PD1+PDL1++) 97  unexpected result, the image analysis platform detected spatial interactions of triple positive cells, where cell-to-cell spatial relations of CD3+CD8+CD79a+ cells next to CD3+CD8+CD79a+ cells had non-zero MNG values, suggesting a relatively high presence of this interaction, especially for CD3+CD8+CD79a+ given nested combinations of more general cell types are present in the tertiary lymphoid structures present in ROI (bMNG(CD3+CD8+CD79a+ / CD3+CD8+CD79a+) = 0.1514 MNG, second column of Figure 3.14A). The specific incidence of these events occurs erratically in an image, as indicated by large dispersion of VAR for the triple positive cells. A number of tumors have large negative VAR value affecting initial plotting in Statistica graphs (refer to cases with asterisks (*) in third column of Figure 3.14B, VAR(CD8+PD1+PDL1+ / CD8+PD1+PDL1+   CD8+PD1+PDL1++ / CD8+PD1+PDL1++)  = -999, for full scale view refer to third column of Appendix B, B.4B). Removal of these negative values enabled effective plotting of line of best fits (refer to third column of Figure 3.14B and Appendix A, third column of A.3B). Aforementioned analyses on VAR also exhibit negative values (refer to Sub-section 3.4.3), this negative variance is most prevalent among cases in the analyses of triple positive cells (35% of tumors in VAR(CD8+PD1+PDL1+ / CD8+PD1+PDL1+))  and is reduced when the PDL1 threshold is more stringent (15% with negative VAR(CD8+PD1+PDL1++ / CD8+PD1+PDL1++), third column of Figure 3.14B). These results highlight that the triple positive cells analyzed to spatial analyses have fewer immune cell quantities to calculate variance and greater marker combinations, which warrants further investigation. Collectively, these results suggest that technical factors may contribute to data interpretation of this image analysis system. Sufficient quantity of cells may be required to perform cell-to-cell spatial analyses. Potentially, detection of overlapping cells may be more challenging for certain panel combinations. Lastly, analysis of a very small population of cells 98  for cell-to-cell spatial relations with Asterics may be limited, and potentially may require additional optimization if a biological studies requires a high sensitivity to detect a similarly low abundance of a specific cell type.      3.4.6 Cell-to-cell spatial relations of different neighbor general cell types   As an even more complex scenario, we determined whether the effects of ROI sampling strategy derived precise data on spatial relations of two different cell type neighbors and identified potential technical factors affecting this analysis. Again, combinations were analyzed in this sequence as general populations (defined by a single marker) were expected to have a greater pool of cell quantity for analysis. Quantities of these general cell types were already analyzed in Sub-section 3.4.2. Combinations of these general markers were selected to compare cell-to-cell spatial relations, as they are known to biologically localize, including directly interacting with one another. The following combinations were used for this analysis where the latter was subdivided into an additional two parts: (1) general CD3+ T cell subset next to CD79a+ B cells or plasma cells, such as in the organization of tertiary lymphoid structures and (2) general PD1+ cell interaction with (2a) cognate ligand PDL1+, and (2b) with a more stringent threshold defined by PDL1++ cells.  In this analysis, we also evaluated whether the cell-to-cell spatial relations would be different by reciprocally assigning which cell would be the reference cell in a specific interaction. For instance, the cell-to-cell spatial relations between CD3+ and CD79a+ cells were assessed, where CD3+ is a reference cell, and then CD79a+ as the reference cell. If cell-to-cell spatial relations are dependent on the reference cell, this would suggest that the ROI sampling 99  strategy detected that cellular neighborhoods for individual cell types are different, indicating that there may be biological complexity of spatial microhabitats of cells within the tumor microenvironment. We first assessed whether cell-to-cell spatial relations of two different neighbors of general cell types could be detected with Asterics. We expected that cell-to-cell spatial relations could be detected for one cell combination (CD3+ next to CD79a+ cells) as sufficient quantities of these cell groups were present in lung AC (bpercentage(CD3+) = 10.8268% and bpercentage(CD79a+) = 7.6887%, first column of Figure 3.11). As expected, we identified that a small percentage of these cell-to-cell spatial relations were able to be detected across the lung cohorts (bMNG(CD3+ / CD79a+)  = 0.075 MNG,  bMNG(CD79a+ / CD3+)  = 0.1208 MNG;    bMNG(CD3+ / CD79a+   CD79a+ / CD3+)  > 0, first column of Figure 3.15A). As a negative control, we also expected that cell-to-cell spatial relations would be very few for another heterogenous combination of cells (PD1+ next to PDL1+ or PDL1++), as quantities of some of these cell types were initially very low (bpercentage(PD1+) = 1.1279%,  bpercentage(PDL1+) = 8.0289%, and bpercentage(PDL1+) = 1.8788%, first column of Figure 3.12). As anticipated, we identified that a low amount of these cell-to-cell spatial relations were detected in the lung cohort for the majority of PD1+ and PDL1+ or PDL1++ combinations (bMNG(PD1+ / PDL1+) = 0.05 MNG,  bMNG (PDL1+/ PD1+)  = 0.0049 MNG, and bMNG (PD1+/PDL1++) = 0.0054 MNG, bMNG (PDL1 ++ / PD1+) = 0.004 MNG;   bMNG(PDL1+/ PD1+    PD1+/PDL1++   PDL1 ++ / PD1+)  ≃ 0, first column of Figure 3.15B). These results emphasize the importance of combining quantitative metrics, including pairing percentage data with cell-to-cell spatial relations analysis for analyzing different neighbor types, to facilitate the interpretation of data using Asterics.  We noticed that the quantity of cell-to-cell spatial relations differed depending on which cell is the reference cell. There appears to be very low frequency that PDL1+ cells are next to a  100                                                                                                                        A CD3+ / CD79a+         CD79a+ / CD3+          PD1+ / PDL1+          PDL1+ / PD1+         PD1+ / PDL1++          PDL1++ / PD1+   B                 MNG                                                    VAR * * 101  Figure 3.15    General cell type next to different general cell type neighbor cell quantity and spatial relations and vice versa. (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains ROI with large negative value (VAR = -999 indicating case has not enough cells to assess variance), which was excluded from analysis. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with no line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.  PD1+ cell bMNG(PDL1+/ PD1+) = 0.0049 MNG ≃ 0), while higher frequency that a PD1+ is next to a PDL1+ cell (bMNG(PD1+ / PDL1+) = 0.05 MNG), suggesting that the cell neighborhoods are different in this pair of interactions, highlighting biological complexity in the microhabitat of tumors (first column of Figure 3.15B). There are relatively no interactions with PD1+ and PDL1++ cells in both conditions and the difference between reciprocal assignment is negligible (bMNG(PD1+ / PDL1++) = 0.0054 MNG, bMNG(PDL1++/PD1+) = 0.004 MNG, |bMNG(PDL1++/PD1+- PD1+ / PDL1++)| = 0.0014 MNG (first column of Figure 3.15B). For both combinations investigated, we observed that generally, the ROI sampling strategy produced precise data (as illustrated in Figure 3.10E),  suggesting that cell-to-cell spatial relations are relatively constant across different areas in tumors with high immune cell infiltration. The variability of these spatial relations appears to have consistent variability within images for some cell combinations, as precise values of VAR were present (as illustrated by Figure 3.10C), given the tight cluster of cells of the five ROI with each other for majority of cases (bVAR(CD3+ / CD79a+) = 0.1264 VAR, bVAR(CD79a+ / CD3+ ) = 0.1428 VAR,  bVAR(PDL1+ / PD1+) = 0.0252 VAR, bVAR(PDL1++ / PD1+) = 0.0185 VAR, second column of Figure 3.15B). However, the negative VAR was also exhibited for some cases with cell-to-cell spatial analysis between heterogeneous general cell types (VAR(PD1+ / PDL1+) and  VAR(PD1+ / PDL1++): case 05L54 = -999, VAR asterisks (*) in second column of Figure 3.15B, refer to Appendix B, second column of Appendix B.5B), indicating that not enough cells are present to 102  calculate variance. Removal of this negative VAR value enabled plotting of the line of best fit (second column of Figure 3.15B and second column of Appendix A.4B). Collectively, these results indicate that Asterics is sensitive to detect a small population of cell-to-cell spatial relations if sufficient quantities of cell numbers are initially present. Together, these results indicate that the ROI selection strategy gave precise data for analysis of heterogenous general cell subsets and relatively consistent variance of these relationships within images. Furthermore, these results highlight the complexity of microhabitat of single cells could be detected with Asterics. One case was identified with too few cells for variance analysis, suggesting additional optimization is required to account. Future work should explore whether the sensitivity of Asterics is specific to this panel or if these results are generalizable – that Asterics is sensitive to detect differences in single cell microhabitats that is reference cell dependent.   3.4.7 Cell-to-cell spatial relations of general and specific cell type neighbors  Lastly, we assessed whether precise data was generated for cell-to-cell spatial relations between different cell neighbors, where one group was more specific than the other. This is the most sophisticated model of the tumor microenvironment that was tested with Asterics. Rather than general CD3+ T cell subset or PD1+ cells, cell-to-cell spatial relations of more specific CD3+CD8+ T cells, including cytotoxic T cells and CD8+PD1+ T cell, were examined in relation to their general cell type counterparts. The following biologically relevant cell combinations were tested: CD3+CD8+ cells next to CD79a+ cells, CD8+PD1+ cells next to PDL1+ or PDL1++ cells, and vice versa (reciprocal assignment of reference cell) to assess whether the sampling strategy captures potential differences in microhabitats in these more specific types of cells.  103  To determine if the image analysis system has sensitive detection, percentage data was correlated with cell-to-cell spatial relation quantities. We expected that less CD3+CD8+ next to CD79a+ cell-to-cell spatial relations to be present, as fewer CD3+CD8+ cells were compared to general CD3+ cells in the same lung cohort image dataset (bpercentage(CD3+CD8+) = 7.9835% versus bpercentage(CD3+) = 10.8268%, first column of Figure 3.13A and first column of Figure 3.11 respectively). As expected, we observed the presence of CD3+CD8+ and CD79a+ cell-to-cell spatial relations, which were less than general CD3+ (bMNG(CD3+CD8+ / CD79a+ = 0.0562 MNG, bMNG(CD79a+ / CD3+CD8+) = 0.0681 MNG, first column of Figure 3.16A versus bMNG(CD3+ / CD79a+) = 0.075 MNG, bMNG(CD79a+ / CD3+ ) = 0.1208 MNG, first column of Figure 3.15A). The cell-to-cell spatial relations between the reciprocal assignment of reference cells were much similar in the specific cell combinations (|bMNG(CD3+CD8+ / CD79a+- CD79a+ / CD3+CD8+)| = 0.0119 MNG first column of Figure 3.16A versus |bMNG(CD3+ / CD79a+ - CD79a+ / CD3+)| = 0.0458 MNG, first column of Figure 3.15A). These results suggest that the cell-to-cell spatial relations are reference cell independent for analyzing different neighbor specific cell next to general cell types and vice versa. Different cell sizes may contribute to differences cell percentage. For instance, a larger cell may have more area for potential cells surrounding rather than a smaller cell. The sensitivity for double marker cell detection cell-to-cell spatial relations is possible, though the sensitivity may not be as strong with the fewer amount of cells present. As a small cohort was used in this study, future work could investigate if these quantities are accurate to the population mean and to determine the lowest cell percentage that can be used to accurately detect cell-to-cell spatial relations. More cell combinations are required (i.e. two marker versus one marker) to determine if reference cell independence is a feature of these specific markers or generalizable to different marker combinations. As a negative control, it was expected that very low cell-to-cell spatial relations 104  would be present in the CD8+PD1+ next to PDL1+ or PDL1++, as aforementioned some of these individual, general subsets had very low abundance of cells initially present (i.e. bpercentage(PD1+) = 1.1279%, bpercentage(PDL1+) =1.8788%, first column of Figure 3.12) and the cell type of interest, CD8+PD1+, also had few cells present (bpercentage(CD8+PD1+) = 0.9355%, first column of Figure 3.13B). The quantity of cell-to-cell spatial relations in the more specific cell types compared to the general cell types have similar values (bMNG(CD8+PD1+/ PDL1+) = 0.0504 MNG, bMNG(PDL1+ / CD8+PD1+) = 0.0049 MNG, bMNG(CD8+PD1+/ PDL1++) = 0.0064 MNG, bMNG(PDL1++/ CD8+PD1+) = 0.004 MNG, first column of Figure 3.16B, bMNG(PD1+/ PDL1+) = 0.05 MNG, bMNG(PDL1+/ PD1+) = 0.0049 MNG, bMNG(PD1+ /PDL1+) = 0.0054 MNG, bMNG(PDL1+ /PD1+) = 0.004 MNG, first column of Figure 3.15B). The difference between cell-to-cell spatial relations in a more specific and general cell type neighbors is small (|(bMNG(CD8+PD1+ /PDL1+ - PD1+ /PDL1+)| = 0.0004 MNG, |(bMNG(PDL1+ / CD8+PD1+ - PDL1+ / PD1+)|= 0 MNG, |(bMNG(CD8+PD1+ /PDL1++ - PD1+ /PDL1++)| = 0.001 MNG, |(bMNG(PDL1++ / CD8+PD1+ - PDL1++ / PD1+)|= 0 MNG, first column of Figure 3.16B and Figure 3.15B). This low probability of cell-to-cell spatial relations is expected as few quantities of cells are present in tissue. We noticed that the precision of points for cell-to-cell spatial relations was also generally precise (given the tight cluster of cells of the five ROI with each other as illustrated by Figure 3.10C) for the majority of cases in all graphs with few cases exhibiting low precision (first column of Figure 3.16A and Figure 3.16B). These results illustrate that the detection limit of Asterics for both general and specific cell types for cell-to-cell spatial relations may depend on the initial quantity of cells present, and is shown here to be precise. Since little variation was present across ROI, we then investigated if variation was present within each ROI. We noticed that VAR was relatively precise for most cases as ROI of cases were close together for the majority of specific cell-to-cell spatial relations tested in this  105                            A B CD3+CD8+ / CD79a+           CD79a+ / CD3+CD8+           CD8+PD1+ / PDL1+          PDL1+ / CD8+PD1+           CD8+PD1+ / PDL1++         PDL1++ / CD8+PD1+                                                              MNG                                                    VAR * * 106  Figure 3.16    General cell type next to specific cell type neighbor and vice versa cell quantity and spatial relations. (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that whiskers of SD for the indicated case contains large negative variance, suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Graphs that case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.   section (as illustrated in Figure 3.10C), suggesting that each ROI has a rather precise variance across the whole cohort of lung AC as observed by close 95% CI with line of best fit (second  column of Figure 3.16A and Figure 3.16B). For different neighbor specific-general cell combinations with positive cell-to-cell spatial relations, variance was observed to be independent of reference, as observed by small difference in VAR between reciprocal assignments   (bVAR(CD3+CD8+ / CD79a+) = 0.1096 VAR; bVAR(CD79a+ / CD3+CD8+) = 0.1088 VAR, |bVAR(CD3+CD8+ / CD79a+- CD79a+ / CD3+CD8+)| = 0.0008 VAR, second column of Figure 3.16A). However, for cases that were expected to have few cell-to-cell spatial relations, the analysis was disrupted by an outlier case that exhibited significant negative VAR (case 06L30: VAR(CD8+PD1+ / PDL1+) = -999 VAR; VAR(CD8+PD1+ / PDL1++) = -999 VAR, asterisks (*) in second column of Figure 3.16B and refer to Appendix B, second column of B.6B). The negative VAR indicates that for several cases that not enough cells are present to calculate variance. Negative VAR skewed the interpretation of average line of best fit (bVAR(CD8+PD1+ / PDL1+) = 4.3028 VAR; bVAR(CD8+PD1+ / PDL1++) = 4.2308 VAR, vertical line drawn in Statistica as out of bounds, refer to Appendix A, second column of A.5B). However, removal of the negative VAR enabled plotting of the line of best fit (second column of Figure 3.16B). Values of VAR are close to zero and are rather precise as 95% CI bands are close together in combinations of CD8+PD1+/PDL1+ and CD8+PD1+/PDL1++, including combinations with the reciprocal reference cell (second column of Figure 3.16B). 107  Thus, these results indicate that the variance of cell-to-cell spatial relations across images is rather precise, as variability across ROI is low within tumors and across tumors.  In summation, these results indicate that the ROI selection strategy gave precise data for analysis of heterogenous general cell subsets. Furthermore, these results suggest that the complexity of microhabitats of single cells may be less likely detected with specific cell types if present at a low quantity. Thus, these results indicate that presently the analysis of specific cell types and general cell types may not be sensitive enough to detect changes in the microhabitat. It may be potentially better to use more general subsets for different neighbor analysis, where a greater cell quantity was present as observed from previous analysis. Additional validation of this relationship is required, and future work would need to verify if these results are biologically accurate in a larger cohort.   3.4.8 Summary of quantity and cell-to-cell spatial relations analyses with Asterics   Taken together, these findings have illustrated a proof-of-principle of the feasibility of using Asterics to study the cellular geography and sociology of cell microhabitats within tumor microenvironments. Furthermore, these findings identified potential associated conditions to contribute towards optimal cell quantity and cell-to-cell spatial relation analysis using Asterics. These results highlight areas for further optimization in the future. Briefly, these analyses have identified that the manual ROI sampling strategy presented in this study for immune cell infiltrate areas generates variability, where some cases may require more sampling while others less for precise data. These results have also suggested that several technical factors may bias data interpretation (including stain type, threshold establishment, initial cell percentage in 108  images, and segmentation to separate overlapping cells). Experiments validating if these speculated technical factors effect Asterics are required. The study highlights the value of incorporating multiple quantitative cell metrics, including cell quantity, to inform interpretation of cell-to-cell spatial analysis. As the feasibility of this cell-to-cell spatial analysis has been demonstrated in this pilot study, a larger cohort is required to determine if data precision corresponds with accurate results. This study also identified that negative variance was observed using Asterics for cell-to-cell spatial analysis, indicating that for some cases not enough cells are present to calculate variance. A detailed summary of conclusions can be found in Table 3.10 of this thesis. In this section, we implemented a manual approach to determine areas for image acquisition, which has also been used in other studies with a blinded pathologist [122]. It would be interesting to explore the application of an automated system to detect ROI using deep learning methods as performed by other investigators and commercial systems, such as identification of immune cell hot spots, and to compare the accuracy of the manual approach of ROI selection with this automated approach for virtual biopsy of tumors [115, 124, 135, 150, 167]. Professional and regulatory societies, including the American Thoracic Society (ATS) and European Respiratory Society (ERS) are aware of the limitations of two dimensional morphometry and quantitative analysis and are beginning to take precautions by creating standards to measure lung structure using three dimensional analysis or stereology [161]. Discrepancies between two dimensional and three dimensional analyses of cell quantification have been observed in the past decades [161]. Cognizant of these limitations, image registration development for this image platform is rationalized as an important tool to account for stereology of tissue as a potential approach to reduce selection bias of digital pathology. 109  Sub-section # Cell population Examples from panels Parameters  assessed Asterics metric Conclusions / Future Directions Neighbor type  (# of cell subsets) Population properties  (# of markers) T&B cell  Adaptive immune resistance  3.4.2 Same neighbor (1) General (1) CD3+ CD79a+ CD8+ CD8+ PD1+ PDL1+ PDL1++ Cell quantity Percentage Significance: analysis with the least complex scenario with the largest available pool of cells (no spatial modeling)   Five ROI may be sufficient, insufficient, or oversufficient to represent cell percentages of lung AC tumors   ROI sampling strategy cannot be generalized to be precise or not and these results are reproducible in different panels  A larger cohort is required to confirm if variation related to biological factors (i.e. accuracy of results to population mean, section specific effects from tumor variability)  Technical factors that correlate with data interpretation of cell percentages need to be verified (i.e. differences in stains and threshold biases in determining data precision) 3.4.3 Same neighbor (1) General A (1) / General A (1) CD3+ / CD3+ CD79a+ / CD79a+ CD8+ / CD8+ CD8+ / CD8+ PD1+ / PD1+ PDL1+ / PDL1+ PDL1++ / PDL1++ Cell-to-cell spatial relations MNG  VAR Significance: analysis with more complex scenario with the largest available pool of cells (with spatial modeling)   Five ROI may be sufficient, insufficient, or oversufficient to represent cell percentages of lung AC tumors   A larger cohort is required to confirm if variation related to biological factors (i.e. accuracy of results to population mean, section specific effects from tumor variability)  Variation is present in cell-to-cell spatial relations   Combining different quantitative metrics with cell-to-cell spatial relations is valuable for data interpretation (i.e. percentage and cell-to-cell spatial relations)  Technical factors that correlate with data interpretation of cell-to-cell spatial relations need to be verified (i.e. differences in stains and threshold biases in determining data precision)  For several cases, not enough cells are present to calculate VAR (indicated by large negative VAR in data) 3.4.4 Same neighbor (1) Specific (2) CD3+CD8+ CD8+PD1+ Cell quantity Percentage Significance: assesses sensitivity of Asterics to detect more specific cell subsets. Analysis with the least complex scenario with the smaller available pool of cells (no spatial modeling):   Specific cell types were found to be of smaller population as expected, but also larger than general cell type populations   Technical factors effecting this unexpected result require verification (i.e. segmentation of clusters and overlapping cells)  For case with that had sufficient number of specific cell types present – five ROI may be sufficient, insufficient, or oversufficient sampling in some tumors  Specific cell types (detected with two markers) can be detected, though additional optimization may need to be required Specific (3) CD3+CD8+CD79a+ CD8+PD1+PDL1+ CD8+PD1+PDL1++ Cell quantity  Percentage Significance: Negative control - determine if non-conventional cells are detected in images  Non-conventional cells were rarely detected in one panel as expected, while non-conventional cells were present, potentially due to colocalization of cells   Presence of non-conventional cells may be due to colocalization of cells   Future work may consider improved detection of overlapping cells Table 3.10   Summary and future directions of analyses performed with Asterics. (Continued on page 110).  110   Sub-section # Cell population Examples from panels Parameters  assessed Asterics metric Conclusions / Future Directions Neighbor type  (# of cell subsets) Population properties  (# of markers) T&B cell  Adaptive immune resistance  3.4.5 Same neighbor (1) Specific A (2) / Specific A (2) CD3+CD8+ / CD3+CD8+ CD8+PD1+ / CD8+PD1+ Cell-to-cell spatial relations MNG  VAR Significance: analysis with more complex scenario with smaller available pool of cells (with spatial modeling)   For cell-to-cell spatial relations analysis with sufficient cell quantity present, cell-to-cell spatial relations is relatively precise for specific cell types  For several cases, not enough cells are present to calculate VAR (indicated by large negative VAR in data)  Therefore, baseline level of cell percentage is important for cell-to-cell spatial relations analysis Specific A (3) / Specific A (3) CD3+CD8+CD79a+/ CD3+CD8+CD79a+ CD8+PD1+PDL1+ / CD8+PD1+PDL1+ CD8+PD1+PDL1++ / CD8+PD1+PDL1++ Cell-to-cell spatial relations MNG  VAR Significance: determine if non-conventional cells are detected in images using spatial relations analysis  Cases with high cell quantity had high cell-to-cell spatial relations  For several cases, not enough cells are present to calculate VAR (indicated by large negative VAR in data) 3.4.6 Different neighbor  (2) General A (1) / General B (1) CD3+ / CD79a+ PD1+ / PDL1+ PD1+ / PDL1++ Cell-to-cell spatial relations MNG  VAR Significance: most complex scenario (with spatial modeling) with larger available pool of cells and assess if reference cell matters is dependent on ROI sampling (i.e. sensitive to detect changes in microhabitat)  Heterogenous general cell-to-cell spatial relation in cases with higher cell percentage can be detected, where as those with low percentage had few cell-to-cell spatial relations detected  Combining different quantitative metrics with cell-to-cell spatial relations is valuable for data interpretation of heterogenous general cell types (i.e. percentage and cell-to-cell spatial relations)  Some cell combinations that had different cell-to-cell spatial relations, suggesting Asterics may be sensitive to detect differences in microhabitat for heterogenous general subsets  Cell-to-cell spatial relations were precise and constant across image  For several cases, not enough cells are present to calculate VAR (indicated by large negative VAR in data) General B (1) / General A (1) CD79a+ / CD3+ PDL1+ / PD1+ PDL1++ / PD1+ Cell-to-cell spatial relations MNG  VAR 3.4.7 Different neighbor  (2) General (1) /  Specific (2) CD3+CD8+ / CD79a+ CD8+PD1+ / PDL1+ CD8+PD1+ / PDL1++ Cell-to-cell spatial relations MNG  VAR  Significance: most complex scenario (with spatial modeling) with least available pool of cells and assess if reference cell matters is dependent on ROI sampling (i.e. sensitive to detect changes in microhabitat)  Cell-to-cell spatial analyses were reference cell independent, indicating that the software may not be sensitive to detect cell-to-cell spatial relations of different neighbors of specific next to general cell types or vice versa  ROI sampling strategy correlated with precise data  For several cases, not enough cells are present to calculate VAR (indicated by large negative VAR in data) Specific (2) /  General (1) CD79a+ / CD3+CD8+ PDL1+ / CD8+PD1+ PDL1++ / CD8+PD1+ Cell-to-cell spatial relations MNG  VAR Dashed line = divides analysis within same sub-section; General = refers to cells defined by a single marker; Heterogenous = refers to two different cell populations or neighborhood groups being assessed; Homogenous = refers to two same cell populations or neighborhood groups being assessed; MNG = mean neighborhood group; Slash (/) = ‘next to’; Specific = refers to cells defined by multiple markers; VAR = variance of MNG 111  3.5  Image registration  Image registration is an important component of image analysis by merging datasets derived from more than one image for multidimensional analysis [168]. Image datasets can include phenotypic and functional information [169]. In the context of this hyperspectral image analysis system, image registration would enable analysis of spatial distribution of the same or different cell populations in more than one plane of the tumor. This capacity is important, accounting for the fact that tumors are three dimensional structures with tissue architecture that exhibit heterogeneous spatial organization and two dimensional analyses of single sections fail to completely capture this information, introducing selection bias [161]. Thus, the BCCRC hyperspectral image analysis system valuably has an image registration component to include prospective multimodal analysis of different stains and molecular markers or multiplanar spatial analysis to provide stereological insights on tumor immune cell distribution. The type of image registration used by Imreg.m is intensity-based, where similar intensities were matched between fixed and moving images, and non-rigid, as an affine was used.   To assess the workflow of the image registration software of the BCCRC hyperspectral image analysis system, Imreg.m, a subset of tumors were selected for this analysis. Hematoxylin and CD8+ were selected to perform image registration as these markers were stained in both panels (T&B cell and AR panel), and image features are expected to match in these image pairs. To have sufficient stain intensity to assess accuracy of image registration, cases were selected with the highest of any type of CD8+ cell. Any type of CD8+ refers to the sum of single marker, double marker, and triple marker cells positive for CD8. In order to have the most data available to calculate averages, cases also needed to have data available for all five ROI where the 112  majority of cells in hyperspectral images of tumor were able to be segmented. With these inclusion criteria, the following three lung AC were selected from the cohort to perform image registration: 05L14, 05L4, and 06L29 (Table 3.11). Despite the same antibody clone used in both  sections, the percentage of any CD8+ cells was different in adjacent sections, as with total CD8+ cells for each case (first column of Figure 3.11 and Figure 3.12). Percentage of any CD8+ cells was lower in the adaptive immune resistance panel compared to the T&B Cell panel.   Images could not be registered at the cell-to-cell level between adjacent sections. Image  registration of single cells still presents as a challenge, as immune cells are small in size (human lymphocytes range from 7 to 10μm, where larger granular lymphocytes range from 10 to 12μm), which may  not span the width of tissue sections (4 μm thick in this study) if multiple adjacent sections are required [170, 171]. Such challenges also exist with image registration of single cells with sequential acquisition methods, where a single section is imaged with a first set of markers added and reimaged after stain removal and restaining with a second set of markers [41]. In addition, cells may get physically pushed into one section or the other by the thickness of the sectioning blade. Thus, the highest resolution at which the image registration software can currently assess is determining spatial distributions of immune cells between sections and not at the single cell level. Presently, no objective measures of image registration are available with the BCCRC image registration software. However, visual assessment of registered images is considered an accepted approach for image registration assessment [172]. The following image registration figures in this section (Section 3.5) have been compressed.        113  Table  3.11    Sample selection for image registration analysis. (A) T&B Cell panel. (B) Adaptive Immune Resistance. Averages of five ROI shown. Red cells indicate cases with less than five ROI and were not selected for analysis.  Yellow cells indicate that cases had highest average of percentage of any cell with CD8+ and were selected for image registration.      Case Mean of any CD8+ cell (percentage) 05L14 16.9878 06L29 15.60655 05L4 14.65004 06L50 13.15282 05L17 11.93655 05L54 10.96252 06L74 10.7725 05L27 10.70428 06L57 10.64163 06L3 10.28953 06L42 8.796242 05L39 8.041601 06L71 7.031383 05L12 5.825822 06L30 4.876695 06L10 4.590661 05L49 4.514275 05L9 4.273412 05L46 4.134632 06L14 3.000233 06L19 2.813438   The average file size of registered images (including reciprocal alignments) was ~114 MB (7,187,076,080 bytes per registered image ÷ 1,048,576 bytes per MB ÷ 60 registered images = ~114.2 MB). Total file size of all output files containing registered images was 6.69 GB. Average file size of individually registered image files are smaller than ‘Scaled_Inverted Concentration’ images in Section 3.1 (148 MB – 114 MB = 34 MB). Several adjacent sections Case Mean of any CD8+ cell    (percentage) 06L29 29.78246 05L4 23.34376 06L50 22.25003 06L19 21.14594 05L14 20.23605 05L27 19.3632 05L54 18.21045 06L57 17.39396 06L74 15.27533 05L17 13.92758 06L42 11.94573 06L3 10.31713 05L39 9.562668 06L71 8.470634 05L12 8.198894 06L30 7.947859 05L49 7.533698 06L10 7.448415 05L46 7.045617 05L9 5.283316 06L14 4.003303 A B 114  could be reproducibly aligned (Figure 3.17). Images had more grey areas rather than brightly colored areas (green and magenta), indicating similar intensities, however, colored areas do not necessarily indicate that areas are misregistered to Section 3.8 (Figure 3.17).        Figure 3.17    Examples of well registered images. Grey areas indicate that images of adjacent sections have similar intensities. Colored areas indicate that intensities of area in image are brighter than the other, where magenta indicates moving image, and green indicates fixed image. Figure has been compressed.       115   Image registration with this software revealed that some adjacent sections could not be reproducibly aligned (Figure 3.18). Technical factors affecting image registration are described here and in Section 3.6 to 3.8. Manual imaging was observed to be different between adjacent sections for cases, which may potentially affect the reproducibility of image registration with Imreg.m. Image registration of ROI onto WSI to confirm location may be a method to further confirm the location of images and improve image quality for image registration [173]. Interpretation of imaging and subsequent analysis will be relevant in clinical settings, as inter-observer variability was present when pathologists were asked to image the same area [84].   Presently, tissue deformation, such as from tissue processing, was not corrected prior to  image registration. Tissue stretching is known to occur before image analysis, including during fixation and sectioning of tissues. Non-rigid transformation methods have been used to correct cutting distortions from kidney sections [174]. Pre-processing of images is a potential solution and is an important factor consider for standardizing digital pathology images [175]. Correction of regional deformation, such as the image warping approach used for two dimensional gels, may also present as a potential solution [176]. Fiducial marker placement in tissue may be a potential method as described in the skull for CT scans to obtain ground truth transformation [168]. Collectively, image quality presents as an important factor to consider for the workflow of this hyperspectral image analysis system and object measure to determine image quality is an important factor to consider for standardizing digital pathology images [175].           116            Figure 3.18   Examples of differences in reproducibility for image registration. Grey areas indicate that images of adjacent sections have similar intensities. Colored areas indicate that intensities of area in image are brighter than the other, where magenta indicates moving image, and green indicates fixed image. Figure has been compressed.             117  3.5.1 Different stains  Registration of multimodal information provides different functional information of the same area, such as multimodal method of image registration with MRI and histopathology [177]. Although there is value in such an approach, feasibility remains an issue. However, different immunohistochemical stains could potentially affect image registration of two images. To determine if image registration software of the BCCRC hyperspectral image analysis system was effected by different stains, registered images of markers of same stain (hematoxylin) were compared with images of the same marker using different stains (CD8, red and brown). Images with the same stain, such as hematoxylin, of adjacent sections given sufficient similarity in appearance were observed to be well-aligned (Figure 3.17). However, images with different chromogen stains were observed to not be sufficiently similar in appearance or not well-aligned (Figure 3.19). Antibody clones were the same in both adjacent sections, but differences were observed in cell counts (Table 2.2 and 2.3, Figure 3.19). Antibody differences are a challenge, even for established biomarkers, such as PDL1, where further standardization is required [178-180]. A potential solution would be to record the geometric transformation of same stain images with high similarity as a control, such as counterstain and apply to the alignment to differently stained images [181]. Classifiers with a reference slide can be applied to serial sections to enable multimodal stain image registration as done by commercial systems [115]. Multiresolution block matching and methods to improve structure similarity for differently stain adjacent sections have also been performed [182, 183]. Another method requires no stain of tissue by visualizing morphology and features by Fourier transform infrared (FT-IR) spectroscopic imaging of various non-malignant, precancerous, and malignant tissues [184]. Methods for stain normalization are 118  also implemented, including control slides for calibrating heterogenous staining, or with sparse autoencoders (StaNoSA) [127, 185, 186].      Figure 3.19    Examples of differences in stain of CD8. Grey areas indicate that images of adjacent sections have similar intensities. Colored areas indicate that intensities of area in image are brighter than the other, where magenta indicates moving image, and green indicates fixed image. Figure has been compressed.        119  3.5.2 Local landmarks  Landmarks in tissue may be used as a guide to accurately register two serial images in a process known as ‘image matching’ [168]. For other imaging modalities, such MRI, anatomical landmark detection is essential for accurate image registration [187]. Imreg.m performs image registration by matching intensities of pixels in the moving and fixed image. This is referred to as intensity-based image registration. Although Imreg.m performs intensity-based image registration, the size of landmarks was observed to correlate with registration of the BCCRC hyperspectral image analysis system. Images with large local landmarks were relatively well-aligned based on visual observation (Figure 3.17). Images with small local landmarks were challenging to assess registration in comparing sufficient similarity for adjacent sections and were considered unclassified (Figure 3.20). This finding corroborates with other studies, that poor histological structures in images effected the quality of image registration as for image registration of slides with sequential labeling and erasing methods [87]. A metric to measure whether images are sufficiently similar prior to image registration may be useful such as an initial image difference of common information such as with thresholded images as in one study, where stereology approaches for ROI detection or using tumor masks to distinguish areas within the tumor may contribute to additional automated method to verify image matching during or after image acquisition in adjacent sections [181, 188, 189].  With stereology, the staining of every three to four sections to observe nuclei morphology at different levels to improve image registration correlation, and an interactive interface to confirm image registration by viewing different transparency and opacity of images is another approach used to verify landmarks  120       Figure 3.20    Examples of small landmarks. Grey areas indicate that images of adjacent sections have similar intensities. Colored areas indicate that intensities of area in image are brighter than the other, where magenta indicates moving image, and green indicates fixed image. Figure has been compressed.            121   at different levels [127]. Automated methods for landmark detection would be valuable or even a zoomable interface for ROI detection as available for the FDA-approved WSI digital pathology systems by OmnyxTM and Philips [135, 190]. With this current system, potential solutions include that prior to image acquisition, ROI are selected with large, prominent local landmarks for image registration. Alternatively, artificial introduction of landmarks, such as fiducial marker placements in tissue, could present as another solution. The use of TMA rather than whole tissue may better ensure the same region would be imaged and potentially limit intra-observer variability in imaging [191].   3.5.3 Tissue artifacts  Tissue artifacts may be observed in images, including tissue wrinkles, folds or pleats, and scratches [87]. ROI of this image registration experiment were observed to have tissue artifacts. Images with large tissue artifacts were not well-aligned based on visual confirmation (Figure 3.21). Images without large tissue artifacts could be well-aligned (Figure 3.17). Therefore, tissue artifacts are a potential technical factor that would affect image registration. The observation of tissue artifacts indicates that upstream events prior image analysis will affect subsequent image analysis. Such features are important for image quality control [111]. Image quality control will be an important aspect to address for telepathology methods and establish for routine digital pathology analysis. In addition to tissue processing artifacts, other artifacts exist, including those during pre-fixation or fixation. Pre-fixation artifacts include those during surgery from injection,     122               Figure 3.21   Examples of tissue artifacts. Folds in tissue were not well aligned. Magenta image indicates moving image. Green image indicates fixed image. Figure has been compressed.           123  forceps incorporating crush artifact or fulguration from surgical cautery equipment or during fixation, freezing, and transport [192]. Tissues can shrink by 33% or if fixation is prolonged,  secondary shrinkage could occur, which may distort the dimensions and scaling of the tumor architecture [192, 193]. Potential solutions to mitigate observable tissue artifacts include image preprocessing which are discussed in more detail in the following section, Section 3.9.   3.5.4 Image preprocessing with image cropping  Image preprocessing refers to improving image features for downstream processing operations. Image preprocessing may be useful for images with tissue artifacts. Incorporating algorithms for recognition of tissue artifacts, such as for folds, is one approach to identify tissue artifacts and improve image quality [194]. Other methods of this operation include color saturation and intensity based methods to eliminate artifacts [111, 195, 196]. Image cropping is another method to improve image matching, which has been performed to smaller ROI in lung adenocarcinoma to improve image registration [127]. Virtual image cropping is also performed in other commercial systems [124]. To test the application of image preprocessing for the BCCRC image registration software, a total of three sample images that contained image skew, irreproducible alignment, and tissue artifacts were selected for the analysis from Section 3.5 to 3.9, including Figure 3.18 and 3.19. To test the incorporation of preprocessing steps with Imreg.m, images were cropped in Microsoft Office Picture Manager to areas where registered images overlapped or to areas without tissue artifacts and then registered using Imreg.m (Figure 3.22).         124                                   Figure 3.22  Image preprocessing with image cropping in Microsoft Office Picture Manager. (A) Images selected with misregistration for image preprocessing analysis. (B) Areas cropped in image registration. Black box indicates cropped area selected. Figure has been compressed.   A B T&B Cell ARPDL1high 125  Initially, Imreg.m could not proceed to complete the run and create an output file. Preliminary image displayed (imshow) the test image, indicating that the final image is similar. The error indicates that the file size is too small to perform image registration. It was also observed in MATLAB workspace that images that have undergone image preprocessing with Microsoft Office Picture Manager were read as three dimensional matrices rather than two dimensional, which made the images incompatible with the script (Figure 3.23). Additional lines of syntax were added to command image registration software to issue the preprocessed images in the first page, being the red (R) matrix containing image data of hematoxylin in the three dimensional RGB matrix. Implementation of this fix enabled preprocessed images to be  registered in a test run when one moving image was preprocessed and when both images are                                             Figure 3.23   Preprocessing of images with Microsoft Office Picture Manager is recognized as a three dimensional image in MATLAB workspace. (A) Preprocessed image recognized as 3 dimensional by Imreg.m. (B) Fix to Imreg.m script. Preprocessed images recognized as 2 dimensional and additional variables indicated for geometric transformation of image registration output file. Examples are of different images.     A Fixed = preprocessed Moving = without preprocessing Fixed = preprocessed Moving = preprocessed B 126  preprocessed (Figure 3.24). Syntax to measure execution time of image registration software was then implemented to the Imreg.m with fixes for preprocessed images by incorporating the MATLAB tic toc function and additional syntax to print performance time into a Microsoft Excel file and move files to an external folder were also included. Image registration script did not measure execution time of Microsoft Excel file printing of time and moving image files, as values needed to be first defined for time function for software performance. Thus, execution time only measures the elapsed time to perform image registration.     Figure 3.24  Image registration with fixes implemented of preprocessed images. (A) Image registration output files with Imreg.m. (B) Image registration of preprocessed images with fixes implemented in Imreg.m. Grayscale image pairs of hematoxylin were first cropped with Microsoft Office Picture Manager. Fixes were implemented for preprocessed images to be on the first page. Figure has been compressed.   A B 127     Figure 3.25   Classes of preprocessed image tested with Imreg.m with fixes.   Various combinations of images with or without preprocessing were tested (Figure 3.25). Average size of image registration output files were 136 MB (815 MB ÷ 6 images = 136 MB) for input images without preprocessing, 108 MB ([661 MB + 635 MB] ÷ 12 images = 108 MB) for single preprocessed input image, and 49 MB (296 MB ÷ 6 images = 49 MB) for both preprocessed input images (Table 3.12). Thus, image preprocessing by image cropping can dramatically reduce file size of registered images for more effective file storage.  Table 3.12  File sizes of preprocessed image classes. Calculations were performed on rounded file sizes.    Unique ID Size  (MB) Size on Disk  (MB)  Number of images  Average size  per image (MB)  No preprocessing  815  815  6  ~136  Single preprocessed  (fixed)  661  662  12  ~108  Single preprocessed (moving)  635  635  Both preprocessed  296  297  6  ~49   128  Average time elapsed of image registration was calculated by averaging the time of duplicates and averaging for each image pair group. Average time elapsed for image required approximately 224 seconds for preprocessed image, 137 seconds for single preprocessed image, and 97 seconds for both preprocessed images (Table 3.13). Thus, image preprocessing with image cropping reduces running time and may improve software performance in regards to efficiency.  Interestingly, for single preprocessed images, it was observed that when the preprocessed image was assigned as the fixed image, the running time was faster (104 seconds compared to 169 seconds) (Table 3.14). Image preprocessing with image cropping improved image registration especially when both images were preprocessed as observed in 05L14_4 and 05L4_1 (Figure 3.26). However, it was difficult to assess if image registration was accurate if both moving and fixed image are cropped to an exceedingly small size as in 05L4_4, illustrating the requirement for sufficient identifiable landmark to provide context when manually assessing image registration. An interesting approach would be to perform image registration on scaled versions of images and apply this to geometric transformation to the larger resolution images [188]. Images were able to be reproducibly aligned when one image was cropped and one was not preprocessed. However, there was some dependency on which image pair was assigned as the fixed or moving image, as observed in 05L14_4 where T&B Cell was preprocessed (Figure 3.26). Thus, preprocessing of images with image cropping may present as a potential solution to improve image quality, accuracy, and efficiency of image registration for some images where skewing exists in imaging. A priori knowledge of the object in space may be required in order to determine where to perform image cropping for landmark based image registration. 129   Table 3.13  Running time of Imreg.m with fixes on images without preprocessing and with preprocessing.    Fixed (+/-preprocessing) Moving (+/- preprocessing) ID_ROI Time of image registration (seconds) Time of reciprocal image registration (seconds) Average time of duplicates (seconds) Average time of samples (seconds) T&B Cell  - preprocessing ARPDL1high  - preprocessing 05L14_4 194.8521 219.5953 207.2237 224.3262 05L4_1 238.6409 231.3881 235.0145 05L4_4 221.3210 240.1596 230.7403 T&B Cell  - preprocessing ARPDL1high  + preprocessing 05L14_4 179.9735 114.4780 147.2258 136.7976 05L4_1 228.2138 188.1644 208.1891 05L4_4 117.3220 31.0050 74.1635 T&B Cell + preprocessing ARPDL1high - preprocessing 05L14_4 115.8091 176.0541 145.9316 05L4_1 146.4185 200.6933 173.5559 05L4_4 30.8595 112.5801 71.7198 T&B Cell  + preprocessing ARPDL1high  + preprocessing 05L14_4 110.0066 112.6197 111.3132 97.1436 05L4_1 153.1980 162.3976 157.7978 05L4_4 22.0522 22.5877 22.3200      Table 3.14  Single image preprocessed running time. Groups classified into whether moving or fixed image is preprocessed.  Fixed (+/-preprocessing) Moving (+/- preprocessing) ID_ROI Time of image registration, where Fixed = T&B Cell (seconds) Time of image registration where Fixed = ARPDL1high (seconds) Average time of duplicates (seconds) Average time of samples (seconds) - preprocessing + preprocessing 05L14_4 179.9735 176.0541 178.0138 169.1395 05L4_1 228.2138 200.6933 214.4535 05L4_4 117.3220 112.5801 114.9511 + preprocessing - preprocessing 05L14_4 115.8091 114.4780 115.1436 104.4558 05L4_1 146.4185 188.1644 167.2915 05L4_4 30.8595 31.0050 30.9323  130     Figure 3.26  Image registration of preprocessed image with Imreg.m with fixes.  Figure has been compressed.  05L14_4              05L4_1                        05L4_4 131  Chapter 4: Conclusions  4.1 General conclusions and significance  This work aimed to assess the workflow optimization of a hyperspectral image analysis system to study tumor-immune cell spatial organizations within lung tumor microenvironments developed by Dr. Calum MacAulay and Dr. Martial Guillaud’s research team at the BCCRC. The specific hypothesis of this work is that the workflow of the BCCRC hyperspectral image analysis system will facilitate the determination of spatial relationships of cell populations within the tumor microenvironment. Here, this work has demonstrated that the BCCRC hyperspectral image analysis system is a platform that is valuable for quantitatively and spatially profiling tumor heterogeneity at the cell-to-cell level and distribution at a multi-plane level, although further optimization of this method is required for application in non-clinical or clinical settings. Technical factors affecting workflow optimization of this system were identified at different stages of digital image analysis, including in image acquisition, segmentation, specialized equipment, staining reproducibility, and sampling strategy. Potential solutions were suggested that could improve the workflow of this method for non-clinical or clinical settings (Figure 4.1 and Table 4.1 to Table 4.3). For multiplex IHC, selection of stains will be an important factor to select (Figure 4.1A). Experiments to verify whether different stains would yield different results for image registration will be needed. The ROI selected will be an important factor to address selection bias for future studies implementing this method. Importantly, a study determining what the optimal number of ROI needed to represent the whole tissue section of lung tumors or for other solid tumors will be essential for future studies using this in situ multiplex detection 132  system. Large landmarks will be required in ROI selection for studies deciding to use image registration (Figure 4.1B, first decision). The testing of fiducial markers in tissue could be used to determine whether this is an effective method to study areas with few large landmarks. Regarding segmentation, the system currently relies on selection criteria of either ROI with high density of cell nuclei or low density of cell nuclei (Figure 4.1B, second decision). Additional preprocessing methods such as image smoothing or image sharpening applied to the image can be investigated to determine whether further optimizations can be made to segment the whole field of view with variable densities of cell nuclei (Figure 4.1C). This study revealed that image preprocessing by image cropping is a potential solution to eliminate tissue artifacts (Figure 4.1B, third decision). The elimination of tissue artifacts was able to improve image registration (Figure 4.1D). Additional steps needed for future studies include optimal size for image cropping and investigation of a priori knowledge required for image cropping may be required. Integration of other algorithms for tissue artifact recognition could be explored. Other less noticeable tissue artifacts, such as tissue stretching and other non rigid tissue deformation, would require experimental testing with other image preprocessing, such as auxiliary geometric transformations to virtually correct deformed tissue. Regarding file storage, future experiments could include determining what image compression method is optimal to best preserve image data while conserving computer memory. Furthermore, acquiring a larger network may be considered, though optimizing for image compression may be a more sustainable method (Figure 4.1B, third decision). Future studies may consider using fewer wavelengths to perform spectral unmixing and to develop a multispectral image acquisition system implementing these fewer wavelengths to improve efficiency of this image analysis system.   133                                         Figure 4.1     Flowchart of workflow optimization of the BCCRC hyperspectral image analysis system. Oval indicates start of workflow. Diamond indicates decision, and darkly colored diamond indicates technical factors identified in this study affecting workflow optimization. Rounded rectangle indicates finish. Arrow indicates connector. Parallelogram indicates output, data retrieved. Rectangle indicates process.     134       Workflow feature Approaches by other investigators / manufacturers Cell images  (human/mouse/rat/other/unknown) References Approach by BCCRC hyperspectral image analysis system Software performance Larger step size with low-power objective or low resolution images for faster performance of image acquisition time  Heart mitral (h)  [129]    Reduce image file size with image cropping for faster image registration performance (T)  Perform spectral unmixing with six illumination wavelengths rather than sixteen illumination wavelengths for faster performance (T)  Develop image acquisition system to image six illumination wavelengths rather than sixteen illumination wavelengths to reduce image acquisition time (FD)  Consider using a GPU for image acquisition for faster performance (FD)  Reimage slides that are manually detected to be blurry or out of focus (T)    Automated image acquisition system that can be imaged in batches to increase image acquisition time Lung adenocarcinoma (h) [127] Use of Intel HD Graphics 2500 Display Card in workstation  Non small cell lung cancer (u) [197] Suggests use of multi-core processors or GPU to increase performance with multi-threading ability Lung cancer (u) Colon cancer (u) [188] Software with a web interface, such as Cytomine, that uses simultaneous multi-user platform  to enable collaboration of image datasets (analogous to multi-user interface such as Google Docs) Lung cancer (u) [135, 136] Blur detection and sharpness quantification using machine learning Clear cell renal  cell carcinoma (h) Prostate cancer (h) Hippocampal sclerosis (h) [198] inForm Tissue Finder (PerkinElmer) performs automated segmentation with proprietary pattern recognition algorithms to increase performance  Images (hundreds or thousands) can be segmented through batch processing and analyzed 20 (for 32bit) or 400 (for 64 bit) for maximum number of images per training set  Unlimited number of images per batch set  Training processing speed takes seconds per image in training set while batch processing takes minutes for multiple images - [123] Definiens Tissue Studio conducts proprietary updates to processing workflow   Automatic proprietary method for image registration of adjacent sections  Identifies factors including image size, tissue amount on slide, nature of tissue (density of nuclei), effects performance for processing  Use lower resolution images for increased performance of ROI detection  Enables batch processing - [114, 124]  Table 4.1  Proposed solutions for the BCCRC hyperspectral image analysis system for performance and specialized equipment. (Continued on page 135). 135   Workflow feature Approaches by other investigators / manufacturers Cell images  (human/mouse/rat/other/unknown) References Approach by BCCRC hyperspectral image analysis system File size Lossless image compression is not required, lossy and lossless image compression produced similar image analysis results  Helicobacter pylori gastritis (h) [121] Image acquisition software can image ROI in adjacent sections with reduced file size relative to WSI (T)  Perform image preprocessing (image cropping) to further reduce file size (T)  Larger server needed to accommodate large file size of images (FD)  Determine compatible image compression to reduce file size and maintain image quality for this image analysis platform (FD)  Determine if less than five ROI is representative of tumor to reduce number of images needed to acquire for study (FD) Image compression with JPEG2000 of ROI from WSI without effecting interpretation of pathologist  Breast cancer (u) [120] HALO Link (Indica labs) enables remote image management and  cloud server access to various electronic devices (computer, tablet, smartphone) for efficient access - [119] inForm is compatable with monochrome or color images including lossy and lossless compression images (JPEG, single-layer TIFF, BMMP, PNG), and other PerkinElmer Multispectral Image file formats (.im3, .im4) - [123] Definiens Tissue Studio is compatible with various image formats from different image acquisition systems (including, Aperio ScanScope, Leica SCN, Nikon, Olympus, Roche Ventana, TissueGnostics, Unic, Zeiss)   Identifies that size, resolution of images effects hard disk requirements - [124]  Use of lossy image compression (such as JPEG) file formats for images Non-small cell lung cancer (h) [122] Perform image registration on scaled versions of images and apply to larger resolution image Lung cancer (u) Colon cancer (u) [188] Selection bias Hotspot and cell cluster detection  Pancreatic neuroendocrine tumor (u) [150] Selection bias – to be determined, identify method to sample tumor areas to limit bias, such as automatic ROI selection (FD)  Image registration to assess patterns in 3D space (T) Use of tissue microarray Non-small cell lung cancer (h) [191] Blinded pathologist selected 3 fields of views by qualitative observation of greatest cellular areas  Non-small cell lung cancer (h) [122] Semi-automatic ROI detection by using training set Lung cancer (m) [135, 167] HALO Classifier (Indica Labs) enables automatic tissue selection to train for selections ROI of interest  - [115] Definiens enables creation of ROI from training dataset in addition to manual ROI selection - [124]  FD: Future direction; T: Tested         136  Table 4.2  Proposed solutions for the BCCRC hyperspectral image analysis system for cell-to-cell spatial analysis. (Continued on page 137)  Workflow feature Approaches by other investigators / manufacturers Cell images  (human/mouse/rat/other/unknown) References Approaches by BCCRC hyperspectral image analysis system Cell-to-cell spatial analysis HALO (Indica Labs) is a commercial software capable of spatial analyses, including proximity, neighborhood, infiltration analysis (algorithms proprietary)  -  [115] Asterics performs higher resolution quantitative characterization of microhabitat of cell (proportion measurements) (T)  Cell-to-cell spatial analysis can be performed with over two markers, including hematoxylin (T) Investigator indicates that a number of manufacturers only work with 2 antibodies, including HALO (Indica Labs) - [181] Quantification of cytotoxic and memory T cell densities with the immunoscore significantly correlate with patient survival Colorectal cancer (h) Melanoma (h) [110]  Proximity of PD1 and PDL1 cells significantly correlates with response or progression with pembrolizumab (used HALO proximity analysis, Indica Lab)  Melanoma (h) [99] Segmentation For single pixels use spatial location related to two-dimensional Gaussian mixture model (GMM), with a priori information of appearance of object  Seminavis robusta (o) [138] Modify cluster A and B in DUnit to refine thresholds and further segment large clusters (T)  Segmentation of either low cluster or high cluster areas based on framing research study (T)  Have multiple users to increase thoroughput of segmentation (T)  Avoid areas with too dense cell nuclei for image analysis (T)  Determine method to automate segmentation for faster performance (FD)  Deep learning and deformation model to segment clusters  Cervical cancer pap smear (h) [139] Extract boundary of nucleus using multiscale LoG filter and adaptive scale selection with concavity measurement applied, three classes of split-point candidates of binary nuclei image to search for optimal split-point pair, to determine nuclear occlusion  Kidney renal clear cell carcinoma (h) [142] Segmentation of touching nuclei by approximate convex decomposition, lines of Sight (LoS) concept with local adaptive pre-processing  Embryo (m) Breast cancer spheroid (h) Pancreatic cancer spheroid (h) [143] Bayesian method to separate touching nuclei  Breast cancer (h) [144] Color space, mean shift clustering, illumination adjustment, and nucleus mark watershed operation (NMWO) for segmenting occluded cells  Normal peripheral blood (h) Acute myeloid leukemia  blood (h) [145] Deep learning using multiscale convolutional network (MSCN) to segment touching nuclei  Cervical cancer (h) [140] Deep convolutional neural network and selection-based sparse shape model and local repulsive deformable model to segment nuclei  Brain tumor (u) Pancreatic neuroendocrine tumor (u) Breast cancer (u) [141] Algorithm for immune cell conglomerates  Colorectal cancer (h) Liver metastases (h) Colon mucosa (h) Liver (h) [149] Automatic segmentation of touching nuclei with sparse reconstruction  with trivial templates and stacked denoising autoencoders with a  MATLAB workstation Lung cancer (u) Brain cancer (u) [146] Clumps of nuclei segmented by watershedding Glioblastoma multiforme (u) Lower grade glioma (u) [147] inForm (PerkinElmers) is trainable for segmentation  - [123]                137                                 Workflow feature Approaches by other investigators / manufacturers Cell images (human/mouse/rat/other/unknown) References Approaches by BCCRC hyperspectral image analysis system Threshold Deep learning approach – identify image patches for training positive or negative class   Determine a threshold of image with color deconvolution to identify negative class Breast tumor (u) [152, 153] Gallery as supplemental method to verify individual cells for classification (T)  Establish multiple grey level thresholds to quantify strength of staining intensity (T)  Have multiple observers to account for interobserver variability in establishing thresholds (T) inForm (PerkinElmers) performs various scoring methods (% positivity, 0/1+/2+/3+, co-localization and more) - [158] Tissue Studio (Definiens) sets thresholds for low, medium, high intensity cells  Uses HSD models rather than RGB space to increase performance for threshold application   Tissue-background separation for brightfield or immunoflurescent images based on autothreshold or manual threshold selection (homogeneity, brightness, tissue min size) - [124] FD: Future direction; T: Tested               138   Table 4.3  Proposed solutions for the BCCRC hyperspectral image analysis system for image registration. (Continued on page 139).  Workflow feature Approaches by other investigators / manufacturers Cell Images  (human/mouse/rat/other/unknown) References Approach by BCCRC hyperspectral image analysis system Image matching Perform image registration of ROI onto WSI to confirm location Colon (u) Sarcoma (u) [173] Image cropping improves image registration with MATLAB based image registration software (T)  Perform image cropping for areas misaligned from image acquisition and further ensure image quality (T)  Determined that image registration can be non-reproducible depending on which adjacent section is used as the fixed image with MATLAB based image registration program (T)  Avoid selecting areas for image registration with weak local landmarks (T)  Correct nonrigid deformation in imaged sections with image preprocessing (FD)   Test using fiducial markers to create common landmarks or detect areas with large landmarks for ROI image acquisition (FD)    Registration with sequential labeling and erasing and identified that poor histological structures effected image registration quality Colorectal cancer (h) Tonsil (u) Brain (u) Glioma (u) [87] ROI detection method for when adjacent sections do not match perfectly using stereology approaches Breast mammary gland (h) [189] Tumor mask to distinguish inside or outside tumor and correlate with adjacent section Breast cancer (h) [181] Correct cutting distortions using non-rigid transformation Kidney (m/r) [174] Overlay segmentation from more than one image  Automated sectioning machine to create consistent, high quality section s  Staining of H&E every three to four sections to observe nuclei morphology at different level to improve image registration correlation  Interactive interface to confirm image registration by viewing different transparency and opacity of images  Registered areas can be cropped to smaller region of interest Lung adenocarcinoma (h) [127] Extract common information between different slides, including preprocessing grey scale images with K-means classification to determine a threshold Lung cancer (u) Colon cancer (u) [188] Landmark detection Danio rerio [135] Fiducial markers in skull for CT scans to obtain ground truth transformation Skull (h) [168] Serial Section Analysis by HALO (Indica Labs) creates classifier in a reference slide that can be applied to serial sections to enable multimodal stain image registration - [115] Zoomable interface for ROI detection (including OmnyxTM Pathologist Workstation, Philips Pathology Viewing System) - [190]         139     Workflow feature Approaches by other investigators / manufacturers Cell Images (human/mouse/rat/other/unknown) References Approach by BCCRC hyperspectral image analysis system Tissue artifacts Tissue fold detection by using a soft and hard threshold and including neighborhood criteria Kidney clear cell carcinoma (h) Ovarian serous adenocarcinoma (h)  [199] Perform image preprocesing (image cropping) to remove areas with tissue artifacts (T)  Avoid imaging areas with tissue artifacts (T)   Develop method such as an algorithm to detect and exclude tissue artifact areas in image (FD)  Correction of tissue folds by using k-means clustering Aorta (m)  [194] Tissue fold detection by color-based method and fixed threshold - Breast (h) Liver (h) Esophagus (h) [195, 196] Tissue Studio (Definiens) has feature to modify unprocessed images, such as manual marking to correct for artifacts  - [124] Rescanning of images containing artifacts Lung adenocarcinoma (h) [127]  Multimodal stains Nonrigid image registration of differently stained adjacent sections using multiresolution block matching and two-dimensional unsupervised content classification to improve structure similarity Liver cirrhosis (h)  Ovine intervertebral disc (o) [182, 183] Avoid using different stains to perform image registration (T)  Use same stains to perform image registration as template of geometric transformation for image registration (FD) Multimodal method of image registration with MRI and histopathology  Prostate cancer (h) [177] Visualize tissue morphology and features without using dyes by Fourier transform infrared (FT-IR) spectroscopic imaging  Normal breast (h) Non-malignant breast (h) Pre-cancer breast (h) Breast cancer (h)  [184] Save transformation matrix from hematoxylin and apply it to a mask of protein expression variability in adjacent tissue sections to account for biological variability  Breast cancer (h) [181] Automated method to normalize stain by mitigating batch effects (referred to as intensity centering and histogram equalization) Lung cancer (h) [185] Rescanning of images containing weak staining  Control slides for calibration of heterogenous staining Lung adenocarcinoma (h) [127] Normalize stains with sparse autoencoders (StaNoSA) Breast (u) Gastrointestinal (u) [186]  FD: Future direction; T: Tested   140  4.2 Future directions  The work presented in this thesis has been conducted on a small cohort. Technical factors affecting workflow optimization of this method were identified, which may be valuable for future biological studies employing this method. For future studies, it would be beneficial to use a larger cohort for analysis with clinical-pathological features or immunotherapy information to verify if MNG of particular cell types could present as a valuable prognostic biomarker relative to histopathology TNM staging. Additional geographical information of tumor tissue architecture, including whether digitally sampled sites are from the invasive margin or tumor core would be valuable to predict prognosis, as attained by the Immunoscore. Future studies might compare the measurements of cell counts and neighborhood groups, including economical costs of this software to commercial software that perform spatial analyses. In addition, the development of platforms for multi-omic information is of value to analyze spatial organization, such as images of ISH multiplexed with IHC or correlating with other multimodal images, such as CT to study tumor aggressiveness [41]. One must take into consideration that a single section is typically analyzed in a clinical setting, which may not be representative of the entire tissue and an evaluation of this representation may be useful for this hyperspectral image analysis system. Studies on determining what cohort of lung cancer patients will benefit most from this type of analysis will also be valuable.     141  Bibliography  1. Canadian Cancer Society’s Advisory Committee on Cancer Statistics. Canadian Cancer Statistics 2017. 2017. 2. Uramoto, H. and F. Tanaka, Recurrence after surgery in patients with NSCLC. Transl Lung Cancer Res, 2014. 3(4): p. 242-9. 3. Goodgame, B., et al., A clinical model to estimate recurrence risk in resected stage I non-small cell lung cancer. Am J Clin Oncol, 2008. 31(1): p. 22-8. 4. Riessk, J., Shifting paradigms in non-small cell lung cancer: an evolving therapeutic landscape. Am J Manag Care, 2013. 19(19 Suppl): p. s390-7. 5. Goldstraw, P., et al., The IASLC Lung Cancer Staging Project: proposals for the revision of the TNM stage groupings in the forthcoming (seventh) edition of the TNM Classification of malignant tumours. J Thorac Oncol, 2007. 2(8): p. 706-14. 6. Mulshine, J.L., Screening for lung cancer: in pursuit of pre-metastatic disease. Nat Rev Cancer, 2003. 3(1): p. 65-73. 7. Harders, S.W., S. Balyasnikowa, and B.M. Fischer, Functional imaging in lung cancer. Clin Physiol Funct Imaging, 2014. 34(5): p. 340-55. 8. Sharma, D., T.G. Newman, and W.S. Aronow, Lung cancer screening: history, current perspectives, and future directions. Arch Med Sci, 2015. 11(5): p. 1033-43. 9. Petersen, I., The morphological and molecular diagnosis of lung cancer. Dtsch Arztebl Int, 2011. 108(31-32): p. 525-31. 10. Leung, A.W., et al., Synthetic lethality in lung cancer and translation to clinical therapies. Mol Cancer, 2016. 15(1): p. 61. 11. Pikor, L.A., et al., Genetic alterations defining NSCLC subtypes and their therapeutic implications. Lung Cancer, 2013. 82(2): p. 179-89. 12. Marshall, E.A., et al., Emerging roles of T helper 17 and regulatory T cells in lung cancer progression and metastasis. Mol Cancer, 2016. 15(1): p. 67. 13. Travis, W.D., et al., The 2015 World Health Organization Classification of Lung Tumors: Impact of Genetic, Clinical and Radiologic Advances Since the 2004 Classification. J Thorac Oncol, 2015. 10(9): p. 1243-60. 14. Chen, Z., et al., Non-small-cell lung cancers: a heterogeneous set of diseases. Nat Rev Cancer, 2014. 14(8): p. 535-46. 15. Eguchi, T., et al., The new IASLC-ATS-ERS lung adenocarcinoma classification: what the surgeon should know. Semin Thorac Cardiovasc Surg, 2014. 26(3): p. 210-22. 16. Russell, P.A., et al., Does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society international multidisciplinary lung adenocarcinoma classification. J Thorac Oncol, 2011. 6(9): p. 1496-504. 17. Korpanty, G.J., et al., Biomarkers That Currently Affect Clinical Practice in Lung Cancer: EGFR, ALK, MET, ROS-1, and KRAS. Front Oncol, 2014. 4: p. 204. 18. Mirsadraee, S., et al., The 7th lung cancer TNM classification and staging system: Review of the changes and implications. World J Radiol, 2012. 4(4): p. 128-34. 142  19. Rami-Porta, R., et al., Lung cancer - major changes in the American Joint Committee on Cancer eighth edition cancer staging manual. CA Cancer J Clin, 2017. 67(2): p. 138-155. 20. Sellke, F.W.D.N., Pedro J; Swanson, Scott J, Sabiston and Spencer Surgery of the Chest, Ninth Edition. 2015, Elsevier: Philadelphia, PA  21. Purandare, N.C. and V. Rangarajan, Imaging of lung cancer: Implications on staging and management. Indian J Radiol Imaging, 2015. 25(2): p. 109-20. 22. National Lung Screening Trial Research, T., et al., Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med, 2011. 365(5): p. 395-409. 23. Pastorino, U. and N. Sverzellati, Lung cancer: CT screening for lung cancer--do we have an answer? Nat Rev Clin Oncol, 2013. 10(12): p. 672-3. 24. Abbott, A., Researchers pin down risks of low-dose radiation. Nature, 2015. 523(7558): p. 17-8. 25. McWilliams, A., et al., Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med, 2013. 369(10): p. 910-9. 26. Aberle, D.R., et al., Results of the two incidence screenings in the National Lung Screening Trial. N Engl J Med, 2013. 369(10): p. 920-31. 27. Tammemagi, M.C., et al., Selection criteria for lung-cancer screening. N Engl J Med, 2013. 368(8): p. 728-36. 28. Field, J.K. and S.W. Duffy, Lung cancer CT screening: is annual screening necessary? Lancet Oncol, 2016. 17(5): p. 543-4. 29. Field, J.K., et al., UK Lung Cancer RCT Pilot Screening Trial: baseline findings from the screening arm provide evidence for the potential implementation of lung cancer screening. Thorax, 2016. 71(2): p. 161-70. 30. Figueiredo, V.R., et al., Current status and clinical applicability of endobronchial ultrasound-guided transbronchial needle aspiration. J Bras Pneumol, 2013. 39(2): p. 226-37. 31. Gelberg, J., S. Grondin, and A. Tremblay, Mediastinal staging for lung cancer. Can Respir J, 2014. 21(3): p. 159-61. 32. Ost, D.E. and M.K. Gould, Decision making in patients with pulmonary nodules. Am J Respir Crit Care Med, 2012. 185(4): p. 363-72. 33. Leong, S.P., et al., Unique patterns of metastases in common and rare types of malignancy. J Surg Oncol, 2011. 103(6): p. 607-14. 34. Chandrasekaran, S. and M.R. King, Microenvironment of tumor-draining lymph nodes: opportunities for liposome-based targeted therapy. Int J Mol Sci, 2014. 15(11): p. 20209-39. 35. Rami-Porta, R., J.J. Crowley, and P. Goldstraw, The revised TNM staging system for lung cancer. Ann Thorac Cardiovasc Surg, 2009. 15(1): p. 4-9. 36. Cancer, A.J.C.o., AJCC Cancer Staging Manual. 2002: Springer New York. 37. Jimenez-Bonilla, J.F., et al., The Role of PET/CT Molecular Imaging in the Diagnosis of Recurrence and Surveillance of Patients Treated for Non-Small Cell Lung Cancer. Diagnostics (Basel), 2016. 6(4). 38. Cancer Care Ontario. Lung Cancer Diagnosis Pathway: Disease Pathway Management Secretariat Version 2012.2. [cited December 2017]; Available from: https://www.cancercare.on.ca/common/pages/UserFile.aspx?fileId=124606  143  39. Moles Lopez, X., Debeir, O., Salmon, I. & Decaestecker, C, Whole slide imaging and analysis for biomarker evaluation in digital pathology. In Microsc. Adv. Sci. Res. Educ, (Formatex Research Center, 2014): p. 776–787. 40. Hayat, M.A., Microscopy, Immunohistochemistry, and Antigen Retrieval Methods for Light and Electron Microscopy. 2002, New York Kluwer Academic/Plenum Publishers. 41. Mansfield, J.R., Phenotyping Multiple Subsets of Immune Cells In Situ in FFPE Tissue Sections: An Overview of Methodologies. Methods Mol Biol, 2017. 1546: p. 75-99. 42. Schallenberg, S., S. Merkelbach-Bruse, and R. Buettner, Lung cancer as a paradigm for precision oncology in solid tumours. Virchows Arch, 2017. 43. U.S. Food & Drug Administration. Premarket Approval (PMA). 2017. [cited December 2017]; Available from: https://www.accessdata.fda.gov/scrIpts/cdrh/cfdocs/cfpma/pma.cfm?id=P110012  44. Dagogo-Jack, I. and A.T. Shaw, Screening for ALK Rearrangements in Lung Cancer: Time for a New Generation of Diagnostics? Oncologist, 2016. 21(6): p. 662-3. 45. Tsao, M., Hirsch, FR, and Yatabe, Y, IASLC Atlas of ALK Testing in Lung Cancer. 2013, International Association for the Study of Lung Cancer,: Colorado, USA. 46. Paulsen, E.E., et al., Assessing PDL-1 and PD-1 in Non-Small Cell Lung Cancer: A Novel Immunoscore Approach. Clin Lung Cancer, 2017. 18(2): p. 220-233 e8. 47. Rami-Porta, R., H. Asamura, and P. Goldstraw, Predicting the prognosis of lung cancer: the evolution of tumor, node and metastasis in the molecular age-challenges and opportunities. Transl Lung Cancer Res, 2015. 4(4): p. 415-23. 48. Schwartzberg, L., et al., Precision Oncology: Who, How, What, When, and When Not? Am Soc Clin Oncol Educ Book, 2017. 37: p. 160-169. 49. Shao, W., D. Wang, and J. He, The role of gene expression profiling in early-stage non-small cell lung cancer. J Thorac Dis, 2010. 2(2): p. 89-99. 50. Tomczak, K., P. Czerwinska, and M. Wiznerowicz, The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemp Oncol (Pozn), 2015. 19(1A): p. A68-77. 51. Zhang, Y., et al., Differential expression profiles of microRNAs as potential biomarkers for the early diagnosis of lung cancer. Oncol Rep, 2017. 37(6): p. 3543-3553. 52. Chen, H.Y., et al., A five-gene signature and clinical outcome in non-small-cell lung cancer. N Engl J Med, 2007. 356(1): p. 11-20. 53. Beer, D.G., et al., Gene-expression profiles predict survival of patients with lung adenocarcinoma. Nat Med, 2002. 8(8): p. 816-24. 54. Martinez, V.D., et al., Unique somatic and malignant expression patterns implicate PIWI-interacting RNAs in cancer-type specific biology. Sci Rep, 2015. 5: p. 10423. 55. Enfield, K.S., et al., Deregulation of small non-coding RNAs at the DLK1-DIO3 imprinted locus predicts lung cancer patient outcome. Oncotarget, 2016. 7(49): p. 80957-80966. 56. Boutros, P.C., et al., Prognostic gene signatures for non-small-cell lung cancer. Proc Natl Acad Sci U S A, 2009. 106(8): p. 2824-8. 57. Guo, N.L., et al., Confirmation of gene expression-based prediction of survival in non-small cell lung cancer. Clin Cancer Res, 2008. 14(24): p. 8213-20. 58. Navab, R., et al., Prognostic gene-expression signature of carcinoma-associated fibroblasts in non-small cell lung cancer. Proc Natl Acad Sci U S A, 2011. 108(17): p. 7160-5. 144  59. Newman, A.M., et al., Robust enumeration of cell subsets from tissue expression profiles. Nat Methods, 2015. 12(5): p. 453-7. 60. Fridman, W.H., et al., The immune contexture in cancer prognosis and treatment. Nat Rev Clin Oncol, 2017. 61. Dunstan, R.W., et al., The use of immunohistochemistry for biomarker assessment--can it compete with other technologies? Toxicol Pathol, 2011. 39(6): p. 988-1002. 62. Satija, R., et al., Spatial reconstruction of single-cell gene expression data. Nat Biotechnol, 2015. 33(5): p. 495-502. 63. Nawaz, S. and Y. Yuan, Computational pathology: Exploring the spatial dimension of tumor ecology. Cancer Lett, 2016. 380(1): p. 296-303. 64. Giraldo, N.A., et al., The immune contexture of primary and metastatic human tumours. Curr Opin Immunol, 2014. 27: p. 8-15. 65. Wood, S.L., et al., The role of the tumor-microenvironment in lung cancer-metastasis and its relationship to potential therapeutic targets. Cancer Treat Rev, 2014. 40(4): p. 558-66. 66. Halvorsen, E.C., S.M. Mahmoud, and K.L. Bennewith, Emerging roles of regulatory T cells in tumour progression and metastasis. Cancer Metastasis Rev, 2014. 33(4): p. 1025-41. 67. Hanahan, D. and R.A. Weinberg, Hallmarks of cancer: the next generation. Cell, 2011. 144(5): p. 646-74. 68. Hoos, A., Development of immuno-oncology drugs - from CTLA4 to PD1 to the next generations. Nat Rev Drug Discov, 2016. 15(4): p. 235-47. 69. Kelly, R.J., J.L. Gulley, and G. Giaccone, Targeting the immune system in non-small-cell lung cancer: bridging the gap between promising concept and therapeutic reality. Clin Lung Cancer, 2010. 11(4): p. 228-37. 70. Matthay, R.A., et al., Intratumoral Bacillus Calmette-Guerin immunotherapy prior to surgery for carcinoma of the lung: results of a prospective randomized trial. Cancer Res, 1986. 46(11): p. 5963-8. 71. Al-Moundhri, M., M. O'Brien, and B.E. Souberbielle, Immunotherapy in lung cancer. Br J Cancer, 1998. 78(3): p. 282-8. 72. Boutros, C., et al., Safety profiles of anti-CTLA-4 and anti-PD-1 antibodies alone and in combination. Nat Rev Clin Oncol, 2016. 13(8): p. 473-86. 73. Reck, M., et al., Pembrolizumab versus Chemotherapy for PD-L1-Positive Non-Small-Cell Lung Cancer. N Engl J Med, 2016. 375(19): p. 1823-1833. 74. Borghaei, H., et al., Nivolumab versus Docetaxel in Advanced Nonsquamous Non-Small-Cell Lung Cancer. N Engl J Med, 2015. 373(17): p. 1627-39. 75. Rittmeyer, A., et al., Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial. Lancet, 2017. 389(10066): p. 255-265. 76. Langer, C.J., et al., Carboplatin and pemetrexed with or without pembrolizumab for advanced, non-squamous non-small-cell lung cancer: a randomised, phase 2 cohort of the open-label KEYNOTE-021 study. Lancet Oncol, 2016. 17(11): p. 1497-1508. 77. Rojo, M.G., G. Bueno, and J. Slodkowska, Review of imaging solutions for integrated quantitative immunohistochemistry in the Pathology daily practice. Folia Histochem Cytobiol, 2009. 47(3): p. 349-54. 145  78. Brown, D.L., Practical Stereology Applications for the Pathologist. Vet Pathol, 2017. 54(3): p. 358-368. 79. Chen, J.M., et al., Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review. Tumour Biol, 2017. 39(3): p. 1010428317694550. 80. von Wasielewski, R., et al., Tissue array technology for testing interlaboratory and interobserver reproducibility of immunohistochemical estrogen receptor analysis in a large multicenter trial. Am J Clin Pathol, 2002. 118(5): p. 675-82. 81. Webster, J.D. and R.W. Dunstan, Whole-slide imaging and automated image analysis: considerations and opportunities in the practice of pathology. Vet Pathol, 2014. 51(1): p. 211-23. 82. Tsujikawa, T., et al., Quantitative Multiplex Immunohistochemistry Reveals Myeloid-Inflamed Tumor-Immune Complexity Associated with Poor Prognosis. Cell Rep, 2017. 19(1): p. 203-217. 83. Van Eycke, Y.R., et al., Image processing in digital pathology: an opportunity to solve inter-batch variability of immunohistochemical staining. Sci Rep, 2017. 7: p. 42964. 84. Pantanowitz, L., Digital images and the future of digital pathology. J Pathol Inform, 2010. 1. 85. Bengtsson, E., et al., Computer-aided diagnostics in digital pathology. Cytometry A, 2017. 91(6): p. 551-554. 86. Christensen, G.E., et al., Tracking lung tissue motion and expansion/compression with inverse consistent image registration and spirometry. Med Phys, 2007. 34(6): p. 2155-63. 87. Moles Lopez, X., et al., Registration of whole immunohistochemical slide images: an efficient way to characterize biomarker colocalization. J Am Med Inform Assoc, 2015. 22(1): p. 86-99. 88. Metzger, G.J., et al., Development of multigene expression signature maps at the protein level from digitized immunohistochemistry slides. PLoS One, 2012. 7(3): p. e33520. 89. Mueller, D., D. Vossen, and B. Hulsken, Real-time deformable registration of multi-modal whole slides for digital pathology. Comput Med Imaging Graph, 2011. 35(7-8): p. 542-56. 90. Meyer, J. and G. Pare, Telepathology Impacts and Implementation Challenges: A Scoping Review. Arch Pathol Lab Med, 2015. 139(12): p. 1550-7. 91. Gabril, M.Y. and G.M. Yousef, Informatics for practicing anatomical pathologists: marking a new era in pathology practice. Mod Pathol, 2010. 23(3): p. 349-58. 92. Park, S., et al., The history of pathology informatics: A global perspective. J Pathol Inform, 2013. 4: p. 7. 93. Evans, A.J., et al., Primary frozen section diagnosis by robotic microscopy and virtual slide telepathology: the University Health Network experience. Hum Pathol, 2009. 40(8): p. 1070-81. 94. Pantanowitz, L., et al., Review of the current state of whole slide imaging in pathology. J Pathol Inform, 2011. 2: p. 36. 95. Indu, M., R. Rathy, and M.P. Binu, "Slide less pathology": Fairy tale or reality? J Oral Maxillofac Pathol, 2016. 20(2): p. 284-8. 96. Bhargava, R. and A. Madabhushi, Emerging Themes in Image Informatics and Molecular Analysis for Digital Pathology. Annu Rev Biomed Eng, 2016. 18: p. 387-412. 146  97. Hipp, J., et al., Computer aided diagnostic tools aim to empower rather than replace pathologists: Lessons learned from computational chess. J Pathol Inform, 2011. 2: p. 25. 98. Galon, J., et al., Cancer classification using the Immunoscore: a worldwide task force. J Transl Med, 2012. 10: p. 205. 99. Tumeh, P.C., et al., PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature, 2014. 515(7528): p. 568-71. 100. Remark, R., et al., The non-small cell lung cancer immune contexture. A major determinant of tumor characteristics and patient outcome. Am J Respir Crit Care Med, 2015. 191(4): p. 377-90. 101. Haratani, K., et al., Tumor Immune Microenvironment and Nivolumab Efficacy in EGFR Mutation-Positive Non-Small Cell Lung Cancer Based on T790M Status after Disease Progression During EGFR-TKI Treatment. Ann Oncol, 2017. 102. Kaplan, K.J. Omnyx Integrated Digital Pathology Solution with New High-Volume Scanner Receives Health Canada License. 2014. [cited August 2017]; Available from: http://www.omnyx.com/press-releases/gehealthcares-omnyxintegrated-digital-pathology-system-awarded-healthcanadaclearance-for-routine-diagnostic-use/4717. 103. Tetu, B. and A. Evans, Canadian licensure for the use of digital pathology for routine diagnoses: one more step toward a new era of pathology practice without borders. Arch Pathol Lab Med, 2014. 138(3): p. 302-4. 104. U. S. Department of Health and Human Services. FDA News Release: FDA allows marketing of first whole slide imaging system for digital pathology. 2017. [cited December 2017]; Available from: https://www.fda.gov/newsevents/newsroom/pressannouncements/ucm552742.htm  105. General Electric Healthcare. Omnyx™ Integrated Digital Pathology system achieves CE Mark in the European Union [press release]. 2013. [cited August 2017]; Available from: http://www.omnyx.com/press-releases/omnyx-integrated-digital-pathology-system-achieves-ce-mark-in-the-european-union/4662/13  106. Philips Digital Pathology Solution Receives CE Mark for Diagnosis to Help Advance Oncology Diagnosis: CE mark includes diagnosis for routine pathology. 2014. 107. Hanna, M.G., L. Pantanowitz, and A.J. Evans, Overview of contemporary guidelines in digital pathology: what is available in 2015 and what still needs to be addressed? J Clin Pathol, 2015. 68(7): p. 499-505. 108. Nishino, M., et al., Monitoring immune-checkpoint blockade: response evaluation and biomarker development. Nat Rev Clin Oncol, 2017. 109. Galon, J., et al., Towards the introduction of the 'Immunoscore' in the classification of malignant tumours. J Pathol, 2014. 232(2): p. 199-209. 110. Galon, J., et al., Immunoscore and Immunoprofiling in cancer: an update from the melanoma and immunotherapy bridge 2015. J Transl Med, 2016. 14: p. 273. 111. Kothari, S., et al., Pathology imaging informatics for quantitative analysis of whole-slide images. J Am Med Inform Assoc, 2013. 20(6): p. 1099-108. 112. PerkinElmer. Image Analysis Software. 2017. [cited December 2017]; Available from: http://www.perkinelmer.com/category/image-analysis-software  113. Definiens. Research. 2017. [cited December 2017]; Available from: http://www.definiens.com/solutions/research  147  114. Definiens. Definiens Doubles the Speed of Tissue Analysis with Tissue Studio® 4.1. 2015. [cited December 2017]; Available from: http://www.definiens.com/about-us/press/press-releases/definiens-doubles-the-speed-of-tissue-analysis-with-tissue-studio-4-1. 115. Indica Labs. Halo Platform. 2015. [cited December 2017]; Available from: http://www.indicalab.com/#section-products. 116. Khojasteh, M., et al., A framework for quantitative assessment of Ki67 distribution in preneoplastic bronchial epithelial lesions. Anal Quant Cytol Histol, 2012. 34(3): p. 120-38. 117. Dubrowski, P., et al. Automated tissue m-FISH analysis workstation for identification of clonally related cells. 2008. Proceedings Volume 6859, Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues VI; 68591T  118. Pirici, D., et al., Antibody elution method for multiple immunohistochemistry on primary antibodies raised in the same species and of the same subtype. J Histochem Cytochem, 2009. 57(6): p. 567-75. 119. Indica Labs. HALO Link. 2015. [cited December 2017]; Available from: http://www.indicalab.com/halolink/. 120. Krupinski, E.A., et al., Compressing pathology whole-slide images using a human and model observer evaluation. J Pathol Inform, 2012. 3: p. 17. 121. Kalinski, T., et al., Lossless compression of JPEG2000 whole slide images is not required for diagnostic virtual microscopy. Am J Clin Pathol, 2011. 136(6): p. 889-95. 122. McDermott, S.P., et al., Quantitative assessment of cell block cellularity and correlation with molecular testing adequacy in lung cancer. Journal of the American Society of Cytopathology, 2016 5: p. 196-202. 123. PerkinElmer, inForm Advanced Image Analysis Software – For Accurately Quantifying Biomarkers in Tissue Sections. 2012. 124. Definiens. Definiens Tissue Studio® 4.2 User Guide. 2015. [cited December 2017]; Available from: http://uacc.arizona.edu/sites/default/files/tissue_studio_4.2_user_guide.pdf. 125. Avanaki, A.R.N., et al., An observer model for quantifying panning artifacts in digital pathology. Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment, 2017. 10136. 126. Campanella, G., et al., Towards Machine Learned Quality Control: A Benchmark for Sharpness Quantification in Digital Pathology. Computerized Medical Imaging and Graphics, 2017. 127. Yagi, Y., et al., Development of a database system and image viewer to assist in the correlation of histopathologic features and digital image analysis with clinical and molecular genetic information. Pathol Int, 2016. 66(2): p. 63-74. 128. Ameisen, D., et al., Towards better digital pathology workflows: programming libraries for high-speed sharpness assessment of Whole Slide Images. Diagn Pathol, 2014. 9 p. Suppl 1 S3. 129. Schultz, R.A., et al., Hyperspectral imaging: a novel approach for microscopic analysis. Cytometry, 2001. 43(4): p. 239-47. 130. Panasyuk, S.V., et al., Medical hyperspectral imaging to facilitate residual tumor identification during surgery. Cancer Biol Ther, 2007. 6(3): p. 439-46. 148  131. Kuruvilla, J., et al., A Review on Image Processing and Image Segmentation. Proceedings of 2016 International Conference on Data Mining and Advanced Computing (Sapience), 2016: p. 198-203. 132. Gurcan, M.N., et al., Histopathological image analysis: a review. IEEE Rev Biomed Eng, 2009. 2: p. 147-71. 133. Zhang, C., C. Sun, and T.D. Pham, Segmentation of clustered nuclei based on concave curve expansion. J Microsc, 2013. 251(1): p. 57-67. 134. Jung, C. and C. Kim, Impact of the accuracy of automatic segmentation of cell nuclei clusters on classification of thyroid follicular lesions. Cytometry A, 2014. 85(8): p. 709-18. 135. Maree, R., et al., Collaborative analysis of multi-gigapixel imaging data using Cytomine. Bioinformatics, 2016. 32(9): p. 1395-401. 136. Marée, R., et al., A rich internet application for remote visualization and collaborative annotation of digital slides in histology and cytology. Diagnostic Pathology, 2013. 8(1): p. S26. 137. Dieu-Nosjean, M.C., et al., Tertiary lymphoid structures in cancer and beyond. Trends Immunol, 2014. 35(11): p. 571-80. 138. Svensson, C.M., et al., Segmentation of clusters by template rotation expectation maximization. Computer Vision and Image Understanding, 2017. 154: p. 64-72. 139. Song, Y., et al., Accurate Cervical Cell Segmentation from Overlapping Clumps in Pap Smear Images. IEEE Trans Med Imaging, 2017. 36(1): p. 288-300. 140. Song, Y., et al., Accurate Segmentation of Cervical Cytoplasm and Nuclei Based on Multiscale Convolutional Network and Graph Partitioning. IEEE Trans Biomed Eng, 2015. 62(10): p. 2421-33. 141. Xing, F., Y. Xie, and L. Yang, An Automatic Learning-Based Framework for Robust Nucleus Segmentation. IEEE Trans Med Imaging, 2016. 35(2): p. 550-66. 142. Song, J., L. Xiao, and Z. Lian, Boundary-to-Marker Evidence-Controlled Segmentation and MDL-Based Contour Inference for Overlapping Nuclei. IEEE J Biomed Health Inform, 2017. 21(2): p. 451-464. 143. Mathew, B., et al., Robust and automated three-dimensional segmentation of densely packed cell nuclei in different biological specimens with Lines-of-Sight decomposition. BMC Bioinformatics, 2015. 16: p. 187. 144. Fatichah, C., et al., Nuclei Segmentation of Microscopic Breast Cancer Image using Gram-Schmidt and Cluster Validation Algorithm. Proceedings 5th IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2015), 2015: p. 236-241. 145. Liu, Z., et al., Segmentation of White Blood Cells through Nucleus Mark Watershed Operations and Mean Shift Clustering. Sensors (Basel), 2015. 15(9): p. 22561-86. 146. Su, H., et al., Robust Cell Detection and Segmentation in Histopathological Images Using Sparse Reconstruction and Stacked Denoising Autoencoders. Med Image Comput Comput Assist Interv, 2015. 9351: p. 383-390. 147. Barker, J., et al., Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal, 2016. 30: p. 60-71. 149  148. Zarei, N., et al., Automated Region-based Prostate Cancer Cell Nuclei Localization. Part of a Prognostic Modality Tool for Prostate Cancer Patients. Anal Quant Cytopathol Histpathol, 2016. 38(2): p. 59-69. 149. Halama, N., et al., Estimation of immune cell densities in immune cell conglomerates: an approach for high-throughput quantification. PLoS One, 2009. 4(11): p. e7847. 150. Niazi, M.K.K., et al., Hotspot Detection in Pancreatic Neuroendocrine Tumors: Density Approximation by alpha-shape Maps. Medical Imaging 2016: Digital Pathology, 2016. 9791. 151. Sonka, M., V. Hlavac, and R. Boyle, Image Processing, Analysis, and Machine Vision, Third Edition. 2008, United States of America: Thomson Learning. 152. Ruifrok, A.C. and D.A. Johnston, Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol, 2001. 23(4): p. 291-9. 153. Janowczyk, A. and A. Madabhushi, Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. J Pathol Inform, 2016. 7: p. 29. 154. Chae, Y.K., et al., Biomarkers for PD-1/PD-L1 Blockade Therapy in Non-Small-cell Lung Cancer: Is PD-L1 Expression a Good Marker for Patient Selection? Clin Lung Cancer, 2016. 17(5): p. 350-361. 155. Hirsch, F.R., et al., PD-L1 Immunohistochemistry Assays for Lung Cancer: Results from Phase 1 of the Blueprint PD-L1 IHC Assay Comparison Project. J Thorac Oncol, 2017. 12(2): p. 208-222. 156. Tsao, M.S., et al., IASLC ATLAS OF PD-L1 IMMUNOHISTOCHEMISTRY TESTING IN LUNG CANCER. 2017, International Association for the Study of Lung Cancer: Colorado, USA. 157. Hansen, A.R. and L.L. Siu, PD-L1 Testing in Cancer: Challenges in Companion Diagnostic Development. JAMA Oncol, 2016. 2(1): p. 15-6. 158. PerkinElmer, inForm Cell Analysis. 2017. 159. Certo, S.T., et al., Sample selection bias and Heckman models in strategic management research. Strategic Management Journal, 2016. 37(13): p. 2639-2657. 160. Krautenbacher, N., F.J. Theis, and C. Fuchs, Correcting Classifiers for Sample Selection Bias in Two-Phase Case-Control Studies. Computational and Mathematical Methods in Medicine, 2017. 161. Brown, D.L., Bias in image analysis and its solution: unbiased stereology. J Toxicol Pathol, 2017. 30(3): p. 183-191. 162. Hegedus, E.J. and J. Moody, Clinimetrics corner: the many faces of selection bias. J Man Manip Ther, 2010. 18(2): p. 69-73. 163. West, M.J., New stereological methods for counting neurons. Neurobiol Aging, 1993. 14(4): p. 275-85. 164. Enwere, E.K., et al., Expression of PD-L1 and presence of CD8-positive T cells in pre-treatment specimens of locally advanced cervical cancer. Mod Pathol, 2017. 30(4): p. 577-586. 165. Kim, H.R., et al., PD-L1 expression on immune cells, but not on tumor cells, is a favorable prognostic factor for head and neck cancer patients. Sci Rep, 2016. 6: p. 36956. 166. Oishi, S., Geography and personality: why do different neighborhoods have different vibes? Proc Natl Acad Sci U S A, 2015. 112(3): p. 645-6. 150  167. Marée, R., et al., A hybrid human-computer approach for large-scale image-based measurements using web services and machine learning. Biomedical Imaging (ISBI), 2014 IEEE 11th International Symposium on, 2014. 168. Oliveira, F.P. and J.M. Tavares, Medical image registration: a review. Comput Methods Biomech Biomed Engin, 2014. 17(2): p. 73-93. 169. Roberts, N., et al., Toward routine use of 3D histopathology as a research tool. Am J Pathol, 2012. 180(5): p. 1835-42. 170. Linden, M., J.M. Ward, and S. Cherian, Hematopoietic and Lymphoid Tissues. In: Treuting PM, Dintzis SM, editors. Comparative Anatomy and Histology: A Mouse and Human Atlas. , ed. P. Treuting and S. Dintzis. 2012, Amsterdam, Netherlands: Elsevier. 171. Rich, R., et al., Clinical Immunology: Principles and Practice, Fourth Edition. 2013, United States: Elsevier Limited  172. Hill, D.L., et al., Medical image registration. Phys Med Biol, 2001. 46(3): p. R1-45. 173. Gallas, B.D., et al., Evaluation environment for digital and analog pathology: a platform for validation studies. J Med Imaging (Bellingham), 2014. 1(3): p. 037501. 174. Zhang, Y.L., et al., Non-rigid landmark-based large-scale image registration in 3-D reconstruction of mouse and rat kidney nephrons. Micron, 2015. 68: p. 122-9. 175. Yagi, Y. and J.R. Gilbertson, Digital imaging in pathology: the case for standardization. J Telemed Telecare, 2005. 11(3): p. 109-16. 176. Gustafsson, J.S., A. Blomberg, and M. Rudemo, Warping two-dimensional electrophoresis gel images to correct for geometric distortions of the spot pattern. Electrophoresis, 2002. 23(11): p. 1731-44. 177. Le Nobin, J., et al., Prostate tumour volumes: evaluation of the agreement between magnetic resonance imaging and histology using novel co-registration software. BJU Int, 2014. 114(6b): p. E105-E112. 178. Ilie, M., et al., Assessment of the PD-L1 status by immunohistochemistry: challenges and perspectives for therapeutic strategies in lung cancer patients. Virchows Arch, 2016. 468(5): p. 511-25. 179. Aguiar, P.N., Jr., et al., The role of PD-L1 expression as a predictive biomarker in advanced non-small-cell lung cancer: a network meta-analysis. Immunotherapy, 2016. 8(4): p. 479-88. 180. Rimm, D.L., et al., A Prospective, Multi-institutional, Pathologist-Based Assessment of 4 Immunohistochemistry Assays for PD-L1 Expression in Non-Small Cell Lung Cancer. JAMA Oncol, 2017. 3(8): p. 1051-1058. 181. Ma, Z., et al., Data integration from pathology slides for quantitative imaging of multiple cell types within the tumor immune cell infiltrate. Diagn Pathol, 2017. 12(1): p. 69. 182. Song, Y., et al., Unsupervised content classification based nonrigid registration of differently stained histology images. IEEE Trans Biomed Eng, 2014. 61(1): p. 96-108. 183. Magee, D., et al., Histopathology in 3D: From three-dimensional reconstruction to multi-stain and multi-modal analysis. J Pathol Inform, 2015. 6: p. 6. 184. Mayerich, D., et al., Stain-less staining for computed histopathology. Technology (Singap World Sci), 2015. 3(1): p. 27-31. 185. Tam, A., J. Barker, and D. Rubin, A method for normalizing pathology images to improve feature extraction for quantitative pathology. Med Phys, 2016. 43(1): p. 528. 151  186. Janowczyk, A., A. Basavanhally, and A. Madabhushi, Stain Normalization using Sparse AutoEncoders (StaNoSA): Application to digital pathology. Comput Med Imaging Graph, 2017. 57: p. 50-61. 187. Han, D., et al., Robust anatomical landmark detection for MR brain image registration. Med Image Comput Comput Assist Interv, 2014. 17(Pt 1): p. 186-93. 188. Obando, D.G., et al., Multi-staining registration of large histology images. Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on, 2017. 189. Schaadt, N.S., et al., Image analysis of immune cell patterns in the human mammary gland during the menstrual cycle refines lymphocytic lobulitis. Breast Cancer Res Treat, 2017. 164(2): p. 305-315. 190. Ruddle, R.A., et al., The Design and Evaluation of Interfaces for Navigating Gigapixel Images in Digital Pathology. ACM Transactions on Computer-Human Interaction (TOCHI), 2016 23 (1). 191. Parra, E.R., et al., Image Analysis-based Assessment of PD-L1 and Tumor-Associated Immune Cells Density Supports Distinct Intratumoral Microenvironment Groups in Non-small Cell Lung Carcinoma Patients. Clin Cancer Res, 2016. 22(24): p. 6278-6289. 192. Chatterjee, S., Artefacts in histopathology. J Oral Maxillofac Pathol, 2014. 18(Suppl 1): p. S111-6. 193. Rastogi, V., et al., Artefacts: a diagnostic dilemma - a review. J Clin Diagn Res, 2013. 7(10): p. 2408-13. 194. Palokangas, S., J. Selinummi, and O. Yli-Harja, Segmentation of folds in tissue section images. Conf Proc IEEE Eng Med Biol Soc, 2007. 2007: p. 5642-5. 195. Bautista, P.A. and Y. Yagi, Detection of tissue folds in whole slide images. Conf Proc IEEE Eng Med Biol Soc, 2009. 2009: p. 3669-72. 196. Bautista, P.A. and Y. Yagi, Improving the visualization and detection of tissue folds in whole slide images through color enhancement. J Pathol Inform, 2010. 1: p. 25. 197. Dennis, E., et al., Validation of an electronic program for pathologist training in the interpretation of a complex companion diagnostic immunohistochemical assay. Hum Pathol, 2016. 56: p. 194-203. 198. Al-Shaikhli, S.D.S., M.Y. Yang, and B. Rosenhahn, Multi-region labeling and segmentation using a graph topology prior and atlas information in brain images. Computerized Medical Imaging and Graphics, 2014. 38(8): p. 725-734. 199. Kothari, S., J.H. Phan, and M.D. Wang, Eliminating tissue-fold artifacts in histopathological whole-slide images for improved image-based prediction of cancer grade. J Pathol Inform, 2013. 4: p. 22.  152  Appendices   Appendix A -  Manual graphs of cell-to-cell spatial analyses (raw data without exclusion of data indicating not enough cells for analysis, value = -999)   A.1    General cell type cell quantity and spatial relations of same neighbors in adaptive resistance panel (raw data plot).   X-axis indicates 20 cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates case contains large negative variance (VAR = -999), suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with no line of best fit and 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.         153              Percentage                                                  MNG                                                      VAR CD8+            PD1+           PDL1+            PDL1++       * Percentage (PDL1++) Percentage (PDL1+) Percentage (PD1+) Percentage (CD8+) 154  A.2    Specific cell type identified by double markers cell quantity and spatial relations of same neighbors (raw data plot).   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that case contains large negative variance (VAR = -999), suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case                         B A *  Percentage                                         MNG                                                VAR Percentage (CD3+CD8+) CD3+CD8+             CD8+PD1+  Percentage (CD8+PD1+) 155  A.3   Specific cell type identified by triple markers cell quantity and spatial relations of same neighbors (raw data plot).   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that case contains large negative variance (MNG = -999; VAR = -999), suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit or no 95% CI or lines not shown is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.               156                             * * * *   * * * * * * * CD3+CD8+CD79a+           CD8+PD1+PDL1+           CD8+PD1+PDL1++             CD8+PD1+  Percentage (CD8+PD1+PDL1+) Percentage (CD8+PD1+PDL1++) Percentage (CD3+CD8+CD79a+) A B 157  A.4    General cell type next to different general cell type neighbor cell quantity and spatial relations and vice versa (raw data plot).   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that case contains large negative variance (VAR = -999), suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with no line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                   158                                                                                                                       CD3+ / CD79a+         CD79a+ / CD3+          PD1+ / PDL1+          PDL1+ / PD1+         PD1+ / PDL1++          PDL1++ / PD1+   B * * A 159  A.5    General cell type next to specific cell type neighbor and vice versa cell quantity and spatial relations (raw data plot).   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Asterisk (*) indicates that case contains large negative variance (VAR = -999), suggesting case has not enough cells to assess variance. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval (CI) of mean of ROI. Graph with vertical line of best fit and no 95% CI is out of bounds from skew of negative VAR values. Graphs that case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                      160                         B * * A CD3+CD8+ / CD79a+           CD79a+ / CD3+CD8+           CD8+PD1+ / PDL1+          PDL1+ / CD8+PD1+           CD8+PD1+ / PDL1++         PDL1++ / CD8+PD1+                                                              MNG                                                    VAR 161  Appendix B -  Autoscaled graphs of cell-to-cell spatial analyses   B.1    General cell type cell quantity and spatial relations of same neighbors in T&B cell panel with graph autoscaling.  X-axis indicates 20 cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.             162                             Percentage                                                  MNG                                                                     VAR CD3+            CD79a+             CD8+    Percentage (CD3+) Percentage (CD79a+) Percentage (CD8+) 163  B.2    General cell type cell quantity and spatial relations of same neighbors in adaptive resistance panel with graph            autoscaling.  X-axis indicates 20 cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                164                        Percentage                                           MNG                                              VAR   CD8+          PD1+           PDL1+           PDL1++       Percentage (CD8+) Percentage (PDL1++) Percentage (PDL1+) Percentage (PD1+) 165  B.3    Specific cell type identified by double markers cell quantity and spatial relations of same neighbors with graph             autoscaling.   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                      Percentage                                                  MNG                                                                   VAR A B Percentage (CD3+CD8+) CD3+CD8+              CD8+PD1+  Percentage (CD8+PD1+) 166  B.4    Specific cell type identified by triple markers cell quantity and spatial relations of same neighbors with graph            autoscaling.  (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                   167        A B CD3+CD8+CD79a+           CD8+PD1+PDL1+           CD8+PD1+PDL1++             CD8+PD1+  Percentage (CD3+CD8+CD79a+) Percentage (CD8+PD1+PDL1+) Percentage (CD8+PD1+PDL1++) 168  B.5    General cell type next to different general cell type neighbor and vice versa cell            quantity and spatial relations with graph autoscaling.  (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                          169                                CD3+ / CD79a+         CD79a+ / CD3+          PD1+ / PDL1+          PDL1+ / PD1+          PD1+ / PDL1++         PDL1++ / PD1+                   MNG                                                        VAR A B 170  B.6    General cell type next to specific cell type neighbor and vice versa cell quantity and            spatial relations with graph autoscaling.   (A) T&B cell panel. (B) ARPDL1high panel. X-axis indicates 20 lung AC cases. Each column in each plot represents one case. Whiskers indicate SD. Points indicate value for one ROI, where there are five ROI sampled in total for each case (column). Red solid line indicates line of best fit of mean of ROI. Dotted blue bands indicate 95% confidence interval of mean of ROI. Graphs that Case 06L50 and 06L19 have less than five ROI, as ROI were not able to be segmented. Dash indicates average parameter of each case.                     171                                CD3+CD8+ / CD79a+           CD79a+ / CD3+CD8+           CD8+PD1+ / PDL1+          PDL1+ / CD8+PD1+           CD8+PD1+ / PDL1++          PDL1++ / CD8+PD1+                                                             MNG                                                 VAR A B 172  Appendix C - List of publications and selected conference proceedings  The appendix lists all the publications that were published or are prepared for submission and selected conference proceedings, including those published in journals that I have contributed to during my degree. Co-first authorships are underlined. Obelus (†) indicates corresponding authorship.   Publications  1. Hubaux R, Thu KL, Vucic EA, Pikor LA, [Kung SHY], Martinez VD, Mosslemi M, Becker-Santos DD, Gazdar AF, Lam S, Lam WL. (2015). Microtubule affinity-regulating kinase 2 contributes to cisplatin sensitivity through modulation of the DNA damage response in non-small cell lung cancer. International Journal of Cancer. 137(9):2072-82. [PMID:25907283].  2. Conway EM, Pikor LA, [Kung SHY], Hamilton MJ, Lam S, Lam WL, and Bennewith KL. (2015). Macrophages, inflammation, and lung cancer. American Journal of Respiratory and Critical Care Medicine. 193(2):116-30. [PMID:26583808].           -- This publication was included in a part of Chapter 1 of this thesis. --     3. Marshall EA, Ng KW, [Kung SHY†], Conway EM, Martinez VD, Halvorsen EC, Rowbotham DA, Vucic EA, Plumb AW, Becker-Santos DD, Enfield KSS, Kennett JY, Bennewith KL, Lockwood WW, Lam S, English JC, Abraham N, Lam WL†. (2016). Emerging roles of T helper 17 and regulatory T cells in lung cancer progression and metastasis. Molecular Cancer. 15(1), 67. [PMID: 27784305].          -- This publication was included in a part of Chapter 1 of this thesis. --   4. Enfield KSS, Martinez VD, Marshall EA, Stewart GL, [Kung SHY], Enterina JR, Lam WL. (2016). Deregulation of small non-coding RNAs at the DLK1-DIO3 imprinted locus predicts lung cancer patient outcome. Oncotarget. 7(49):80957-66. [PMID: 27829231].          -- This publication was cited in Chapter 1 of this thesis. --          173  Conference proceedings published in research journals  1. Enfield KSS, Martin SD, Martinez VD, [Kung SHY], Gallagher P, Milne K, Chen ZY, English JC, MacAulay CE, Guillaud M, Lam WL. (2017). P2.02-038 Imaging platform for the quantification of cell-cell spatial organization within the tumour-immune microenvironment. Journal of Thoracic Oncology. 12(11), Supplement 2, S2112–S2113 [DOI: http://dx.doi.org/10.1016/j.jtho.2017.09.1216].          -- This published conference proceeding was included in Chapter 2 of this         thesis. This proceeding was presented at the IASLC 18th World Conference  on Lung Cancer at  Yokohama, Japan as a late-breaking abstract by KSS Enfield. --  2. Marshall EA, Ng KW, Enfield KSS, Martin SD, Milne K, [Kung SHY], MacAulay CE, Lam WL. (2017). MA 10.09 Increased T Follicular Helper Cell Infiltration in Lung Adenocarcinoma Tertiary Lymphoid Organs. Journal of Thoracic Oncology. 12(11), Supplement 2, S1842 [DOI: http://dx.doi.org/10.1016/j.jtho.2017.09.539].  --  A part of this thesis was included in Chapter 2 of this thesis. This proceeding was presented as an invited talk by EA Marshall at the IASLC 18th World Conference on Lung Cancer at Yokohama, Japan as a mini oral presentation --    3. [Kung SHY], Enfield KSS, Rowbotham DA, Marshall EA, Holly A, Pastrello C, Minatel BC, Dellaire G, Berman J, Jurisica I, Macaulay CE, Lam S, Lam WL. (2016). P2.01-024 Expression of miR-106 paralogs improves prognostic value of mesenchymal signatures but only miR-106b promotes invasiveness. Journal of Thoracic Oncology. 12(1), Supplement, S799. [DOI: http://dx.doi.org/10.1016/j.jtho.2016.11.1076].  4. Enfield KSS, [Kung SHY], Gallagher P, Milne K, Chen Z, Piga D, Lam S, English JC, Guillaud M, MacAulay CE, Lam WL. (2016). P2.01-065 Quantification of tumour-immune cell spatial relationships in the lung tumour microenvironment using single cell profiling. Journal of Thoracic Oncology. 12(1), Supplement, S826–S827. [DOI: http://dx.doi.org/10.1016/j.jtho.2016.11.1117].         --  A part of this thesis was included in Chapter 2 of this thesis --            174  Additional international conference proceedings (related to research in this thesis)  5. Enfield KSS, Martin SD, [Kung SHY], Gallagher P, MacAulay CE, Guillaud M, Lam WL (2018). Hyperspectral imaging tools capture the spatial organization of cell subsets within the tumour microenvironment. 5th AACR-IASLC International Joint Conference: Lung Cancer Translational Science from the Bench to the Clinic. San Diego, USA. --  A part of this thesis was included in Chapter 2 of this thesis. The following conference proceeding will be presented as a plenary by KSS Enfield and will be published in the Journal of Thoracic Oncology. --  6. Enfield KSS, Martin SD, Martinez VD, [Kung SHY], Gallagher P, Chen Z, Lam S, English JC, Lam WL, MacAulay CE, Guillaud MD. (2018). Hyperspectral Cell Sociology Tools for the Study of Tumour-Immune Cell Interactions in Lung Adenocarcinoma. United States and Canadian Academy of Pathology (USCAP 2018) Annual Meeting. Vancouver, Canada.                --  A part of this thesis was included in Chapter 2 of this thesis --  7. Enfield KSS, [Kung SHY], Gallagher P, Milne K, Chen ZY, Lam S, English JC, Guillaud M, MacAulay CE, Lam WL. (2016). Single cell profiling and cell sociology of the lung tumor microenvironment. Cell-VIB Symposium: Hallmarks of Cancer, Ghent, Belgium.                --  A part of this thesis was included in Chapter 2 of this thesis --            

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0363386/manifest

Comment

Related Items