Applied Science, Faculty of
Biomedical Engineering, School of
DSpace
UBCV
Sheikhzadeh, Fahime
Doctor of Philosophy - PhD
University of British Columbia
Two directions are explored for improving the current cervical cancer diagnosis procedure. The first investigates the future deployment of in vivo confocal imaging in the clinic, for detecting precancerous tissues, and the second proposes an algorithm for automatic interpretation of histology images (acquired by light microscopy). We acquired i) confocal microscopy images of cervical biopsies taken from 50 patients, at different tissue depths and ii) histology images of different sections cut from each biopsy. From the confocal images, we identified four features that carry enough information relevant to cell morphology and tissue architecture. We demonstrated that the relevant information in these features is comparable to that extracted from the same features in histology images. This implies that we can obtain the relevant information from confocal imaging, without having to cut a biopsy from the patient’s cervix. We then studied the confocal images and determined the grade lesion of every biopsy and found that confocal imaging resulted in less false positives than the diagnosis given by the gynecologist (based on the appearance of the cervix under colposcopy). Utilizing confocal microscopy technology in the clinic would thus decrease the number of unnecessary biopsies. We then developed a deep learning algorithm that automatically and quantitatively assesses HPV contaminated and proliferating cells in histology images of biopsy sections. The automatic assessment of this procedure is important as it plays a significant role in differentiating between disease grades but forms a challenging and complex task and demands a large amount of time when performed manually by a pathologist. We demonstrated that this algorithm could help the pathologists to differentiate between different grades of cervical precancerous tissues. Our results are also more reproducible compared to other methods (like color deconvolution) that are widely being used in the field of digital pathology. The in vivo imaging and automatic image analysis algorithms demonstrated in the thesis can potentially enable i) real time diagnosis in the clinic, and ii) fast interpretation of histology images in a reproducible and cost-effective manner. While developed for cervical neoplasia, these methods could be extended to oral cavity, skin, and other epithelial tissue cancers.
IMPROVING CERVICAL NEOPLASIA DIAGNOSIS VIA NOVEL IN VIVO IMAGING TECHNOLOGIES AND DEEP LEARNING ALGORITHMS by Fahime Sheikhzadeh B.Sc. Amirkabir Univeristy of Technology, 2007 M.Sc. Amirkabir Univeristy of Technology, 2009 M.A.Sc., The University of British Columbia, 2012 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (BIOMEDICAL ENGINEERING) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) April 2018 © Fahime Sheikhzadeh, 2018 ii Abstract Two directions are explored for improving the current cervical cancer diagnosis procedure. The first investigates the future deployment of in vivo confocal imaging in the clinic, for detecting precancerous tissues, and the second proposes an algorithm for automatic interpretation of histology images (acquired by light microscopy). We acquired i) confocal microscopy images of cervical biopsies taken from 50 patients, at different tissue depths and ii) histology images of different sections cut from each biopsy. From the confocal images, we identified four features that carry enough information relevant to cell morphology and tissue architecture. We demonstrated that the relevant information in these features is comparable to that extracted from the same features in histology images. This implies that we can obtain the relevant information from confocal imaging, without having to cut a biopsy from the patient’s cervix. We then studied the confocal images and determined the grade lesion of every biopsy and found that confocal imaging resulted in less false positives than the diagnosis given by the gynecologist (based on the appearance of the cervix under colposcopy). Utilizing confocal microscopy technology in the clinic would thus decrease the number of unnecessary biopsies. We then developed a deep learning algorithm that automatically and quantitatively assesses HPV contaminated and proliferating cells in histology images of biopsy sections. The automatic assessment of this procedure is important as it plays a significant role in differentiating between disease grades but forms a challenging and complex task and demands a large amount of time iii when performed manually by a pathologist. We demonstrated that this algorithm could help the pathologists to differentiate between different grades of cervical precancerous tissues. Our results are also more reproducible compared to other methods (like color deconvolution) that are widely being used in the field of digital pathology. The in vivo imaging and automatic image analysis algorithms demonstrated in the thesis can potentially enable i) real time diagnosis in the clinic, and ii) fast interpretation of histology images in a reproducible and cost-effective manner. While developed for cervical neoplasia, these methods could be extended to oral cavity, skin, and other epithelial tissue cancers. iv Lay Summary Cervical cancer remains a major health problem, especially in developing countries. Currently, colposcopic examination is used to detect high-grade cervical lesions and taking biopsies in patients with a history of abnormal Pap-smears. Then a pathologist examines the biopsy sections under a microscope to detect and grade the disease. New technologies are needed to improve the accuracy and effectiveness of this procedure. We propose improvements to the current procedure by: i) implementing in vivo fluorescence confocal microscopy in the clinic to identify high-grade lesions, and ii) developing fully automatic algorithms for quantitative assessment of cell proliferation in microscopy images of biopsy sections. We demonstrate that utilizing in vivo confocal imaging in the clinic would reduce the number of unnecessary biopsies taken in the current diagnosis procedure. Also, we demonstrate that our developed automatic algorithm could help the pathologist to differentiate between different grades of precancerous tissues in a fast, reproducible, and cost-effective manner. v Preface This dissertation presents the research conducted by Fahime Sheikhzadeh. The research was supervised by Prof. Rabab Ward at the University of British Columbia and co-supervised by Dr. Martial Guillaud at British Columbia Cancer Research Centre. The list of the scientific papers published by Fahime Sheikhzadeh during the course of her Doctor of Philosophy studies at UBC is as follows:  J1: Fahime Sheikhzadeh, Rabab K. Ward, Anita Carraro, Zhaoyang Chen, Dirk van Niekerk, Dianne Miller, Tom Ehlen, Calum MacAulay, Michele Follen, Pierre Lane, Martial Guillaud. (2015) Quantification of confocal fluorescence microscopy for the detection of cervical interaepithelial neoplasia. Biomedical Engineering Online 14 (96).  J2: Fahime Sheikhzadeh, Rabab K. Ward, Dirk van Niekerk, Martial Guillaud. (2018) Automatic labeling of molecular biomarkers of immunohistochemistry images using fully convolutional networks. PLoS ONE 13 (1).  C1: Fahime Sheikhzadeh, Qian Ye, Nasir Zulkafly, Anita Carraro, Jagoda Korbelic, Zhaoyang Chen, Alan Harrison, Michele Follen, Calum MacAulay, Rabab K. Ward, Martial Guillaud. (2014) CI-Slide: calibration slide for quantitative microscopy imaging in absorbance. In Proceedings of SPIE 8945, Design and Performance Validation of Phantoms Used in Conjunction with Optical Measurement of Tissue VI, 894507 (3 March 2014)  C2: Fahime Sheikhzadeh, Rabab K. Ward, Anita Carraro, Zhaoyang Chen, Dirk van Niekerk, Calum MacAulay, Michele Follen, Pierre Lane, Martial Guillaud. (2015) Confocal vi fluorescence microscopy for detection of cervical preneoplastic lesions. In Proceedings of SPIE 9420, Medical Imaging, Digital Pathology, 942009 (27 April 2015).  C3: Fahime Sheikhzadeh, Anita Carraro, Jagoda Korbelik, Calum MacAulay, Martial Guillaud, Rabab K. Ward. (2016) Automatic labeling of molecular biomarkers on a cell-by-cell basis in immunohistochemistry images using convolutional neural networks. In Proceedings of SPIE 9791, Medical Imaging 2016: Digital Pathology, 97910R (23 March 2016). Fahime Sheikhzadeh was the primary author and the main contributor in all the papers. Her roles included: conceptualization, proposing methodology, developing mathematical and imaging algorithms, formal analysis, Software, validation, data curation, visualization, and writing the original drafts. Prof. Rabab K. Ward supervised the development of work, and provided extensive scientific and technical feedback as well as editorial comments throughout writing of all the papers (J1, J2, C1, C2 and C3). Dr. Martial Guillaud supervised the development of work, directed the data curation, and provided extensive feedback from the biostatistics and clinical point of view as well as editorial comments throughout writing of all the papers. Dr. Dianne Miller and Dr. Tom Ehlen were the gynecologists at Vancouver General Hospital, Women’s Clinic, who performed the examination of the cervix in the clinic, and collected the cervical biopsies. Dr. Dirk van Niekerk was the study pathologist who reviewed all the histological specimens used in papers J1, J2, C2, and C3 and validated the pathological findings in J2. Ms. Anita Carraro, and Ms. Jagoda Korbelic stained the tissue sections, and helped the vii primary author with microscopy imaging of the tissue sections. Dr. Pierre Lane set up the ex vivo confocal microscopy used for imaging in J1 and C2. Drs Calum MacAulay and Michele Follen provided useful feedback and comments in papers J1, C1 and C2. Mr. Zhaoyang Chen and Mr. Alan Harrison developed the software of the in house imaging system (Getafics) that was used to acquire and analyze some of the images in papers J1 and C1. A version of part of Chapter 2 of this dissertation appeared in C1. Versions of Chapter 3 appeared in J1 and C2. Versions of Chapter 4 appeared in J2 and C3. The research presented in this dissertation was approved by the UBC BCCA Research Ethics Board and the Vancouver Coastal Health Research Institute (Protocols H09-03303 and H03-61235). The work was funded through a Natural Sciences and Engineering Research Council of Canada grant (RGPIN 2014-04462) to RKW and a National Institute of Health funded program project grant (P01–CA-82710). viii Table of Contents Abstract .......................................................................................................................................... ii Lay Summary ............................................................................................................................... iv Preface .............................................................................................................................................v Table of Contents ....................................................................................................................... viii List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiii List of Abbreviations ................................................................................................................. xvi Glossary .................................................................................................................................... xviii Acknowledgements ......................................................................................................................xx Dedication .................................................................................................................................. xxii Chapter 1: Introduction ................................................................................................................1 1.1 Cervical Cancer and Diagnosis/Staging Procedure ........................................................ 4 1.2 Motivation ....................................................................................................................... 5 1.3 Research Objective ......................................................................................................... 9 1.4 Research Scope ............................................................................................................. 12 1.5 Thesis Structure ............................................................................................................ 19 Chapter 2: Specimen Collection, Data Acquisition and Quality Control ...............................20 2.1 Clinical Trial and Data Acquisition .............................................................................. 22 2.1.1 Patient Recruitment and Specimen Collection ......................................................... 22 2.1.2 Specimen Processing and Staining ........................................................................... 24 2.2 CI-Slide, a Calibration Imaging Slide ........................................................................... 25 ix 2.2.1 Experimental Design for Analyzing the Characteristics of CI-Slide ........................ 27 2.2.1.1 CI-Slide Design ................................................................................................. 27 2.2.1.2 Imaging Devices ............................................................................................... 30 2.2.1.3 Image Analysis .................................................................................................. 31 2.2.1.4 Feature Calculation: Photometric, Morphology and Architecture Features ..... 34 2.2.1.5 Repeatability and Reproducibility .................................................................... 36 2.2.1.6 Effects of Magnification, Transmission, Size and Density of Objects on Feature Measurements ................................................................................................................... 37 2.2.2 Results and Discussion ............................................................................................. 38 2.2.2.1 Repeatability and Reproducibility .................................................................... 38 2.2.2.2 Effects of Magnification, Transmission, Object Size and Pitch on Feature Measurements ................................................................................................................... 41 2.2.2.3 Effect of Segmentation Algorithm on Feature Measurements ......................... 45 2.2.2.4 Effect of Architectural Pattern on Feature Measurements ................................ 46 2.3 Summary and Conclusions ........................................................................................... 47 Chapter 3: Developing in vivo Imaging Tools ...........................................................................49 3.1 Introduction ................................................................................................................... 49 3.2 Materials and Methods .................................................................................................. 51 3.2.1 Confocal Fluorescence Microscopy .......................................................................... 52 3.2.2 Employing CI-Slide for Image Calibration ............................................................... 53 3.2.3 Confocal Image Analysis .......................................................................................... 56 3.2.4 Imaging of Histological Sections Stained with Thionin-Feulgen ............................. 57 3.2.5 Quantitative Tissue Phenotype (QTP) Analysis ....................................................... 58 x 3.3 Results ........................................................................................................................... 60 3.3.1 Quantitative Tissue Phenotype Analysis of Confocal Images .................................. 63 3.3.2 Quantitative Tissue Phenotype Analysis of Histological Images ............................. 66 3.3.3 Comparison of QTP Analysis of Confocal Images with Colposcopic Appearance . 67 3.3.4 Quantitative Tissue Phenotype Analysis of Confocal Images at Different Depths .. 69 3.4 Discussion ..................................................................................................................... 72 3.5 Summary and Conclusion ............................................................................................. 76 Chapter 4: Developing Automatic Algorithms for the Analysis of Histological Images .......77 4.1 Introduction ................................................................................................................... 77 4.2 Dataset........................................................................................................................... 83 4.2.1 Imaging Dual-Immunostained Specimens ............................................................... 83 4.3 Biomarker Labeling Method ......................................................................................... 85 4.3.1 Nuclei Classification using N-Net ............................................................................ 87 4.3.2 Biomarker Localization in Whole Slide Images Using WI-Net ............................... 88 4.4 Experiments and Results ............................................................................................... 89 4.4.1 Biomarker Localization ............................................................................................ 89 4.4.1.1 Preparing Nuclei Images for Training N-Net ................................................... 90 4.4.1.2 Training and Testing N-Net .............................................................................. 91 4.4.1.3 Labeling Biomarkers in IHC Images Using WI-Net ........................................ 97 4.4.2 Validation .................................................................................................................. 99 4.4.2.1 Comparison with Manual Biomarker Labeling .............................................. 101 4.4.2.2 Performance Evaluation in Quantification of Cell Proliferation .................... 104 4.5 Discussion ................................................................................................................... 111 xi 4.6 Summary and Conclusion ........................................................................................... 115 Chapter 5: Conclusion s and Future Work .............................................................................116 5.1 Summary ..................................................................................................................... 116 5.2 Contributions............................................................................................................... 119 5.3 Future Areas of Research ............................................................................................ 121 Bibliography ...............................................................................................................................124 xii List of Tables Table 2.1. Summary of number of scheduled and recruited patients in the clinical trial. ........... 23 Table 2.2. Size, pitch, and number of circles in different circle boxes. ........................................ 29 Table 2.3. Analysis of Variance; SS: the Sum of Squares, Df: the degree of freedom, MS: the Mean Squares. ............................................................................................................................... 41 Table 3.1. Features generated from quantitative tissue phenotype analysis ................................. 59 Table 3.2. Classification of cervical biopsies based on colposcopic impression for the different histopathological grade. ................................................................................................................ 61 Table 3.3. Sensitivity and specificity of colposcopic appearance, ENC ratio, and 3NDND for classifying different grades of cervical dysplasia (imaging depth is 15 µm). .............................. 69 Table 3.4. Sensitivity and specificity of ENC ratio and 3NDND for the detection of cervical dysplasia at different confocal imaging depths. ............................................................................ 71 Table 4.1. Distribution of nuclei in the training and test sets. ..................................................... 91 Table 4.2. Confusion matrices of the classification results on the training set (top) and test set (bottom) using N-Net. ................................................................................................................... 94 Table 4.3. WI-Net performance evaluation of two IHC images taken from two WSIs. ........... 103 xiii List of Figures Figure 1.1. Current workflow of cervical cancer diagnosis ........................................................... 5 Figure 1.2. Potential improvements to cervical cancer diagnosis procedure ............................... 11 Figure 1.3. Samples of images acquired ....................................................................................... 18 Figure 2.1. Specimen collection and data acquisition .................................................................. 21 Figure 2.2. CI-Slide layout............................................................................................................ 28 Figure 2.3. Boxes containing objects with different shapes. ........................................................ 28 Figure 2.4. Semi automated analysis of the images from the scans using Dunit .......................... 33 Figure 2.5. Voronoi tessellation of a set of points and its dual Delaunay graph. ......................... 36 Figure 2.6. Experimental design for the study of repeatability and reproducibility. .................... 37 Figure 2.7. Experimental design and object properties in different 45 boxes of CI-Slide that contain circles ............................................................................................................................... 38 Figure 2.8. Relationship between mean of object intensity and transmission .............................. 39 Figure 2.9. Repeatability and Reproducibility study results ......................................................... 40 Figure 2.10. Effect of magnification on ErN measurements ......................................................... 42 Figure 2.11. Effect of transmission on ErN measurments ............................................................. 43 Figure 2.12. Effect of object size on ErN measurements .............................................................. 44 Figure 2.13. Effect of pitch on ErN measurments. ........................................................................ 45 Figure 2.14. Effect of the segmentation algorithm. ...................................................................... 46 Figure 2.15. Relationship between three nearest Delaunay neighbours distance and different pitch values ................................................................................................................................... 47 xiv Figure 3.1. Study design for investigating the potential of in vivo confocal microscopy for detecting high grade cervical preneoplastic lesions. ..................................................................... 52 Figure 3.2. A Siemens star on the CI-Slide .................................................................................. 54 Figure 3.3. Distortion correction of confocal images using the CI-Slide. .................................... 55 Figure 3.4. Image processing pipeline for confocal images ......................................................... 57 Figure 3. 5 ex vivo confocal images of the cervix taken at a depth of 15 µm and the corresponding histological images ................................................................................................ 62 Figure 3.6. Confocal images of a normal area taken at different epithelial depth from the surface of the tissue ................................................................................................................................... 63 Figure 3.7. Voronoi tessellation overlaying the confocal image of a CIN1 lesion, and its corresponding histopathological image ........................................................................................ 63 Figure 3.8. Results from analysis of confocal microscopy images ............................................... 65 Figure 3.9. Analysis of confocal microscopy images ................................................................... 66 Figure 3.10. QTP analysis of histological images ........................................................................ 67 Figure 3.11. Analysis of confocal microscopy images. Relationship between ENC ratio and colposcopic appearance for different histopathological grades .................................................... 69 Figure 3.12. Analysis of confocal microscopy images. Relationship between ENC ratio and colposcopic impression at different imaging depth ...................................................................... 70 Figure 4.1. Percentage of Ki-67 positive cells in the different epithelial layers of a Ki-67- stained CIN3 lesion (Mib) and of a dual-stained CIN 3 lesion (CINtec). ................................................ 81 Figure 4.2. Sample IHC images taken from CINtec-stained cervical tissues……………....…...85 Figure 4.3. A two-step approach to biomarker labeling in IHC imaging. ................................... 86 Figure 4.4. Schematic representation of the architecture of the N-Net ....................................... 88 xv Figure 4.5. Transforming the fully connected layers in the N-Net to convolution layers ........... 89 Figure 4.6. Sample of nuclei images obtained from an IHC image. ............................................ 90 Figure 4.7. Example images of synthetic dataset used for determining the architecture of the N-Net ................................................................................................................................................. 92 Figure 4.8. Illustration of the 96 filters in the convolution layer of the trained N-Net. ............... 95 Figure 4.9. The output of 96 filters in the convolution layer obtained from different nucleus images expressing different biomarker as the input of the N-Net. ............................................... 96 Figure 4.10. Heat-maps resulting from the biomarker labeling on an example IHC image. ....... 98 Figure 4. 11. Comparison between color deconvolution approach and WI-Net approach for locating p16 and Ki-67 positive pixels in two IHC images ........................................................ 100 Figure 4.12. The ROI selected on an IHC image and evenly distributed layers in the ROI parallel to the basal layer. ........................................................................................................................ 105 Figure 4.13. The average percentage of pixels classified as Ki-67 positive, p16 positive, and both Ki-67 and p16 positive in different layers of images of cervical epithelium as assessed for each pathology grade ........................................................................................................................... 106 Fig. 4.14. Expression of Ki-67 per layers in normal squamous cervical epithelia .................... 107 Figure 4.15. Expression of p16 and Ki-67 per layers in CIN1 lesions ...................................... 108 Figure 4.16. Expression of p16 and Ki-67 per layers in CIN2 lesions ...................................... 109 Figure 4.17. Expression of p16 and Ki-67 per layers in CIN3 lesions ...................................... 110 xvi List of Abbreviations ANOVA Analysis of Variance BCCA British Columbia Cancer Agency BCCRC British Columbia Cancer Research Centre CIN Cervical Intraepithelial Neoplasia CI-Slide Cancer Imaging Slide CPU Central Processing Unit DAB Diaminobenzidine DNA Deoxyribonucleic acid ENC Estimated Nucleus Area to Cytoplasmic Area ESACP European Society for Analytical Cellular Pathology GHZ Giga Hertz GPU Graphical Processing Unit H&E Hematoxylin and Eosin HPV Human Papillomavirus IHC Immunohistochemistry MS Mean Squares N-Net Nucleus Network QTP Quantitative Tissue Phenotype RGB Red Green Blue (colour model) ROI Region of Interest SD Standard Deviation xvii SIL Squamous Intraepithelial Lesion SS Sum of Squares TIFF Tagged Image File Format UBC University of British Columbia USAF United States Air Force WI-Net Whole Image Network WSI Whole Slide Image 3NDND Three nearest Delaunay Neighbours Distance xviii Glossary Colposcopy A procedure in which a lighted, magnifying instrument called a colposcope is used to examine the cervix, vagina, and vulva. Confocal Microscopy An optical imaging technique which uses a spatial pinhole to block out of focus light in image formation, enabling capturing multiple two-dimensional images at different depths within an object. Cell Proliferation An increase in the number of cells as a result of cell growth and cell division. Cervical Intraepithelial Neoplasia (CIN) Abnormal cells are found on the surface of the cervix. Cervical intraepithelial neoplasia is usually caused by certain types of human papillomavirus and is found when a cervical biopsy is done. CIN is not cancer, but may become cancer and spread to nearby normal tissue. It is graded on a scale of 1 to 3, based on how abnormal the cells look under a microscope and how much of the cervical tissue is affected. For example, CIN 1 has slightly abnormal cells and is less likely to become cancer than CIN 2 or CIN 3. Dysplasia Cells that look abnormal under a microscope but are not cancer. Epithelial Tissues The tissue type that covers all the exposed surfaces of the body. ex vivo Outside of the living body. Histology The study of tissues and cells under a microscope. It is often carried out by examining a thin slice (called a "Section" or a "Histological Section") of a tissue, under a light microscope, an electron microscope, or an infrared microscope. xix Histological Image Microscopy image of a histological section. These images can be acquired by conventional light microscopy imaging systems or by whole slide scanners. Human Papillomavirus (HPV) A type of virus that can cause abnormal tissue growth (for example, warts) and other changes to cells. Infection for a long time with certain types of HPV can cause cervical cancer. HPV may also play a role in some other types of cancer, such as anal, vaginal, vulvar, penile, and oropharyngeal cancers. Immunohistochemistry A lab test that uses antibodies to test for certain antigens (markers) in a sample of tissue. The antibodies are usually linked to an enzyme or a fluorescent dye. When the antibodies bind to the antigen in the tissue sample, the enzyme or dye is activated, and the antigen can then be seen under a microscope. Immunohistochemistry is used to help diagnose diseases, such as cancer. It may also be used to help tell the difference between different types of cancer. in vivo In the body. Lesion An area of abnormal tissue. Neoplasia Abnormal and uncontrolled cell growth. xx Acknowledgements I sincerely appreciate my supervisor, Prof. Rabab Ward, whom I see as my role model. She has always provided me with her scientific guidance and generous support. I am deeply thankful to Prof. Ward for her trust and understanding when I decided to switch my Ph.D. supervisor and join her research group. She gave me the privilege to choose my research topic, and provided the most insightful feedback to me for pursuing my passion. I would also like to thank Dr. Martial Guillaud for giving me the chance to conduct my Ph.D. project at British Columbia Cancer Research Centre (BCCRC) and providing the resources for my research. In particular, I should thank him for allowing me to get involved in all different steps of a cancer research study from the diagnostic clinic to the research planning. This along with our inspiring discussions gave me an invaluable understanding of different aspects of the clinical research. I feel privileged to work with some of the greatest scientists and well-known researchers in the field of medical imaging, pathology, and biostatistics during my Ph.D. and also research fellowship at the U.S. Food and Drug Administration (FDA). I thank them for broadening my vision of real-world healthcare problems. I immensely thank Dr. Marios Gavrielides at FDA, for the invaluable lessons he has generously taught me in the field of digital pathology. I am thankful to the staff in the Imaging Unit at BCCRC, Dr. Dirk van Niekerk at British Columbia Cancer Agency and the physicians and nurses in Women’s clinic at Vancouver General Hospital (VGH) who kindly helped me in collecting the data. I would also like to thank all my fellow researchers at Signal and Image Processing laboratory at UBC, in particular, Dr. Hamid Palangi for fruitful discussions on deep learning. I thank the UBC Graduate Studies for their financial support through UBC Support Initiative (GSI) Awards. xxi I wholeheartedly thank my friends for being there for me during the ups and downs in the course of my study. I thank my beloved siblings, Farzane and Ehsan, for their ongoing care during all these years. I am grateful to my dearest mother, Sedigheh, for always supporting my dreams and teaching me to believe in myself since I was a little girl, as well as to my lovely father, Ali, for continuously empowering me with his unconditional love. I also owe the deepest gratitude to my best friend of all time, Sardar, for his emotional support as a wonderful partner, and insightful feedbacks as a brilliant researcher, in every step that I took since the beginning of my Ph.D. work. His love and encouragement has carried me through this journey. xxii Dedication To my parents 1 Chapter 1: Introduction The aim of this thesis is to investigate the feasibility of implementing in vivo imaging and to incorporate quantitative digital pathology approaches to improve cervical cancer screening and diagnosis. The focus is on the application of confocal microscopy and advanced image analysis algorithms in cervical cancer diagnosis. The methods and approaches however could be easily modified and extended to other types of cancer. There are optical technologies that could generate cellular level images of the tissue and provide clinicians with the ability to visualize the changes in the tissue in real time. Recent advances in these technologies have improved both the image spatial resolution and the range of molecular changes that can be visualized in the images taken in vivo. As an example, microscopy images acquired in an in vivo fashion is now capable of capturing changes in cell morphology and in tissue architecture. These are also done in real time. Such optical technologies could therefore be employed in cancer diagnosis procedure as they can guide gynecologists in selecting the areas in the cervix to get biopsies from. When more accurate information about the tissue could be provided in real time to the clinician, the number of unnecessary biopsies taken, and therefore the diagnosis cost would decrease. In the long term, improving such optical technologies will ultimately enable us to reach the microscope image resolution presently used in histology but in an in vivo fashion. This will thus eliminate the need for extracting biopsies before their ex vivo analysis of interpretations, which would also be particularly beneficial in low resource settings. 2 From another perspective, in recent years, the field of pathology (like other fields of medicine and biology) is relying on digital means. The diagnostic data have been transformed into digital format and advanced analysis tools for automatic quantification of digital diagnostic data have been developed. It also urged the need for developing image calibration tools that guarantees the uniformity of all the data used. The uniformity of the data is needed to ensure the reliability of obtained results from any clinical study, in particular cross-institutions studies, or the studies that are performed in a long period of time. Currently, there are whole slide scanners that scan tissue slides and convert them to digital images. These images could be captured in different focal planes. In this manner, we can now capture all the information that is embedded in an entire tissue slide, in a digital image. This could make the pathology workflow entirely digital, enabling any pathologist to review any patient’s images irrespective of the physical proximity of the histology lab and the pathologists. On the other hand, some advanced image processing and analysis algorithms that interpret digital images have already been successfully incorporated (LeCun et al. 2015; Krizhevsky et al. 2012; Russakovsky et al. 2015; Cireşan et al. 2013; Cireşan et al. 2012; Irshad et al. 2014). These tools and algorithms can now help the pathologists to quickly extract more morphological and architectural information and can bring their attention to high risk cases (e.g. by automatically detecting and excluding normal samples). These tools are also less prone to human errors and are sometimes capable of identifying patterns that are difficult or not possible to recognize by human perception but could be useful for screening and diagnosis of cancer (Sheikhzadeh et al. 2018). However, there still remains more work that needs to be done to ensure that the quality of the obtained digital images and the accuracy of image analysis and reviewing systems are good 3 enough to eliminate the need for reviewing the original slides (Gallas et al. 2014; Treanor et al. 2015; Gavrielides et al. 2015; Gavrielides et al. 2014). The importance of such investigations made healthcare regulatory authorities to invest on them. During a research fellowship, I collaborated with the scientists at U.S. Food and Drug Administration to perform a research study on this issue. The details of the study are not provided herein as it is out of the scope of this thesis. We aim to improve the current diagnosis related to cervical cancer by examining the use of in vivo imaging and by incorporating new quantitative approaches to screening and diagnosis protocols. The current cervical cancer diagnosis and screening procedure and issues related to them will be explained in the following sections. In the course of this PhD work, there was a well-established program (project) going on at the British Columbia Cancer Research Centre (BCCRC), funded by the National Institutes of Health (grant# P01–CA-82710) (Buys et al. 2012). The ultimate goal of this program was to integrate the advances in cancer biology and optical technology to develop cost-effective tools to aid in early detection of cervical cancer (Buys et al. 2012). This program allowed us to collect the data required in studying the feasibility and effectiveness of in vivo imaging and to perform quantitative histological image analysis study simultaneously as part of the above clinical trial. Although I focused on detecting cervical neoplasia (uncontrolled and abnormal growth of cells in the tissue), the work presented in this thesis could be easily extended for detecting neoplasia in other organs such as oral cavity, skin, and lung. 4 1.1 Cervical Cancer and Diagnosis/Staging Procedure Cervical cancer represents a significant global cancer threat particularly in low-to-middle income countries where the disease incidence is highest (GLOBOCAN; Thun et al. 2009). In 2015, there were 526,000 new cervical cancer cases worldwide and the number of cervical cancer deaths was 239,000. At the global level, one in 68 women developed cervical cancer during a lifetime. The occurrence increases to one in 24 women in less developed countries. Cervical cancer was the most commonly diagnosed cancer in 2015 for women in 11 countries, and the leading cause of cancer deaths amongst women in 50 countries (Fitzmaurice et al. 2017). Currently, the Papanicolaou (Pap)-smear is used for cervical cancer screening. Pap-smear aims to detect potentially precancerous changes in the cervix, caused by human papillomavirus (HPV) which is a sexually transmitted DNA virus. Infection for a long time with certain types of HPV can cause cervical cancer (Bosch et al. 1995). Patients with an abnormal Pap-smear are referred to the colposcopy clinic for disease diagnosis and staging. Figure 1.1 illustrates the workflow of cervical cancer diagnosis in the clinic followed nowadays. In the colposcopy clinic, a gynecologist examines the cervix using a colposcope to detect the areas with potential precancerous lesions. Biopsies from these and other abnormal areas are then extracted. Finally, a pathologist reviews these colposcopically-guided biopsy specimens using a microscope and determines high-risk precancerous cervical lesions. 5 Figure 1.1. Current workflow of cervical cancer diagnosis. First, patients with abnormal Pap-smear are referred to the colposcopy clinic. In the clinic, after examining the cervix by colposcopy, the physician takes biopsies from abnormal areas. In the pathology lab, histological sections of the biopsies are cut, stained and mounted on slides. The pathologists review these slides under a light microscope and determine disease grade. In the past decades, advances in prevention and screening in cervical cancer have greatly reduced the mortality rate. Despite these advances, there is still a need in accurate and cost-effective tests for screening, in lower resource settings. Even for developed nations with cervical screening infrastructure, improved testing accuracy and reduced screening costs would have significant positive impacts on their screening programs. 1.2 Motivation The current procedure described in Sec. 1.1 for screening and staging cervical cancer is costly and time consuming. Although the procedure relying on the Pap-smear has reduced the incidences of cervical cancer worldwide (Wright et al. 2002), it has the disadvantage of producing many false positives. This low specificity results in an excessive number of colposcopy procedures performed. On the other hand, colposcopy examination of the cervix (for detecting precancerous lesions and guiding the biopsy selection) also suffers from a relatively Colposcopy ClinicExamination of cervix by colposcopyAbnormal BiopsyDiagnostic BiopsiesGynaecologist collects biopsies based on colposcopy impression of cervixBiopsy SectionsSections of each biopsies are stained and mounted on glass slidesPathologist reviews slides under microscopeDisease grading by human inspectionPathology LabScreeningPatients with abnormal Pap-smears are referred to colposcopy clinic6 low specificity. The accuracy of colposcopy is highly dependent on the physician’s expertise. This low specificity results in the acquisition of a large number of unnecessary biopsies (Adelman 2014). In particular, colposcopy has been reported to have a high sensitivity (98 %) and a low specificity (45 %) in differentiating between high-grade abnormal tissues and normal tissues (Cantor et al. 2008). Therefore, there is room to improve the effectiveness of this system. One approach for improving the effectiveness of the described system is to enhance the optical technologies that are currently used for detecting abnormal lesions in the clinic (i.e. colposcopy). Such enhancement could in the long term future lead to a new generation of diagnosis system. Enhanced optical technologies could be employed in the clinic in the near future to automatically identify cervical lesions with high sensitivity and specificity, potentially reducing the frequency of unnecessary biopsies. This improved biopsy selection guidance would prevent the discomfort caused for the patients by taking unnecessary biopsies, and save the cost and time spent for processing such biopsies. In the long term, these enhanced optical technologies will be employed in the clinic to: i) perform the optical measurements in a non-invasive manner (i.e. removing the need for extracting biopsies), ii) provide real time diagnosis, and iii) result in possibility of immediate treatment along with the diagnosis in the clinic. In the last 10 years, new imaging and optical technologies have been developed in an attempt to improve standard colposcopic examination of uterus cervix. These include confocal microscopy (Tan et al. 2009; Quinn et al. 2012), optical coherence tomography (Gallwas et al. 2011) and Raman microscopy (Rashid et al. 2014). Among these new technologies, confocal microcopy, either in reflectance or fluorescence mode, has been under development for almost two decades. 7 Confocal microscopy is a non-invasive tool that can interrogate epithelial tissues (the tissue type that covers all the exposed surfaces of the body) to provide readouts related to epithelial changes. This relies on acquiring multiple images of cell nuclei at different tissue depths. These images have sufficient contrast and resolution to allow visualization of individual cells and nuclei. This capability of confocal microscopy has driven its application into a variety of clinical contexts (Jester et al. 1991; Rajadhyaksha et al. 1999; White et al. 1999; El Hallani et al. 2013; V. et al. 2015). Researchers employ confocal microscopy to obtain information about cell morphology and tissue architecture in a non-invasive fashion. The ability of reflectance and fluorescence microscopy in detecting Cervical Intraepithelial Neoplasia (CIN) has also been evaluated by different teams (Collier et al. 2002; K. Carlson et al. 2005; Collier et al. 2007; A. L. Carlson et al. 2007; Tan et al. 2009; Quinn et al. 2012). Besides developing more powerful optical technologies, another approach that can enhance the effectiveness of cervical cancer diagnosis is that of improving the tools that are currently used in examining tissue sections. Such improvements can lead to a decrease in required labor and time, and an improvement in the accuracy of pathology diagnosis. One established diagnostic tool in the field of pathology is the quantitative analysis of tissue sections (Anderson et al. 1997; Johnson et al. 2004). This approach relies on the quantitative assessment of the cell morphology, tissue architecture, and labeled targets in the tissue in two dimensional (2D) microscopy images obtained from the tissue slides (which are called histological images). The precancerous process is characterized by cell morphology and tissue architecture changes. These changes include an increase in the size of the nucleus in the cell, an increase in the ratio of the nucleus to the 8 cytoplasm in the cells, a decrease in the distances between the cells, and a decrease in the differentiation between cell types in the tissue (Blaustein and Kurman 2002). Recently, many microscopy imaging systems have been developed to obtain digital images of tissue slides. Particularly, whole slide scanners have enabled fast and automatic acquisition of high quality images of the entire slide. The resulting digital images, called Whole Slide Images (WSI), are analyzed to obtain objective measurements of morphologic and architectural properties of the tissue sections or quantify the characteristics of labeled targets in tissue samples. All these analysis types could lead to a reliable disease grading in the pathology review process. But most of them are currently performed manually, imposing a great cost to the screening and diagnosis systems. Therefore, developing automatic tools for such analyses is needed. For example, quantitative assessment of different labeled targets (which could be an indicator of different biological states) in a tissue section demands a large amount of time and effort from the pathologist. This has motivated me to develop a fully automatic framework in my PhD, for assessing labeled targets (which were proliferating cells in an HPV positive cervical tissue in our application) in WSIs of cervical tissue sections. Employing automatic quantitative tools could greatly benefit all types of cancer (including cervical cancer) screening and diagnosis procedure; these tools could provide pathology interpretations in a fast and cost-effective manner. However, the development of fully automated tools for performing different analyses in histological images (including WSIs) is still a challenge. This is because of the inherent variability in tissue specimens, their processing and staining procedures. For example, the robust segmentation of cell nuclei in these images is 9 challenging, because different nuclei often appear as touching or overlapping. The complexity of histological images, in addition to the large quantity of images generated in recent years by using whole slide scanners in the field of pathology, makes machine/deep learning based algorithms suitable for analyzing such images. Since the breakthrough advancements in deep learning algorithms in 2012 (Cireşan et al. 2012; Krizhevsky et al. 2012), which made these algorithms the state of the art in the field of computer vision, researchers have performed some works applying these algorithms in analyzing histological images (Cireşan et al. 2013; Wang et al. 2014; Janowczyk and Madabhushi 2016; Xu et al. 2016; Litjens et al. 2016; Han et al. 2017). 1.3 Research Objective In this thesis, we aim at developing new imaging and image analysis tools for improving cervical cancer screening and diagnosis procedures. We have explored two different approaches to improve the conventional methods explained above (see Figure 1.2A):  the implementation of in vivo confocal imaging in the clinic to enhance the accuracy of colposcopy examination  the incorporation of quantitative digital pathology techniques for histological images analysis to aid pathologists in detecting the disease grade The first part of this thesis is focused on improving cervix examination and biopsy selection in the clinic (see Figure 1.2, middle block). We study the potential of developing an in vivo imaging system that could be performed in the clinic in a non-invasive manner, to automatically identify precancerous lesions with higher sensitivity and specificity compared to colposcopy alone. Employing such an in vivo imaging tool in the clinic could potentially reduce the 10 frequency of unnecessary biopsies (Figure 1.2B). In the long term, reaching more advanced in vivo imaging technologies that match the microscope image resolution presently used in histology would enable clinicians to interrogate the tissue in an in vivo fashion. This would eliminate the need for extracting biopsies thus providing a real time diagnosis in the clinic, with the possibility of immediate treatment (Figure 1.2C). The other part of our work is focused on developing automatic tools for analyzing biopsy sections from their microscopy images (i.e. histological images) captured by a whole slide scanner or any conventional microscopy imaging system (see Figure 1.2B, right block). By combining image processing algorithms with machine learning techniques, we have developed quantitative tools for analysis of histological images. These new analysis tools could help the pathologists to better identify high risk precancerous lesions, and therefore improve the current cancer diagnosis procedure. 11 Figure 1.2. Potential improvements to cervical cancer diagnosis procedure. (A) current workflow of cervical cancer diagnosis. (B) Improvements proposed in this thesis. In the clinic: in vivo confocal microscopy could be used to reduce the frequency of unnecessary biopsies. From the pathology perspective: whole slide scanners could be used for obtaining digital images of histological sections cut from biopsies. Advanced image processing algorithm could be employed for automatic quantitative analysis of these images in a fast manner which ultimately leads to better identifying high risk precancerous lesions. (C) Future potential cervical ABCColposcopy ClinicExamination of cervix by colposcopyAbnormal BiopsyDiagnostic BiopsiesGynaecologist collects biopsies based on colposcopy impression of cervixBiopsy SectionsSections of each biopsies are stained and mounted on glass slidesPathologist reviews slides under microscopeDisease grading by human inspectionPathology LabScreeningPatients with abnormal Pap-smears are referred to colposcopy clinicColposcopy Clinicin-vivo confocal microscopy imagingAbnormal AreaBiopsy SectionsSections of each biopsies are stained and mounted on glass slidesPathologists review images and information obtained from analysisDisease gradingPathology LabAutomatic quantitative analysis of confocal imagesMicroscopy images of biopsy sectionsImages are obtained by a whole slide scannerFast and automatic quantitative analysis of whole slide imagesImproved biopsy selection guidanceDiagnostic BiopsiesScreeningPatients with abnormal Pap-smears are referred to colposcopy clinicColposcopy ClinicAbnormal AreaAutomatic quantitative analysis of confocal imagesReal time disease gradingTreatmentin vivo confocal microscopy imaging of abnormal areasExamination of cervix12 cancer diagnosis system. Advances in in vivo imaging will enable the visualization of the cervical tissue with a similar resolution to the resolution of histological images. This would eliminate the need for extracting unnecessary biopsies and reviewing them in the pathology lab, provide real time diagnosis, which ultimately could result in immediate treatment in the clinic. 1.4 Research Scope To address the objective of this thesis including improving the current disease grading procedure in cervical cancer, the following tasks have been undertaken:  Acquiring data and digitizing tissue images. To collect the data used in my thesis, I collaborated with a team of gynecologists and nurses to establish a pipeline for acquiring cervical biopsies. As part of a large clinical trial (Buys et al. 2012), we obtained biopsies from patients with a history of abnormal Pap-smear test. I acquired confocal microscopy images from each biopsy and after imaging transferred the biopsy to the pathology lab. In the pathology lab a pathologist cut the biopsy into sections, examined the sections under microscope, and provided a disease grade for each biopsy. Each of the biopsy sections was dyed with different stains (including H&E, Thionin-Feulgen, and CINtec) for use in different purposes. Afterwards, with the help of two cyto-technicians I acquired microscopy images of the biopsy sections. The details of the clinical trial, staining and image acquisition will be provided in Chapter 2. All data presented herein were obtained from the biopsies acquired in the course of this thesis. Part of the data and images acquired are available to the public in the following link: https://osf.io/c74wu/ 13  Obtaining digital images for each biopsy. We acquired two different image types for each biopsy to provide a thorough dataset to enable us to explore the two proposed studies for improving cervical cancer diagnosis. The first image type was confocal images of the biopsies. Confocal imaging allowed us to acquire multiple images at different depths of the tissue without the need to section the biopsy. These images were obtained ex vivo using confocal fluorescence microscopy. The second image type was the microscopy images of different sections cut from each biopsy (i.e. histological images). As will be explained in Chapter 2, each of these biopsy sections has been stained and imaged for a specific purpose in our studies. These images could be taken in absorbance mode and are obtained either by a camera mounted on a light microscope, or by a commercial whole slide scanner. Using this data set, I explored two approaches for improving the effectiveness of current cervical cancer diagnosis. As described in research objectives in Sec. 1.3, I studied: i) the utility of confocal imaging of the tissue for detecting high-grade lesions in an in vivo fashion, and ii) the potential of incorporating automatic tools for quantitative analysis of histological images.  Distortion and resolution uniformity calibration. An image obtained from a microscopy imaging system may have distortions and its resolution may not be uniform. i.e. an image may not have the exact correspondence with the actual physical sizes of different objects in the tissue. To correct for all non-uniformities, a calibration imaging slide known as CI-Slide was used. This slide has been designed by the researchers at BCCRC, Imaging Unit. However, this slide was never been used before, and its characteristics and performance in applications was not studied and verified. I characterized the properties 14 of the CI-Slide and used it to control the quality of images collected in our studies. Such a calibration tool is necessary for validating the accuracy and reliability of the data acquired for the study. This is especially important in cross labs/hospitals and multi-centers studies where multiple imaging systems are used during a long period of time.  Detecting cervical intraepithelial neoplasia by quantitative confocal microscopy. We studied the feasibility of employing confocal imaging in an in vivo fashion for differentiating between high-grade and low-grade cervical lesions (to improve the standard cervical colposcopic examination). It is worth mentioning that based on the findings and results of our study (Sheikhzadeh, Ward et al. 2015; Sheikhzadeh et al. 2015) other researchers could implement this technology in the clinic for cervical examination (Schlosser et al. 2016). The tasks that have been undertaken in our study are as below: a. Identifying features in the confocal images of each biopsy and the microscopy images of its sections. First, to quantify cell morphology and tissue architecture in confocal images of a cervical biopsy I identified four features (amongst 200 features). Then, to ensure that similar features could quantify cell morphology and tissue architecture in microscopy images of biopsy sections (i.e histological images), I calculated the same four features in these images. The features in confocal images of the biopsy correlated with those quantified histological images. Of course, the obvious advantage of confocal images over histological images is that confocal images could be in the future obtained in vivo (directly from the cervix uteri) and therefore could provide information about cells and tissue structure in a less-invasive manner. 15 b. Evaluating the effectiveness of confocal microscopy. To determine whether quantitative analysis of images obtained by confocal fluorescence microscopy is an effective tool for differentiating between normal cervical tissues, low-grade dysplasia, and high-grade dysplasia, I analyzed the confocal imaging results. The ground truth for the disease grade was given by the pathologist. After confocal imaging, for each biopsy, the pathologist assessed biopsy sections (stained with H&E) under a light microscope and provided the disease grade. Additionally, I extended this evaluation to cases where we studied confocal images of the same biopsy taken at different tissue depths. This extended evaluation plays an important role in utilizing in vivo confocal imaging, because assessing the exact depth of imaging in vivo will be challenging. Despite the importance of this evaluation, it has never been done before. c. Determining the accuracy of high-grade dysplasia detection via confocal microscopy. The accuracy of detection resulted from confocal microscopy is compared to the accuracy of detection obtained by colposcopic impression (which is currently used in clinical settings for detecting high-grade cervical lesions). As will be shown in Chapter 3, confocal microscopy is more accurate in detecting high-risk precancerous lesions. The number of normal or benign tissue areas identified as high-grade lesions (i.e. false positives) were smaller in confocal microscopy compared to colposcopy (in our data set, the specificity of confocal microscopy was 96% vs. 58% for colposcopy). 16  Quantifying the proportion of proliferating cells in each biopsy section by immunohistochemical analysis. a. Developing algorithms for a fully automatic quantitative immunohistochemical analysis. An increased percentage of cell proliferation (an increase in the number of cells) in a biopsy section is an indicator of a high-risk precancerous lesion. In addition, infection with certain types of HPV can lead to cervical cancer. By immunohistochemistry (IHC) staining, we can visualize the proliferating cells and HPV positive cells in a tissue section. For each biopsy, we stained one biopsy section with CINtec which is an IHC stain. By CINtec staining, a biomarker in the nucleus that is associated with cell proliferation (Ki-67 protein), and a biomarker of HPV contamination in a cell (p16 protein) can be visualized. In the CINtec-stained sections, the proliferating cells appear in different colours, depending on whether the tissue is positive in HPV or not. As a result it is difficult or even sometimes impossible for human observers to detect proliferating cells in these sections. I developed a fully automatic framework for labeling and localizing these two biomarkers in a region of interest in an image of CINtec-stained slides (the image is obtained by a whole slide scanner). As will be discussed in details in Chapter 4, because differentiating between these biomarkers is a very challenging task, I used a deep learning-based approach in my analysis. My analysis framework is based on convolutional neural networks implemented at two levels; cellular and tissue levels. b. Validating the accuracy of the presented automatic algorithm. To evaluate the performance of the developed algorithm for labeling and locating the two biomarkers 17 in whole slide images of CINtec-stained tissue sections, I compared its results to: i) the results from biomarker labeling performed manually by experienced cyto-technicians, and ii) the results obtained from a colour deconvolution algorithm (which is being commonly used in analyzing such digital images in the field of pathology). As will be demonstrated in Chapter 4, the performance of the developed algorithm is comparable to manual labeling. Of course, the advantages of the algorithm are being very fast and fully automatic. In addition, we showed that our algorithm resulted in more reproducible results compared to the colour deconvolution algorithm. c. Assessing the percentage of proliferating cells in different epithelial layers. The distribution of proliferating cells in a cervical tissue section correlates with the grade of neoplasia. The greater proportion of cell proliferation in the upper layers of the tissue corresponds with higher grade of neoplasia (Guillaud et al. 2014). To evaluate the potential of our fully automatic framework in predicting the grade of cervical neoplasia, I defined a layer-based approach to analyze a region of interest in a WSI of a tissue section. This analysis enables assessing the proportion of cell proliferation in different epithelial layers of a tissue section. As the ground truth for disease grade, I used the diagnosis provided by the pathologist for each biopsy. 18 Figure 1.3. Samples of images acquired. (A) Confocal microscopy images obtained from the biopsy without cutting it to sections. Image plane is parallel to the surface of the tissue. As illustrated, confocal imaging is done at different depths from the tissue surface. (B) Microscopy images obtained from different biopsy sections. Sections are cut perpendicular to the tissue surface and stained for different purposes (images showed herein are obtained from Thionin-Feulgen and CINtec-stained sections). Sections are imaged either with a camera mounted on a light microscope or a whole slide scanner. Figure 1.3 shows a sample of confocal images obtained from a cervical biopsy, and microscopy images of its sections. During the program project at BCCRC we were able to collect all these images for all participating patients in the study. Confocal images of biopsies were used to investigate the potential of using in vivo confocal imaging for staging the different grades of cervical neoplasia. This is presented in details in Chapter 3. Microscopy images of biopsy sections stained with Thionin-Feulgen were analyzed to extract features capable of differentiating different neoplasia grade and as a gold standard to validate the results obtained ...z = z1z = z2z = znz = z1z = znz = z2Confocal Images of Biopsy Microscopy Images of Sections Cut from BiopsyA B19 from confocal microscopy in Chapter 3. Images of CINtec-stained sections were used to develop a deep learning-based algorithm for a fully automated system for assessing the proportion of cell proliferation in tissue sections in Chapter 4. 1.5 Thesis Structure The thesis is organized into five chapters. The current challenges in the diagnosis of cervical neoplasia and the underlying motivation for the current work are presented in Chapter 1. One of the main challenges in any medical related research is having a comprehensive dataset. In Chapter 2, I describe the clinical trial that we established to collect the data required for the studies presented in Chapters 3 and 4. Furthermore, in Chapter 2, a calibration tool is introduced and characterized for use in ensuring the accuracy and reliability of the data. As one approach to improve the current diagnosis system for cervical cancer diagnosis, the utility of confocal fluorescence microscopy to be used as an in vivo imaging tool for detection of high-grade cervical lesion is evaluated in Chapter 3. As another approach to improve the effectiveness of this system, in Chapter 4, an algorithm for a fully automatic quantitative assessment of cell proliferation and HPV contamination in cervical tissue sections is developed. Finally, in Chapter 5, contributions, limitations, suggested future works, possible applications and areas to which the presented approaches can be extended are briefly discussed. 20 Chapter 2: Specimen Collection, Data Acquisition and Quality Control This chapter describes the pipeline procedure we have designed and established for specimen collection and data acquisition. We first acquired cervical biopsies from patients with abnormal Pap smears. The biopsies were collected in the clinic after colposcopy reviewing of the cervix. This was followed by confocal fluorescence imaging of the biopsies. As will be described in Chapter 3, we used confocal images in assessing the potential of implementing in vivo confocal imaging for precancerous tissue detection. After the biopsies were images by confocal microscopy, the biopsies were cut to histological sections. For each biopsy, different sections were dyed with different stains (including H&E, Thionin-Feulgen, and CINtec) to be used for different purposes. The sections that were stained with H&E were reviewed by a pathologist under a light microscope to provide pathological diagnosis for the corresponding biopsies. Then, microscopy images of tissue sections were obtained, either by a camera mounted on a light microscope, or a whole slide scanner. Images of H&E sections were used to obtain the ground truth of the disease grade for each tissue section. We used images of Thionin-Feulgen sections to quantify cell morphology and tissue architecture properties in the tissue sections. Images of immuno-stained sections were used for cell proliferation assessment in the tissue as will be discussed in Chapter 4. Figure 2.1 outlines the design of the clinical trial and data acquisition described herein. 21 Figure 2.1. Specimen collection and data acquisition. For each patient, colposcopy impression of the cervix was recorded in the clinic and two biopsies from normal and abnormal areas were collected. ex vivo confocal images of the biopsies were obtained. Histological sections were cut from biopsies and different sections were stained with H&E, Thionin-Feulgen, and Immunostain. H&E sections were reviewed by a pathologist to obtain disease grade, the other sections were imaged to obtain histology and IHC images. In addition, to control the quality of obtained images we used Cancer Imaging (CI)-Slide which is a calibration imaging slide (Sheikhzadeh et al. 2014). Image obtained from microscopy imaging system may have different un-uniformities. For example, they could be distorted or their resolution may not be uniform (i.e. images may not have exact correspondence with actual physical tissue size). We used CI-Slide to correct for all non-uniformities. Colposcopy Impression of Cervix in the ClinicNormal BxAbnormal BxBiopsiesFluorescence Stained BiopsiesConfocal en face ImagesImmunostained SectionIHC Transverse ImagesThionin‐Feulgen Stained SectionHistology Transverse ImagesHistological SectionsH&E Stained SectionsDisease Grade22 2.1 Clinical Trial and Data Acquisition To collect the data we needed for the different studies described in this thesis, I collaborated with a team of gynecologists, nurses, and technicians to obtain cervical biopsy specimens during an ongoing clinical trial (Buys et al. 2012). I have been involved in acquisition of all the data presented in this thesis, which have been collected over a period of 22 months (April 2013- February 2015). These specimens were obtained from patients during their baseline visits for colposcopy review at the Women’s Clinic at Vancouver General Hospital. All patients gave informed consent and the study was approved by the UBC BCCA Research Ethics Board and the Vancouver Coastal Health Research Institute (Protocols H09-03303 and H03-61235). During the clinical trial I was mainly in charge of: i) arranging clinical sessions with the supervisor nurse in the clinic, ii) selecting which patients to invite to the study, iii) attending all clinic sessions to collect the biopsies acquired by the gynecologist and the data relevant to the study, iv) acquiring ex vivo confocal images of the biopsies, v) arranging microscopy image acquisition of biopsy sections with the help of two cyto-technicians. 2.1.1 Patient Recruitment and Specimen Collection Patients whose prior Pap-smear test results indicated abnormality had been referred to the clinic. Based on the Bethesda system (Solomon et al. 2002), the abnormality results would have included low-grade or high-grade dysplasia with lesions on the cervix. All the patients that had been referred to the clinic were later diagnosed as having low-grade CIN, high-grade CIN, or no abnormality (normal cervix) (Richart 1990). We considered the following eligibility criteria for considering patients to participate in our study: the patient had to be fluent in English to fully understand our explanations about the study without a need for a translator, and read and 23 understand the consent form; the patient had to be an initial visit patient with abnormal Pap-smear test suggesting low-grade or high-grade cervical dysplasia; and the patient had to be identified with a lesion on the cervix after being examined by the gynecologist using a colposcope. As shown in Table 2.1, from a total of 143 scheduled patients in 62 clinic days, 50 patients were eligible for the clinical trial and had accepted to participate (rate of recruitment = 34%). Three of these 50 patients were excluded from the trial due to unsuccessful image acquisition. Forty seven patients were successfully recruited and normal and abnormal biopsies were collected from them. Participating patients were 20 to 51 years old with an average age of 31.7 years (SD = 8). None of these women were pregnant. Clinic Days Total Scheduled Patients Patients Recruited No Patient Days (no show/declined) Rate of Recruitment (#accept/scheduled) 62 143 50 15 34% Table 2.1 Summary of number of scheduled and recruited patients in the clinical trial. For each of the women, following the topical application of 5% acetic acid to the cervix, a trained gynecologist identified normal and abnormal uterine cervix areas by colposcopic reviewing of the cervix. During this examination, the gynecologist used a biopsy forceps to collect the uterine cervix punch biopsies. For each patient, both normal and abnormal biopsies were collected. I immediately placed the biopsy specimens in saline (0.9% Sodium Chloride solution) after they were collected. I also took the note of colposcopic impression (normal, low-grade CIN, or high-grade CIN) provided by the gynecologist for each biopsy. Each biopsy specimen was ~4 mm in diameter and ~2 mm in depth. 24 2.1.2 Specimen Processing and Staining Biopsies obtained in the clinic were transferred to the imaging lab and underwent ex vivo fluorescence confocal imaging. To prepare the biopsies for the confocal imaging, they were stained with an Acriflavine fluorescence stain for 2 minutes on a shaker at low speed as described in (El Hallani et al. 2013). This imaging was carried in less than one hour after biopsies were collected. Detailed procedure of staining and imaging are provided in Chapter 3, Section 3.2. After acquiring confocal images from the biopsies, the specimens were immersed in formalin fixative and transferred to British Columbia Cancer Agency (BCCA) hospital pathology lab. Biopsy specimens were then embedded in paraffin and nine transverse sections (with the thickness of 4-μm) were cut from each biopsy and mounted on slides. Three of these nine slides which we denoted as slides 1, 5, and 9 were stained with H&E. These H&E stained sections were reviewed by an expert pathologist to establish the disease grade. These three slides were considered as clinical slides and kept at BCCA as part of the patient clinical record. Slide 2 was stained with Thionin–Feulgen stain, which is a stoichiometric stain of DNA (Garner 1994), but slides 3, 4, 6, 7 and 8 were remained unstained. Then, the review results (including normal, reactive atypia, low-grade CIN [CIN1], and high-grade CIN [CIN2 and CIN3]), the Thionin-Feulgen stained slides, and the unstained slides were returned to us for further study. In our study, we used the pathological diagnosis review results as the ground truth for the studies presented in Chapter 3 and 4. We scanned the thionin-feulgen slides using an in-house high resolution imaging system Getafics (Kamalov et al. 2005a). Details of the imaging system and 25 the procedure are provided in Section 2.2.1.2. In chapter 3, we discuss how these images were analyzed and used as one of the means to evaluate the potential of confocal microscopy imaging in identifying high-grade lesions. In chapter 4, we discuss how we also employed these images in to help identify the regions with the worst diagnosis on IHC images. Ultimately, slide 3 was stained with CINtec PLUS kit (Roche mtm laboratories, Mannheim, Germany) for p16/Ki-67 dual immunostaining. The staining was done according to the manufacturer’s instruction, and then the slide was counterstained with Hematoxylin. This slide was used for immunohistochemical analysis that is presented in Chapter 4. We did not use slides 4, 6, 7, and 8 in the study presented in this thesis. Those slides are kept at BCCRC for use in other studies. 2.2 CI-Slide, a Calibration Imaging Slide Numerous commercial whole-slide scanners exist in the field of pathology. Their purpose is to convert glass slides to digital slides, enabling the acquisition of cytological and histological images of the samples by the scanners. Analysis of these digital images allows us to objectively measure the histo-morphological and architectural properties of the tissue. The morphological properties of the nuclei in the tissue and the architectural properties of the tissue are considered by pathologists to determine the presence of malignancies (Webster and Dunstan 2014; Hipp et al. 2011). As is the case in any new medical technology, digital pathology faces numerous challenges. One main challenge is to determine the real characteristics of the imaging systems in practice. This is important because multiple brands of imaging systems and scanners can be installed and used in the same lab; multi centre trials involve different imaging systems used by 26 different canters; studies can last years and the characteristics of the imaging system could change by time. Therefore, we should be able to measure the characteristics of the imaging system at any time. We also need to examine if the feature measurements that are commonly used for diagnosis are robust against the characteristics of different imaging systems. In the past, standardized reference slides (in particular the PRESS-PRO21slide designed and developed by ESACP (Giroud et al. 1998)) were used worldwide in the quantitative evaluation and calibration of image cytometers. For the last 15 years, the imaging unit at BCCRC has been involved in the development of several new optical and imaging technologies for cervical neoplasia (Cantor et al. 2011). As part of the international multi-center clinical trials, they have developed new algorithms for the quantification of histological and cytological changes in cervical pre-neoplastic lesions (Guillaud et al. 2004). The PRESS-PRO21slide was used for the calibration of imaging systems in these studies and a specific quality control process was developed using this slide (Chiu et al. 2004). Unfortunately, this slide was not designed for the new technologies like whole-slide scanners. There are some missing features on this slide: e.g. the presence of any object that has different pitches, patterns that mimic the nuclear texture, objects with the same pattern on both sides of the slide which can be used to quantify the uniformity of the scanning process, and resolution targets. For this reason, BCCRC has designed and developed a new and more comprehensive slide, the CI-Slide for calibrating any imaging microscopy devices as well as all new whole-slide scanners. The overall objective behind designing this slide was to develop a quality control protocol, for the assessment, calibration and comparison of any imaging systems and whole slide scanners. To help with the development of a quality control protocol using CI-Slide, we analyzed the characteristics of this slide. The related analyses are presented in this section. 27 2.2.1 Experimental Design for Analyzing the Characteristics of CI-Slide We use four identical CI-Slides in our analysis. This is to investigate the variations in the measurements that could be caused by using different CI-Slides. For CI-Slide to be used in different labs around the world for calibration imaging systems, it is very important to ensure that all manufactured CI-Slides have the same properties and characteristics. We scanned four identical CI-Slides with Getafics and a Leica Biosystem’s whole slide scanner. The detailed information about these imaging systems is provided in Sec. 2.2.1.2. We processed the high resolution images and segmented the objects in these images and then calculated about 200 photometric, morphologic and architectural features. Finally, we conducted statistical analysis, to investigate whether the measurements obtained by the CI-Slide were repeatable and reproducible. We also used four of the calculated features, to show how different factors such as objective magnification, transmission, object morphology, and architectural patterns affect the feature measurements from different systems. 2.2.1.1 CI-Slide Design The CI-Slide is a proprietary slide of BCCRC. Figure 2.2 shows the layout of the slide. The slide is composed of 138 boxes (square) each having a side length of 1.6 mm. We define each box by the shape, size, transmission, and pitch of objects of that box; e.g. a box that contains circles is called a circle box. As shown in Figure 2.2, the rows are named from A to L, and the columns are numbered from 00 to 18. Outside each box, below the bottom-right corner, an index identifies the row and column the box belongs to. These indices are not shown in Fig. 2.2. 28 Figure 2.2. CI-Slide layout; there is a total of 138 boxes on the slide, containing dark objects with known morphologic and photometric characteristics and overall architectural patterns, on a bright background. Each box contains dark objects with known photometric and morphologic characteristics, and overall architectural patterns, on a bright background. Figure 2.3. shows the patterns in six representative boxes. Figure 2.3. Boxes containing objects with different shapes: (A) circle, (B) oval, (C) square, (D) Siemens stars, (E) United States Air Force (USAF) resolution test target, (F) crosses. A B CD E F29 The left half of the slide is designed to be the mirror of the right half. For example for each box on the left side, there is a similar box (in terms of object size and pitch and transmission) on the right side. Due to the acquisition method used, some scanners have limitations in accurate focusing and in acquiring high quality image continuously throughout the slide (Montalto et al. 2011). The boxes on the two sides of the slide that are similar can be used to determine the uniformity of line scanning. There is a total of 90 boxes containing circles, with 45 permutations: 3 radii (2 µm, 3 µm or 5 µm), 3 pitch/radius ratios (3, 4, and 6), and 5 transmission steps (from 10% to 90% in 20% step). Boxes containing circles are located on rows A-E and H-L, and columns 01-03, 05-06, 12-13, and 16-17. The circles in the boxes on each row have the same size and pitch (Table 2.2.). In each row, from column 01 to 06 (excluding column 04) the transmission changes from 90% to 10% in 20% steps. From column 12 to 17 (excluding column 14) the transmission changes from 10% to 90% in 20% steps. Row Index A B C D E H J K L Radius (µm) 2 2 2 3 3 3 5 5 5 Pitch (µm) 6 9 12 9 13 18 15 21 30 Number 4100 1900 1000 1700 900 500 700 400 200 Table 2.2. Size, pitch, and number of circles in different circle boxes. Columns 04 and 14 contain 12 boxes that containoval shapes with 6 permutations: 3 minor radii and 2 major radius/minor radius ratios; ovals are rotated randomly by 0o, 45o, 90o or 135o. Row 30 G contains 10 boxes of squares with a side length and pitch of 40 µm. The transmission varies from column 01 to 06 (excluding 04) from 90% to 10% in 20% steps, and from column12 to 7 (excluding 14) from 10% to 90%. In addition to these 112 boxes, in the center of the slide on row F, there are 10 blank boxes that vary only in transmission. A United States Air Force (USAF) resolution test target (fitted within the 1.6 mm square), and two boxes containing Siemens stars and crosses are located in each corner and in the middle of the slide. Running along the top and left sides of the slide, there are horizontal and vertical rulers with increments of 1 micron. As discussed, there are many objects with different photometric, morphologic, and architectural characteristics designed on the CI-Slide. Therefore, the slide could be used in different studies and imaging system investigations. Since the morphology of nuclei in histological images is usually the same as the morphology of circles, and as we mainly study nuclei in the following chapters we only show and discuss the results we obtained from circle boxes. We conduct our study using 4 manufactured CI-Slides, with the same design explained above. Analyzing four similar slides helps us to study the variations in our measurements that might be caused by different CI-Slides. 2.2.1.2 Imaging Devices We scan the slides in absorbance mode by Getafics (Kamalov et al. 2005), and by the commercial whole-slide scanner. By analyzing the images obtained from two different imaging systems we could study the variations in histo-morphological and architectural measurements that might be caused by different systems. 31 Getafics uses a 12-bit double correlated sampling MicroImager 1400 digital camera (pixels 6.8μm2) placed in the primary image plane of the microscope. The system also uses a 20X 0.75 NA Plan Apo objective lens (Omega Optical; Brattleboro, VT) and its resolution is 0.58μm. Getafics is operated by Dunit which is a Java-based software developed by the Imaging Unit at BCCRC. Using Dunit, we can move the microscope objective in Getafics to the region of interest on the slide (e.g. a box on the slide), and grab the image. As part of the Getafics set-up, we calibrate the system before each session. The calibration steps includes: (1) visual checking of the interference filter, the numerical aperture of the condenser and the setting of the field iris, (2) controlling the Dunit version, (3) taking bright and dark background images and record their characteristics (mean, standard deviation, intensity, histogram symmetry, and uni-modality) and (4) checking the illumination power (voltage and current). The commercial whole-slide scanner we use in this study is a line scanning system. The system uses a 20X/0.75 NA Plan Apo magnification. The image capture magnification is 20X or 40X (20X with 2X magnification changer) and the image capture region is 22.9×57.9 mm or 45.8×54.9 mm (H×W). The pixel resolutions of the resultant images are 0.496 µm/pixel (20X) and 0.247 µm/pixel (40X). Herein, we refer to this system as the Scanner. We use Imagescope which is a software provided by Leica Biosystems to select the region of interest on the image generated and save it in TIFF format. 2.2.1.3 Image Analysis We use our software Dunit to semi-automatically segment the images obtained from Getafics and the Scanner. The software uses the thresholding method for segmentation. In Dunit, the user 32 manually sets the threshold values for intentisy, to be used for this segmentation of nuclei (in the grey level image). As expected, the segmentation results are affected by this value. Figure 2.4 illustrates the different steps of the image analysis algorithm. 33 Figure 2.4. Semi automated analysis of the images from the scans using Dunit. (A) A grey level image originally scanned by Getafics, (B) after manual thresholding based on intensity, (C) after segmentation, (D) resulting image after manual correction: in blue are objects correctly segmented and in green are objects with bad segmentation, (E) interface of Dunit used for controlling the segmentation quality and calculating the features. A BC DE34 2.2.1.4 Feature Calculation: Photometric, Morphology and Architecture Features We compute about 200 photometric, morphologic and architectural features (Doudkine et al. 1995) for the objects segmented on each digitized image, obtained from previous step. In this chapter, we present three of those features: one photometric feature (mean intensity of objects), one morphologic feature (object area), and one architectural feature (three nearest Delaunay neighbours distance of objects). Nuclei area and average cell distances are two of the most important features that show consistent difference between lesions of different malignant or aggressive potential (Guillaud et al. 2004).  Mean intensity The mean of the intensity function of object (ܫ)̅ is calculated as follows: ܫ ̅ ൌ ∑ ∑ ൫ܫ௜,௝Ω௜,௝൯ெ௝ୀଵ௅௜ୀଵܣ (2.1) where the size of the image that object belongs to is L×M pixels; ܫ௜,௝ is the intensity of the pixel at coordinates i and j; Ω is the object mask function that denotes the region of the image that is the object: if Pi,j belongs to the object then Ωi,j is 1, otherwise it is 0; and ܣ is the object area. We use the average of ܫ ̅of all objects in the image as a photometric feature and refer to it as Mean of Object Intensity. 35  Area Since we need to compare objects of different sizes and different patterns, instead of using the mean area of objects, we use the normalized error in the computed mean area of the objects in each image. The normalized error of area (ErN) is computed as follows: ܧݎே ൌ ܣݎ݁ܽ െ ܣݎ்݁ܽ௛ܣݎ்݁ܽ௛ ൈ 100 (2.2) where Area is the mean of area of the objects in micron, and AreaTh is the theoretical area of objects in this box.  Three Delaunay neighbors distance Given a set of points S in a plane, a Voronoi tessellation of S is the partition of the plane that associates a polygon V(p) with each point p of S in such a way that all locations in V(p) are closer to p than to any other point in S. Figure 2.5 shows the Voronoi tessellation of a set of points. Two points of S are defined to be Delaunay neighbours if their Voronoi tiles share a common boundary. When every pair of Delaunay neighbours is joined by a straight line, the resulting graph is called the Delaunay graph (Fig. 2.5). Three Delaunay neighbours distance, is the average of the distances between each point and its three nearest Delaunay neighbours (De Berg et al. 2008). In this thesis, we call this distance 3NDND. 36 Figure 2.5. (A) Voronoi tessellation of a set of points, and (B) its dual Delaunay graph. To build the Voronoi tessellation and the Delaunay graph, we use the centers of the segmented objects in each image as the set of points. The Voronoi tessellation is being used to extract structural features to quantify tissue architecture (Marcelpoil and Usson 1992; Honda 1978; Kayser and Stute 1989; Sudbø et al. 2000; Guillaud et al. 2014). 2.2.1.5 Repeatability and Reproducibility We examine the repeatability and reproducibility of our measurements by assessing the variations caused by the system and the variations caused by different CI-Slides. Repeatability is the variations in the repeated measurements of the same parameter when the same conditions are used to make the measurements. In our study, repeatability is the variations in the repeated measurements of the same feature in one slide, obtained with the same system and settings. Reproducibility is the variations that result when different conditions are used to make the measurements. In our study, reproducibility is the variations in measurement of the same feature, caused by calculating that feature using different slides and different system settings. Using Getafics we scan the slides in 5 different sessions. At the beginning of each session, we restart and calibrate Getafics, which may result in different system settings e.g. different background light values. The calibration steps includes: i) visual checking of the interference 37 filter, numerical aperture of the condenser, and setting of the field iris, ii) controlling the Dunit version, iii) taking bright and dark background images and record their characteristics (mean, standard deviation, intensity, histogram symmetry, and uni-modality) and iv) checking the illumination power (voltage and current). Before scanning each slide, we also re-calibrate the Getafics which might change the device settings. For Slide 4 in sessions 1 and 2, we repeat the scanning four and five times respectively, without changing the device settings. Figure 2.6. illustrates our described experimental design. Figure 2.6. Experimental design for the study of repeatability and reproducibility on Getafics. Four CI-Slides are scanned with Getafics at magnification 20X in five sessions. 2.2.1.6 Effects of Magnification, Transmission, Size and Density of Objects on Feature Measurements Herein, we study the effects of different factors on the measurements of the features described in Sec. 2.2.1.4. We want to investigate if any of these factors are introducing an error in feature values calculated. These factors included: objective magnification, transmission, object size, and objects’ pitch. Figure 2.7 illustrates the experimental design. Getafics at magnification 20XSlide 1 2 3Session 14Replicate 1 2 3 4 1 2 3 41 2 3 452 3 4 541414138 Figure 2.7. (A) Experimental design: images obtained from scanning 3 slides at one magnification using Getafics, and scanning 1 slide at two magnification using the commercial scanner, (B) schematic of object properties in different 45 boxes that contain circles. *P/S Ratio is the ratio of the pitch to the object size in each box. Boxes with the same P/S ratio are shown in the same colour. 2.2.2 Results and Discussion We scanned three slides with Getafics and Slide 4 with the scanner. We then processed these images, segmented the objects, and measured different features by using the segmented objects. In this section, we report the repeatability and reproducibility of these measurements. We also show how different parameters such as objective magnification, the morphology of objects, architectural patterns etc. cause variations in the feature measurements obtained from different slides or imaging systems. 2.2.2.1 Repeatability and Reproducibility In this section, we measured different features of the boxes that contain circles with 3 µm radius, 13 µm pitch, and 30% transmission. The circles quantitatively mimic the stained cell nuclei. Figure 2.8 shows the mean of object intensity vs. transmission for the three slides, scanned with Getafics. For each of object intensity and transmission variables, the values shown in the plot P/S Ratio* 3P/S Ratio 4P/S Ratio 6Transmission percentage10% 30% 50% 70% 90%Size 2 µm5 µm3 µmScanner20X 40X 4 4GetaficsCI Slide 1 2 3Magnification 20XA B39 are normalized to the range of that variable. For example, transmission values shown in Fig. 2.8 are the original transmission value from the slide, divided by the range of transmission. The slope of the regression line is 1.02 for Slide 1, 0.95 for Slide 2 and 1.01 for Slide 3. Figure 2.8. Relationship between mean of object intensity and transmission for the objects in the two boxes that contains circles with 3 µm radius, 13 µm pitch, and 30% transmission. Different colours illustrate this relationship in different slides. The values are normalized to the range for both variables. Figure 2.9. illustrates the density distribution plots of object area for the data design shown in Figure 2.6. We use R statistical software (version 3.0.1) and kernel density estimation with Gaussian kernel to draw the plots. 0 0.2 0.4 0.6 0.8 100.20.40.60.81TransmissionMean of Object Intensity Slide1Slide2Slide340 Figure 2.9. Repeatability and Reproducibility; the density plot of object Area calculated for circles in the box E05. (A) Four slides scanned with Getafics in five sessions (See Figure 2.6 for data design). The curves with the same color correspond to one slide. Solid curves show the results of the first session and the dotted curves show the results of repeats. (B) Slide 4 scanned with Getafics four times in one session. We model the area by a statistical linear model: ܣݎ݁ܽ ൌ ݄ܶ݁݋ݎ݁ݐ݈݅ܿܽ ܣݎ݁ܽ ൅ ܧ݂݂݁ܿݐ ݋݂ ݈ܵ݅݀݁ ൅ ܧ݂݂݁ܿݐ ݋݂ ܵ݁ݏݏ݅݋݊ ൅ ܧݎݎ݋ݎ (2.3) where Error is the sum of Lack of Fit and Pure Error. Lack of Fit represents the error in the linear model fit to the values. Pure Error is due to pure random errors, which in our study could 41 be approximated by the error in replicate measurements. Table 2.3 shows the ANOVA table which we calculated using Statistica (StasoftTM). Two-way main effects ANOVA and Type VI Sum of Squares (Hocking 2005) are used in this analysis. We use both Pure Error and Error to run the F-test to test the significance of the effect of Slide and the effect of Session. Effect SS df MS Compare to Pure Error Compare to Error F-value p-value F-value p-value Slide 0.409 3 0.137 6.588 0.019 1.64 0.241 Session 2.642 4 0.661 31.852 0.0001 7.93 0.003 Lack of Fit 0.688 3 0.229 Pure Error 0.145 7 0.020 Total 17 Table 2.3. Analysis of Variance; SS: the Sum of Squares, Df: the degree of freedom, MS: the Mean Squares. As shown in Table 2.3., the significance of the variation due to Slide (p-value of 0.24) is smaller than the significance of the variation due to Sessions (p-value of 0.003). If we compare to the variation from Error, the variation due to the factor Slide is not statistically significant (p > 0.05). 2.2.2.2 Effects of Magnification, Transmission, Object Size and Pitch on Feature Measurements We use box-and-whisker plots to illustrate the effects of magnification, transmission, object size and pitch on the normalized error in object area, ErN for circle boxes. The plots are generated by Variability Graph tool in Statistica (StasoftTM). Here, we are not comparing Getafics and 42 Scanner; plots from the two systems are shown together to demonstrate the overall trend in different systems. Figure 2.10. shows the ErN calculated for the circle boxes with 30% transmission. The images from Slide 4, scanned with Scanner at 20X and 40X were used to draw this plot. We observe that the magnification affects ErN. The variation of measurements at 40X is larger than at 20X. We shall further explore this effect in the future. In the following paragraphs, we will only present the results obtained at 20X. Figure 2.10. Effect of magnification on ErN measurements. The plot shows ErN of circle boxes of Slide 4, scanned with Scanner at magnification 20X and 40X (central point: median; box: the first and third quartiles; whiskers: minimum and maximum values.) Figure 2.11, 2.12, and 2.13 show ErN of circle boxes of Slides 1, 2, and 3 scanned with Getafics, and of Slide 4 scanned with Scanner. In Figure 2.11, we observe that ErN tends to decrease as the transmission increases. Larger transmission corresponds to brighter objects. 43 Therefore, the contrast between the background and the objects decreases as the transmission increases, this can cause the objects to be under-segmented in boxes with larger transmission. Figure 2.11. Effect of transmission on ErN . The plot shows ErN of circle boxes of the three slides (Slide 1, 2, 3) scanned with Getafics, and of the Slide 4, scanned with Scanner at magnification 20X (central point: median; box: the first and third quartiles; whiskers: minimum and maximum values). Figure 2.12 shows the ErN calculated for circle boxes with 30% transmission and 9 µm pitch. The plot suggests that the variation in measurements is smaller for 3 µm radius circles than for 2 µm radius circles. 44 Figure 2.12. Effect of object size on ErN measurements. The plot shows ErN of circle boxes with 30% transmission and 9µm pitch of the three slides scanned with Getafics, and Slide 4 scanned with Scanner at magnification 20X (central point: median; box: the first and third quartiles; whiskers: minimum and maximum values) Figure 2.13 shows the ErN of 3 µm radius circle boxes and 30% transmission. We observe that the central value (median) of ErN decreases as the pitch increases, but the variation increases. Both systems show the same trend. 45 Figure 2.13. Effect of pitch on ErN . The plot shows ErN of 3 µm radius circle boxes with 30% transmission for Scanner (based on one slide) and Getafics (based on 3 slides) at 20X (central point: median; box: the first and third quartiles; whiskers: minimum and maximum values). 2.2.2.3 Effect of Segmentation Algorithm on Feature Measurements In all previous experiments, we used one default segmentation settings to analyze the images from the two systems. We refer to this setting as the general setting. In this experiment, we modified the segmentation settings. We scanned Slide 1, 2, and 3 with Getafics at 20X. Using the scan from Slide 1, we found the threshold value that resulted in the smallest ErN for circles. Then, we used the same threshold value for the other 2 slides. Figure 2.14. shows the box plot for the general and optimized settings. We observe that the median of error decreases by optimizing the segmentation, and the variation remains about the same. The data shows that we can optimize the system and the segmentation settings to obtain the exact theoretical values of the area (the feature we chose to perform this study on) for the objects. 46 Figure 2.14. Effect of the segmentation algorithm. The plot shows ErN of 3 µm radius, 13 µm pitch circle boxes with 30% transmission for Getafics (based on 3 slides) at 20X (central point: median; box: the first and third quartiles; whiskers: minimum and maximum values). 2.2.2.4 Effect of Architectural Pattern on Feature Measurements Figure 2.15 shows the correlation between 3NDND and the pitch values of different circle boxes. The value of this feature is not expected to be equal to the pitch value. We observe that this feature has an approximate linear relationship with the pitch. The slopes of the regression lines are 0.91 for 10%, 30% and 50% transmissions, 0.92 for 70% transmission and 0.90 for 90% transmission. 47 PITCHThree Nearest Delaunay Neighbours Distance6 8 10 12 14 16 18 20 22 24 26 28 30681012141618202224262830TRANSMISSION:10%TRANSMISSION:30%TRANSMISSION:50%TRANSMISSION:70%TRANSMISSION:90% Figure 2.15. Relationship between three nearest Delaunay neighbours distance and different pitch values. The plot is for all the circle boxes of the three slides scanned with Getafics. Different transmissions are shown in different colors. 2.3 Summary and Conclusions In this chapter, we described the clinical trial that we established to collect the data used in this thesis. In all clinical studies, obtaining reliable data is an essential part of the study. During a period of 22 months, we collected uterine cervix biopsies of 50 patients. Hsitopatholoical reviews were performed by an expert pathologist. Four different imaging techniques were used during this PhD work: 1) colposcopic imaging, 2) confocal imaging, light microscopy imaging of 3) histological sections, and 4) immuno-stained sections. We provided a unique data set by acquiring different types of images. Later in this thesis, we will describe the different studies that we have done in parallel, using this dataset, to evaluate the performance of each imaging technique in differentiating between various disease grades. 48 We also presented the CI-Slide. This calibration slide was designed and developed in the Imaging Unit at BCCRC. It was specifically designed to quantify the characteristics of any imaging systems and whole-slide scanners. Herein, we scanned four CI-Slides with one imaging system and one commercial scanner and quantified the different patterns using photometric, morphologic and architecture features. We showed that feature measurements of the CI-Slide are repeatable and reproducible. We used these features to show how different factors such as objective magnification, transmission, object morphology, and architectural patterns affect the measurements from different systems. This type of study enables the development of a Statistical Process Control protocol using CI-Slide. However, in order to use such a protocol in real clinical settings, we need to extend the study to more patterns on CI-Slide and also investigate other whole-slide scanners and imaging systems. 49 Chapter 3: Developing in vivo Imaging Tools 3.1 Introduction The ultimate goal of this research is to improve the accuracy of testing and reduce the infrastructure costs in cancer screening. In the case of cervical cancer, screening via the Pap-smear has reduced the incidence of cervical cancer worldwide (Wright et al. 2002). The use of Pap-smear however results in low specificity in an excessive number of colposcopy procedures performed, which unfortunately, results in the acquisition of a large number of unnecessary biopsies. The accuracy of colposcopy is highly dependent on the physician’s expertise. As shown in previous analyses, colposcopic examinations perform well in a diagnostic setting but has a poor performance in a screening setting (Mitchell et al. 1998). In particular, colposcopy results have been reported to have a high sensitivity (96 %) and a low specificity (48 %), when differentiating between abnormal tissues (squamous intraepithelial lesions (SILs)) and normal tissues (normal squamous epithelia and inflammation) (Cantor et al. 2008). That is, it is highly beneficial to improve the effectiveness of the current testing system that relies on histopathological review for colposcopically-guided biopsy specimens (Adelman 2014). Confocal microscopy has previously been used to study changes related to the grade of Cervical Intraepithelial Neoplasia (CIN) lesions (Collier et al. 2002; K. Carlson et al. 2005; Collier et al. 2007; A. L. Carlson et al. 2007; Tan et al. 2009; Quinn et al. 2012). In these studies, features such as cell density, nuclear morphology (nuclear size and nuclear-cytoplasmic ratio), and tissue architecture (mean and minimum distance between each cell and its neighbors) are calculated from sample images to delineate disease states. With histopathology as the gold standard, these 50 studies have demonstrated the ability to discriminate high-grade dysplasia (CIN2 and CIN3) from low-grade dysplasia (HPV associated changes and CIN1) at high sensitivity (86%-100%) and specificity (62%-100%). For most of these earlier studies, analyses were undertaken manually or by qualitative assessment (Collier et al. 2002; K. Carlson et al. 2005; Collier et al. 2007; A. L. Carlson et al. 2007; Tan et al. 2009). Additionally, despite the importance of tissue architecture features in the histopathological diagnosis (Blaustein and Kurman 2002), such features have rarely been quantified to evaluate CIN lesions (Collier et al. 2007). It should be noted, however, that image quality for confocal reflectance microscopy, which has been used by Collier et al. (Collier et al. 2007), is significantly low compared to the image quality of confocal fluorescence microscopy. The images obtained by confocal reflectance microscopy represent the natural differences in refractive indices of cellular structures, while in confocal fluorescence microscopy the tissue is stained with a fluorescent contrast agent - fluorochrome - to improve the cellular contrast (El Hallani et al. 2013). In the recent years, two other studies have also investigated the feasibility of using endo-microscopy in clinical settings for screening or diagnostic purposes (Degueldre et al. 2015; Pierce et al. 2012). The resulting sensitivity and specificity values of these studies in detecting high-grade lesions, even in small cohorts of women, have confirmed the potential of endo-microscopy in real time in vivo pathology (Carignan and Yagi 2012). Nevertheless, more work needs to be done to assess the true value of this technology in clinical settings. In order to move optical technology from bench to clinic, we need to better measure the capability of confocal microscopy. In this context, the objective of this chapter is to validate the findings obtained by previous studies that used independent proprietary instrumentation and confirm their results 51 about the ability of confocal microcopy to identify high-grade lesions. The second objective is to investigate the effects of varying the imaging depth in the performance of this apparatus for the first time. Herein, for the confocal fluorescence microscopic images obtained from cervical biopsy specimens, we identify and quantify features related to nuclear morphology and tissue architecture. A bench-top confocal microscope was used to acquire the images at different depth below the epithelial surface of the biopsy tissue. We developed an image processing algorithm for analyzing the images and calculating the cell morphology and tissue architecture. By analyzing confocal imaging against the gold standard of histopathological interpretation, we provided our evaluation of confocal fluorescence microscopy as a quantitative tool for delineating normal cervical tissues, low-grade CIN, and high-grade CIN. We also quantitatively investigate the performance of confocal imaging at different epithelial depths. 3.2 Materials and Methods Figure 3.1 outlines the design of the study described herein. 52 Figure 3.1. Study design. Based on the colposcopic impression two biopsies are taken; one biopsy from an abnormal area and the other biopsy from a normal area. Confocal fluorescence microscopy images are acquired from all the biopsies, then histological section were cut from the biopsies and images of sections are obtained. Quantitative tissue phenotypic analysis of confocal images and histological images is performed. Correlation between these features and histopathological grades, and also correlation between the features and the colposcopic appearance of cervix are investigated. 3.2.1 Confocal Fluorescence Microscopy As described in Chapter2, section 2.1.1, we obtained biopsies from colposcopically normal and abnormal cervical sites of patients who were referred to the Wemon’s Clinic at Vancouver General Hospital for colposcopy review. Within 1 hour of collection, biopsies obtained in the clinic were taken at BCCRC where they were imaged by a confocal microscope. The biopsies Colposcopy appearance of cervix in the clinicNormal BxAbnormal BxBiopsiesConfocal en face imagesComparing accuracy for detecting dysplasiaHistological transverse imagesMeasuring morphological and architectural features53 were first stained with an Acriflavine fluorescence stain for 2 minutes on a shaker at low speed as described in the study presented by El Halani et al. (El Hallani et al. 2013). The stain was prepared by dissolving 0.05% Acriflavine Hydrochloride (Fluka) in 10% phosphate-buffered saline. Following staining biopsies were washed with saline for 1 min. A 5% solution of acetic acid was then added to each sample. Specimens were next placed on a microscope slide and cover-slipped. Confocal fluorescence microscopy was performed using a bench-top Carl Zeiss Axio Imager Z1 equipped with a custom laser-scanning confocal attachment. The custom confocal attachment employed a resonance scanner and galvanometer (Cambridge Technology) for laser scanning; an Avalanche photodiode (Hamamatsu) for detection; and a frame grabber (Matrox) to digitize the signal. Laser excitation was provided by a 488 nm laser (Coherent). All images were acquired using a 25X/0.80 water-immersion objective lens (lateral resolution of the system = 0.87µm). The time between staining and completion of confocal imaging was approximately 10 minutes. Confocal images were acquired as gray-scale (no colour) and were 512 × 512 pixels in size. En face images (i.e. images with the imaging plane parallel to the surface of epithelium) were acquired every 5µm. The first image of this image stack was acquired from just below the epithelial surface (z = 0µm) while the depth of the last image (z = 80µm) was limited by the working distance of the objective lens (250µm) less the thickness of the cover slip (170µm). 3.2.2 Employing CI-Slide for Image Calibration We employed the CI-Slide, described in Chapter 2, to address two major problems with the confocal imaging system. The first issue was with respect to images obtained by the confocal system being distorted. The second is with regard to obtaining accurate measurements (e.g. 54 measurements of the nuclei size, and the distances between cells measured in µm). The measurements from images are obtained in terms of pixels and not µm. Since we did not know the actual pixel size and the magnification of the confocal system, these values could not have been calculated in terms of µm. We used the CI-Slide to correct the geometric distortion in the confocal images, and to also calibrate the confocal system. The CI-Slide was designed at BCCRC for calibrating cytometry systems. The design of the slide is described in Section 2.2.1.1. Briefly, the layout of the slide consists of 138 boxes (squares) with each box having a side length of 1.6 mm. The boxes contain objects of known morphologic and photometric characteristics. Among these boxes, 10 contain square objects of known sizes and 4 boxes contain the Siemens star which is a resolution test target. Figure 3.2 illustrates a Siemens star and a box containing Siemens stars. Figure 3.2. (A) The Siemens star, and (B) a box containing Siemens stars (at row A and column 08 of the CI-Slide). We scanned one of the Siemens star with the confocal microscope system. After deriving the mask containing the star, we applied a distortion correction algorithm to obtain the transform 55 function so that the shape of the mask becomes a circle. This function was used to correct the confocal images. Figures 3.3a-d illustrate the correction for distortion and Fig. 3.3e-f show the correction for a sample confocal image. The nuclei in the confocal image before applying the correction have the distorted shape. This could cause significant errors in the calculation of features and in particular the morphological ones. Figure 3.3. Geometric distortion correction of confocal images using the CI-Slide: (A) A Siemens star scanned with the confocal imaging system, (B) the mask containing the star in Fig. 3.3A, (C) corrected mask, (D) corrected star shape, (E) a sample of confocal image before correction (distorted objects are marked by red arrows), and (F) the same image after correction (corrected objects are marked by green arrows). After correcting for distortion, we scanned a square box and obtained the size of the squares in pixels. By knowing the size of the squares in µm2, we obtained the pixel size for the system. 56 3.2.3 Confocal Image Analysis We developed the image processing algorithm using MATLAB (Release 2014b, The Mathworks Inc., Natick, MA, US). The objective was to detect and segment the cell nuclei from confocal images which are in TIFF format (Fig. 3.4A). Our segmentation algorithm consisted of two steps: first, the derivative of a Gaussian filter was applied to compute the intensity gradient of the image. Canny edge detector (Canny 1986) was then employed to detect the edges of the nuclei by finding local maxima of gradient (see Fig. 3.4B). Two thresholds are defined in the Canny edge detector for detecting strong and weak edges. We set the lower threshold as 50 and the higher threshold as 100 which were the ±25% of the mean of the grayscale value in our images. In the second step, a binary image of the edges which mostly represent the nuclei boundaries is obtained. An algorithm based on morphological reconstruction (imfill function in MATLAB) (Soille 2003), and two sets of image erosion and image dilation were used to fill the nuclei boundaries and segment the nuclei (see Fig. 3.4C). The sequences of erosion and dilation were applied to remove the edges that do not represent nuclei boundaries. We then selected a Region of Interest (ROI) on the confocal image, to exclude unwanted noise that occurred at the time of imaging (e.g. very bright, dark, de-focused, or saturated regions on the image) (Fig. 3.4D). By reviewing the segmented objects (nuclei) in the ROI marked on the image, we removed any object that did not fulfill the minimum quality requirements such as out of focus objects or objects with bad masks (Fig. 3.4D). Then, we calculated the nuclei morphology features (Fig. 3.4E). and tissue architectural features (Fig. 3.4F). . 57 Figure 3.4. Image processing pipeline. (A) raw confocal image of a normal area. (B) Result of edge detection by Canny algorithm. (C) Object segmentation and creation of their masks followed by extraction of their centers of gravity. (D) Removing the objects that are not nuclei. (E) Calculating morphology features. (F) Calculating architectural features. 3.2.4 Imaging of Histological Sections Stained with Thionin-Feulgen As described in Chapter 2, Section 2.1.2, for each biopsy we obtained the histopathological review results (including normal, reactive atypia, CIN1, CIN2, and CIN3) and six histological slides from BCCA pathology lab. We scanned the slides that had been stained with Thionin-Feulgen using Getafics (Kamalov et al. 2005a) in absorbance mode (for details of the system see Section 2.2.1.2). On each Thionin-Feulgen-stained slide, based on the histopathological review of the corresponding biopsy, we located and imaged the area on the slide that was showing the worst histological diagnosis. Two expert cyto-technicians helped us with locating these areas. We refer to these images as histological images. Basal and superficial membranes were then 58 manually delineated on histological images, defining the ROI as described in (Guillaud et al. 2004). A semi-automatic thresholding segmentation algorithm was used to detect cell nuclei located within the ROI. This thresholding algorithm was used to separate objects (nuclei) from background based on pixel intensity. Auto-focusing and an edge-relocation algorithm (MacAulay et al. 1990) were finally applied to nuclei to precisely and automatically place the edge of the object at the region of highest local gray level gradient. Digital gray-level images of the nuclei were stored in a gallery. The two cyto-technicians then manually reviewed each image and removed the images with segmented nuclei that did not fulfill minimum quality requirements related to masking, focus, etc. 3.2.5 Quantitative Tissue Phenotype (QTP) Analysis QTP analysis of histological and confocal images refers to the quantification of both the phenotype of the nuclei and the overall architectural characteristics of the tissue, as described below. For all digitized images of nuclei and detected nuclear centers of gravity, we evaluated ~200 features associated with tissue architecture, nuclear and cellular shape, size, DNA amount and chromatin texture organization (Guillaud et al. 2004; Doudkine et al. 1995). Based on preliminary results, we have restricted our analysis to 4 features (Table 3.1): i) nuclear area; ii) cell density; iii) estimated nucleus area-to-cytoplasmic area (ENC) ratio; and iv) average distance between a nucleus and its 3 nearest Delaunay neighbors (3NDND). “Nuclear area” 59 represented the mean area of each segmented nuclei in µm2. “Cell density” represented the number of nuclei per one µm2. To calculate the ENC ratio and the 3NDND we applied a Voronoi tessellation and Delaunay graphs, as follows. Given a set of points S (each point is the center of gravity of a nucleus) in a plane, a Voronoi tessellation of the set S is a partition of the plane into polygons such that each polygon V(p) is associated with each point p of S in such a way that all locations inside V(p) are closer to p than to any other point in S (Fig. 2.5). The Voronoi polygon associated with a specific cell nucleus can be interpreted as an approximation of the cytoplasm of the cell (Guillaud et al. 1997). The ratio of nucleus area to the cytoplasmic area can then be approximated by the ENC ratio, which is the ratio between the nucleus area and the Voronoi polygon area. Category Feature name Description Nuclear morphology Nuclear Area Mean area of the nuclei Cellular morphology ENC ratio The area of a nucleus divided by the area of its associated Voronoi polygon. Tissue Architecture Cell density Number of nuclei per mm2 3NDND Average distance between a nucleus and its 3 nearest Delaunay neighbors Table 3.1. Features generated from quantitative tissue phenotype analysis The Delaunay diagram is the dual graph of the Voronoi diagram (De Berg et al. 2008). Two points of S are Delaunay neighbors if their Voronoi polygons share a common edge. Each pair of Delaunay neighbors is joined by a line segment; the sum of these segments forms the Delaunay graph (see Fig. 2.5). Several features measuring the characteristics of the spatial 60 distribution of the nuclei can be computed from the Delaunay graph. Based on previous studies, only one of these features was selected for this analysis; this feature –3NDND– measures the average distance between a nucleus and its three nearest Delaunay neighbors (in µm). The statistical significance of the relationships between different feature measurements and disease grade was assessed using ANOVA and Fisher’s least significant difference (LSD) post hoc test was performed using STATISTICA software (StatSoft Inc., Tulsa, OK, USA). 3.3 Results Analyses were restricted to cases for which the confocal images had an acceptable quality and for which the histopathogical interpretation was available. We considered three tissue types: normal, low-grade CIN (CIN1), and high-grade CIN (CIN2 and CIN3). It is worth mentioning that because we collected a normal biopsy from each patient, most of the collected biopsies were normal tissues or low-grade lesions. To prevent any bias that could be resulted from large number of normal and low-grade tissues in our analysis, we used a set of biopsies with about the same number of tissue types. A total of 46 biopsies, collected from 33 patients, were used; thirteen of them were classified as negative (i.e. normal), 18 as low-grade lesions and 15 as high-grade lesions (9 CIN2 and 6 CIN3). Table 3.2 shows the confusion matrix between the colposcopic appearance and the histological diagnosis. 61 Histopathology diagnosis Total Normal CIN1 CIN2 CIN3 Colposcopic impression Normal 9 6 2 0 17 Low-grade lesions 1 2 2 0 5 High-grade lesions 3 10 5 6 24 Total 13 18 9 6 46 Table 3.2. Classification of cervical biopsies based on colposcopic impression for the different histopathological grade. The results from two main analyses are presented in this section. First, we analyzed the confocal images acquired at a depth of 15µm below the epithelial surface for all biopsies. This depth was selected as at 15 µm, the nuclei were clearly visible for all cases. Figure 3.5 shows the confocal and histological images of the three cases (normal, CIN1, and CIN3). Second, to investigate the effect of depth of confocal imaging on the feature measurements, we compared images of 21 specimens obtained at different depth. For each specimen, three images were acquired at the surface of the epithelium, at 15µm and at 30µm below the epithelial surface. Figure 3.6 shows the confocal images of one specimen obtained at these 3 depths. Figure 3.7 shows the confocal and histological images with corresponding Voronoi tessellations for a CIN1 lesion. 62 Figure 3. 5 ex vivo confocal images of the cervix taken at a depth of 15 µm (left) and the corresponding histological images (right); (A), (B) normal area, (C), (D) CIN1 lesions, and (E), (F) CIN3 lesion. Scale bar is 50 µm. 63 Figure 3.6. Confocal images of a normal area taken at (A) the epithelial surface; (B) at 15 µm below the epithelial surface; and at (C) 30µm below the epithelial surface. Figure 3.7. (A) Voronoi tessellation overlaying the confocal image of a CIN1 lesion, and (B) its corresponding histological image. Scale bar is 50 µm. 3.3.1 Quantitative Tissue Phenotype Analysis of Confocal Images We studied the correlation between the histopathological grade and QTP features of confocal images calculated from 46 cases at 15µm below the tissue surface. We could observe that cell density was higher in the high-grade CIN than in CIN1 or in normal specimens (Fig. 3.5). We also observed that the penetration depth (maximum depth at which images can be obtained) was 64 larger in normal biopsies than in abnormal biopsies (with an average of ~80µm in normal cases vs. ~50µm for abnormal cases). Figure 3.8 shows the distribution of nuclear area, cell density, ENC ratio, and 3NDND values for the three histological groups. The nuclei of normal specimens were significantly smaller than the nuclei of CIN1, CIN2 and CIN3 lesions (respectively p = 0.006,p=0.002,p=0018) (Fig. 3.8A); on the other hand, there were no difference between the different grades of dysplasia. Cell density increased regularly from normal to high-grade lesions with the cell density of high-grade lesions being twice as large as the cell density of low-grade lesions (Fig. 3.8B). Similarly, the ENC ratio increased from normal to high-grade lesions, reflecting an increase in the nuclear area relative to the cytoplasmic area (Fig. 3.8C). The distance between a nucleus and its neighbours, as measured by 3NDND, decreased from normal specimens to high-grade ones (Fig. 3.8D). For these three features, the difference between low-grade and high-grade lesions were statistically significant (p = 2×10-7, p = 8×10-6, and p = 1×10-6, respectively). Neither cell density nor ENC ratio showed a statistical difference between normal and low-grade lesions (p = 0.136, p = 0.052, respectively). However, there was a statistically significant difference in 3NDND between normal and low-grade lesions (p = 0.005). 65 Figure 3.8. Results from analysis of confocal microscopy images. Box-plot distribution of QTP features for different histopathological diagnosis: (A) nuclear area; (B) cell density; (C) ENC ratio; and (D) 3NDND. (In each box-plot, the Centeral point represents median; lower and upper bounds of the box represent first and third quartiles; whiskers represent 5% and 95% quantiles) Figure 3.9 shows the scatter plot of the ENC ratio vs. the nuclear area for the different histopathological groups. There was a better separation between high-grade and low-grade lesions than between low-grade lesions and normal specimens. Normal CIN1 CIN2 CIN3 Normal CIN1 CIN2 CIN33NDND (µm)ENC Ratio1216202428320.00.10.20.30.40.50.6Cell Density1 2 3 40.0000.0010.0020.0030.0040.0050.006y_Normal CIN1 CIN2 CIN3Nucleus Area (µm2 )Normal CIN1 CIN2 CIN320406080100120140160180A BC D66 Figure 3.9. Analysis of confocal microscopy images. Relationship between ENC ratio and nuclear area for different histopathological grades. 3.3.2 Quantitative Tissue Phenotype Analysis of Histological Images From our group of 46 specimens, we were able to successfully image 40 specimens (12 normal, 15 low-grade CIN, and 13 high-grade CIN lesions). The other specimens were discarded form the analysis due to their insufficient quality. Figure 3.10 shows the distribution of mean nucleus area and 3NDND for the different histopathological groups. There was no statistically significant difference in the nucleus area between the three groups (p = 0.119) (Fig. 3.10a). On the other hand, 3NDND decreased regularly as dysplasia grade worsened (Fig. 3.10b). The difference was statistically significant between low-grade and high-grade lesions (p = 0.02) but not between normal specimens and low-grade lesions (p = 0.08). 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.720406080100120140160180NormalCIN1CIN2CIN3Nuclear Area (µm2 )ENC Ratio67 Figure 3.10. QTP analysis of histological images. Box-plot distribution of QTP features for different histopathological diagnosis: (A) nuclear area; and (B) 3NDND. (In each box-plot, the Centeral point represents median; lower and upper bounds of the box represent first and third quartiles; whiskers represent 5% and 95% quantiles) 3.3.3 Comparison of QTP Analysis of Confocal Images with Colposcopic Appearance We compared the colposcopic appearance as given by the clinicians at the clinic (normal, low-grade CIN, or high-grade CIN) with the QTP features measured from confocal images. Figure 3.11 plots the relationship between the colposcopic appearance and ENC ratio calculated from confocal images (obtained at 15µm) for different histopathological groups. We observe that 13 of the 31 colposcopically-defined high-grade CIN (41%) were either low-grade CIN or normal cases and 4 colposcopically-classified low-grade CIN were in fact high-grade lesions. Nuclear Area (µm2 )3NDND (µm)Normal CIN1 CIN2 CIN31 2 3 42030405060708090Normal CIN1 CIN2 CIN37891011121314151617A B68 Figure 3.11. Analysis of confocal microscopy images. Relationship between ENC ratio and colposcopic appearance for different histopathological grades: (circles normal; diamonds CIN1; triangles CIN2; squares CIN3). The Spearman correlation coefficient between histopathological diagnosis and colposcopic appearance was 0.43. The linear correlations between histology diagnosis and nuclear area, cell density, ENC ratio and 3NDND were respectively: 0.56, 0.77, 0.81 and 0.77. To assess the potential of confocal fluorescence microscopy for detecting cervical dysplasia, we calculated the sensitivity and specificity of ENC ratio and 3NDND values for detecting either high-grade lesions (i.e. CIN2 or CIN3) or any lesions (i.e. CIN1, CIN2, or CIN3) (Table 3.3). 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9NormalCIN1CIN2CIN3ENC RatioColposcopic Appearance GradeHigh GradeLow GradeNormal69 Detecting high-grade lesions Detecting any lesion Sensitivity (%) Specificity(%) Sensitivity(%) Specificity(%) Colposcopic impression 73 58 75 69 ENC ratio 93 96 100 84 3NDND 66 90 100 80 Table 3.3. Sensitivity and specificity of colposcopic appearance, ENC ratio, and 3NDND for classifying different grades of cervical dysplasia (imaging depth is 15 µm). Based on the feature distribution given in Fig. 3.8, biopsies were classified into 3 groups : normal if the ENC ratio was lower than 0.08, low-grade if the ENC ratio was larger than 0.08 and smaller than 0.18 and high-grade if the ENC ratio was higher than 0.18. Similarly, biopsies with a 3NDND larger than 25µm were classified as normal, biopsies with a 3NDND value between 21µm and 25µm were classified as low-grade, and biopsies with a 3NDND value smaller than 21 µm as high-grade. The sensitivity and specificity of colposcopic appearance (grading given by the clinicians) for detecting any grade of CIN were respectively 75% and 69%, (Table 3.3). The sensitivity and specificity of both ENC ratio and 3NDND values for detecting any grade of CIN, were comparable to those of clinicians (obtained from colposcopic appearance). The sensitivity and specificity of ENC ratio for detecting high-grade CIN alone were 93% and 96%, respectively. 3.3.4 Quantitative Tissue Phenotype Analysis of Confocal Images at Different Depths To investigate the performance of our technology at different depths, we imaged 21 cervical biopsies at three different depths: epithelial surface, 15µm and 30µm. Figure 3.12 shows the scatter plot of ENC ratios measured at these 3 depths vs. different disease grades estimated by clinicians based on the colposcopic appearance of the cervix. 70 Figure 3.12. Analysis of confocal microscopy images. Relationship between ENC ratio and colposcopic impression at different imaging depth; (A) at the epithelial surface, (B) at 15 µm below the epithelial surface; and (C) at 30 µm below the epithelial surface. 00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32NormalLow-Grade CINHigh-Grade CIN.0 0.1 0.2 0.3 0.4 0.5 0.6NormalLow-Grade CINHigh-Grade CIN.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.320234NormalLow-Grade CINHigh-Grade CINENC Ratio (depth of imaging: 0µm)Colposcopic Appearance GradeHigh GradeLow GradeNormalENC Ratio (depth of imaging: 15µm)ENC Ratio (depth of imaging: 30µm)Colposcopic Appearance GradeHigh GradeLow GradeNormalColposcopic Appearance GradeHigh GradeLow GradeNormalABC71 Table 3.4 shows the sensitivity and specificity of detecting dysplasia using either the ENC ratio or 3NDND measured on images sampled at different depths from the surface of tissue. For classification of biopsies based on their corresponding ENC ratio and 3NDND values, we applied the same criteria as described in section 3.3 (e.g. biopsies with an ENC ratio < 0.08, 0.08 < ENC ratio < 0.18, and ENC ratio > 0.18 were respectively classified as normal, low-grade and high-grade). The sensitivity and specificity of colposcopic appearance for detecting high-grade CIN were respectively 57% and 64%. The sensitivity and specificity of detecting any dysplasia using ENC ratio and 3NDND at any of the 3 depths - were comparable to those of colposcopic appearance (Table 3.4). The sensitivity and specificity of ENC ratio for detecting any dysplasia at 15µm was higher than the ones measured at the surface and at 30µm. Imaging depth QTP features Detecting high-grade lesions Detecting any lesions Sensitivity (%) Specificity (%) Sensitivity (%) Specificity (%)Surface ENC ratio 85 92 100 83 3NDND 71 85 66 83 15 µm ENC ratio 100 92 100 83 3NDND 100 85 80 83 30 µm ENC ratio 50 78 92 66 3NDND 83 71 78 66 Table 3.4. Sensitivity and specificity of ENC ratio and 3NDND for the detection of cervical dysplasia at different confocal imaging depths. Interestingly, we observed the low-grade lesion that showed a high ENC ratio value (measured at 15µm [see Fig. 3.12B] and comparable to high-grade lesions) has an ENC ratio value similar to 72 other low-grade lesions when imaged from the surface or 30µm deep. In addition, we noticed that the confocal image obtained at 15µm depth was saturated due to the maladjustment of the parameters of confocal microscopy at the time of imaging, explaining the overestimation of the nuclear size. 3.4 Discussion After an abnormal Pap-test, the colposcopic examination of the cervix is currently used to detect preneoplastic lesions after an abnormal Pap test and to guide biopsy selection for disease diagnosis and staging. Unfortunately, challenges exist regarding the accuracy and reproducibility of this approach. Pre-neoplastic lesions are associated with a variety of morphologic and tissue architecture alterations and confocal microscopy has previously been used to non-invasively detect changes to cell morphology and tissue architecture. Detecting these changes could be used to confirm histopathological diagnoses and assess progression likelihood (Adelman 2014; Collier et al. 2007; A. L. Carlson et al. 2007; Tan et al. 2009). Ultimately, clinical application of confocal imaging will lead to a new breed of “clinical pathologists” who can detect and grade preneoplastic lesions in vivo using real-time optical imaging (Guillaud et al. 2007). In this chapter, we obtained confocal fluorescence images of 46 fresh cervical biopsy specimens and from these images we calculated around 200 features (including morphologic and architectural) using our automatic software (which I wrote in MATLAB). The results indicated that two features, ENC ratio and 3NDND values, could be used to classify the dysplasia grade in confocal imaging. We demonstrated that these features were capable of differentiating high-73 grade CIN from normal and low-grade CIN with a high sensitivity and specificity that are comparable to the grading given by clinicians based on colposcopic appearance of the cervix (see Table 3.3). We compared the scatter plot of nuclear area vs. ENC ratio presented in Fig. 3.9 with that of a previous study by Collier et al. (Collier et al. 2002), who studied the reflectance confocal microscopy images of normal and abnormal cervical biopsies obtained at a depth of 50μm. These two scatter plots show similar distribution of normal specimens, low-grade and high-grade lesions. Our preliminary results suggest the potential of confocal fluorescence imaging tool in the diagnostic clinic for biopsy site selection. Currently, many unnecessary biopsy specimens are taken in the clinic due to the relatively low specificity of colposcopy (Cantor et al. 2008). The use of confocal microscopy for guiding biopsy excision processes could lower the cost of unnecessary diagnostic procedures and more importantly improve the patients’ experience. In addition, we have shown that quantitative features measured from confocal fluorescence images compare well with features calculated on histological images. These features are also consistent with classical descriptions for grading of cervical dysplasia based on histopathological criterion of H&E-stained slides (e.g. increased nuclear-cytoplasmic ratio or decreased cell-cell distances (Wright et al. 2011)). Of course, the obvious advantage of confocal images over histological images is that confocal images could be obtained in vivo and therefore could provide information about cells and tissue structure in a non-invasive manner. Hence, by acquiring and analyzing in vivo confocal images rather than histological images, we could 74 remove the need to acquire and process tissue specimens which is costly in terms of time and funds. Images obtained in this study were of a higher resolution than those of previous studies done using reflectance microscopy (Collier et al. 2002, 2007). This may be due to use of a contrast agent in confocal fluorescence microscopy, which improves the cellular contrast by staining the nuclei of the epithelium. In this study, Acriflavine dye was used to stain the nuclei for confocal fluorescence microscopy, a process that leads to clearly visible nuclei in resulting images (El Hallani et al. 2013). However, the images obtained in this study were taken at a depth (from the surface of epithelium) that was shallower than those of previous studies (Collier et al. 2002, 2007). Although Acriflavine strongly stained the superficial epithelial cell nuclei, the dye penetration to deeper layers was limited, as already observed by Tan et al. (Tan et al. 2009). Fortunately, unlike in reflectance microscopy, nucleo-architectural features can be robustly measured even at such limited penetration depth. This is due to the fact that high-grade lesions are characterized by the presence of dysplastic cells in the superficial layers of the epithelium. Our results show that the optimal depth of imaging to discriminate high-grade lesions from other lesions is 15 µm. It was challenging, however, to differentiate normal tissue from mild dysplasia based on nuclear and cellular features at 15µm from the surface of the tissue. In the future, additional experiments based on a larger sample size will be necessary to statistically establish the optical imaging depth for in vivo real time confocal microscopy use. It should be noted that other studies that used deeper confocal imaging were also unable to accurately differentiate normal tissue from mild dysplasia (Collier et al. 2002; A. L. Carlson et al. 2007). 75 In conclusion, we first demonstrated that, the ability to use only a few morphological and architectural features to differentiate normal tissue and low-grade lesions from high-grade dysplasia; and second, a similar performance was achieved at different epithelial depths (see Table 3.4). This is a significant finding since it is likely that assessing the exact depth of imaging in vivo will be challenging. Furthermore, by imaging and comparing features at different epithelial depths, we can reach a more accurate and reproducible quantification of dysplastic changes. Indeed, we can foresee that a clinician could use different depth information to increase his/her confident in case the images are not good enough, or as it happens in the cervix in the presence of keratin at the surface of the epithelium, or when the contact is not adequate. By moving up and down with the probe, our data has shown that results will not be affected. Our experimental design suffers from one limitation. In histology images obtained from Feulgen-stained sections an experienced cyto-technician carefully choose the region of interest corresponding to the diagnostic area selected by the study pathologist on the H&E section. This area corresponds to the worst dysplastic area present on the section. For confocal images, to avoid any selection bias, the region of interest was selected randomly without any input from the pathologist. It is possible that the confocal area does not exactly match the corresponding histopathological diagnosis and area selected by the pathologist. Unfortunately, this is a limitation of this study. Nevertheless, we believe that our approach is valid and justified as we are trying to study the average relationship between the outcome of this technology and tissue appearance, as currently tissue appearance in the clinic is what clinicians base their grading on. By choosing random areas, we are more likely to fail to spot dysplastic areas where the lesion is small (low-grade) than where the lesion is large (high-grade), resulting in an under-estimation of 76 the "true" sensitivity of this technology for detecting low-grade lesions. In the future, we believe that a new generation of pathologists/clinicians will become skilled in selecting and identifying the precancerous lesions on confocal images. Our results may actually underestimate the power of confocal microscopy for detecting dysplasia. 3.5 Summary and Conclusion In this chapter we have studied the potential of nuclear-architectural features measured from ex vivo fluorescence confocal images of fresh cervical biopsy specimens, to improve the detection of cervical dysplasia grades. By correlating these features with histopathology diagnosis, we confirmed that confocal fluorescence microscopy can differentiate high-grade lesions from low-grade lesions and normal tissues. More importantly, we have shown that these results were comparable to those obtained from images collected at different depths. New experiments involving deeper tissue examinations by improved staining techniques will help to better differentiate normal tissues from low-grade lesions. This study serves as a critical step towards utilizing confocal approaches in an in vivo clinical application. By utilizing this type of enhanced optical technologies (like confocal imaging), we will be able to automatically identify cervical lesions with high sensitivity and specificity in a non-invasive manner in the clinic. This will reduce the frequency of unnecessary biopsies, and more importantly enable us to provide real time diagnosis in the clinic, with the possibility of immediate treatment by less experienced practitioners. 77 Chapter 4: Developing Automatic Algorithms for the Analysis of Histological Images 4.1 Introduction In the previous chapter, we demonstrated how we can improve the effectiveness of the current procedure for cervical neoplasia diagnosis by employing enhanced optical technology (rather than relying only on colposcopy) in the clinic. One other approach to improve the effectiveness of the current diagnosis procedure is presented in this chapter. By employing advanced techniques for quantitative digital pathology, we could improve the routine that is currently being used for extracting diagnostic information from histological sections in the pathology lab. Such enhancements can lead to less labor/time consuming, and more accurate pathological reviews. In this chapter, we address the automation of quantitative analysis of digital microscopy images of tissue sections. The images that we analyze in this chapter are images of Immunohistochemistry (IHC)-stained sections. One of the powerful tools in pathological review procedure is IHC staining which is a tissue based biomarker imaging technology. Biomarkers are quantifiable substances that indicate a certain biological condition or state. In the process, a fixed tissue sample is first sectioned and mounted on a slide, then stained with different IHC biomarkers, and finally imaged by a scanner or a charge-coupled device color camera mounted on an optical microscope. IHC imaging is widely used in the detection of dysplastic cells and indicating the physiological state (and changes) during the disease process. As an example, in cervical cancer diagnosis, currently p16 staining is used to differentiate between low-grade and high-grade lesions where morphological 78 interpretation of histological slides is not sufficient (Klaes et al. 2001). The information obtained from this staining could also be used to better manage high-grade lesions (Miralpeix et al. 2016). In addition, there is a relationship between how proliferated cells are distributed in the epithelial layer of the tissue and the grade of neoplasia (Reuschenbach et al. 2012; Guillaud et al. 2014). In higher grade CINs, proliferation would be seen anywhere from basal membrane to the surface of the epithelium, while in lower grades it is commonly seen in the basal layer and rarely in the upper layers of epithelium. Furthermore, studies have suggested that the co-expression of p16 and Ki-67 may help to differentiate high-grade dysplasia from other benign lesions (Keating et al. 2001; Van Niekerk et al. 2007; Reuschenbach et al. 2012; Guillaud et al. 2014). In this technology, for any biomarker, the spatial distribution of the biomarker with respect to the tissue histological structures as well as the spatial relationships of the cells expressing the biomarker could be determined. This information could provide insight into the molecular complexity of the disease process, create new cancer screening and diagnosis tools, inform the design of new treatments, monitor treatment effectiveness, and predict the patient response to a treatment (Khojasteh-Lakelayeh 2012). However, the manual quantification and localization of each subset of cells containing a biomarker, under a microscope for each IHC stained tissue demands a large amount of time and effort. In addition, qualitative or semi-quantitative scoring systems suffer from low reproducibility, accuracy, and sensitivity, and decreased ability to distinguish the different stains apart when they overlap spatially. Therefore, there is a need for methods that can automatically quantify IHC images. This is, however, a challenging problem due to the variations caused by the different staining and tissue processing procedures. 79 The techniques proposed in the literature for quantifying histopathology or IHC images are generally based on traditional image processing approaches that rely on extracting pre-defined features of nuclei or cell appearance (Gurcan et al. 2009; Irshad et al. 2014). Using pre-defined features in IHC image analysis however, raises the concern of low reproducibility and lack of robustness, due to variations in the nuclei and cell appearance that are caused by the different staining and tissue processing procedures used by different laboratories. Recently, several studies have shown that deep learning algorithms such as those employing convolutional neural networks (Lecun et al. 1998), could be used in histopathology image analysis with great success (Jain et al. 2007; D. C. Cireşan et al. 2013; Chen and Chefd’hotel 2014; Litjens et al. 2016; Xu et al. 2016; D. Wang et al. 2016; Janowczyk and Madabhushi 2016; H. Wang et al. 2014; Han et al. 2017; Araujo et al. 2017). For example, Jain et al.(Jain et al. 2007) applied a convolutional network to segment electron microscopic images into intracellular and intercellular spaces. Ciresan et al.(Cireşan et al. 2013) and Wang et al.(Wang et al. 2016) employed deep learning to respectively detect mitoses and metastatic breast cancer through quantitative analysis of breast histopathology images. This approach has outperformed other methods by a significant margin (Cireşan et al. 2013; Wang et al. 2016). Janowczyk et al.(Janowczyk and Madabhushi 2016) demonstrated that the performance of their convolutional approach in segmentation (of nuclei, epithelium, tubule, and invasive ductal carcinoma), detection (of lymphocyte and mitosis) and classification (of lymphoma subtype) applications, was comparable or superior to existing approaches or human performance. Han et al. (Han et al. 2017) and Araújo et al. (Araujo et al. 2017) applied deep learning for the classification of breast biopsy H&E images. 80 Herein, we address the automation of IHC image analysis. Cells or nuclei that are expressing different biomarkers appear with different colors in an IHC image. One main challenge that we tackle is to automatically quantify the proliferation characteristics of the epithelium on “dual-stained” tissues (stained for both Ki-67 and p16). Currently, pathologists manually assess the dual-stained tissue to precisely grade the disease. Such assessments are especially needed where morphological interpretation of histological sections is not enough for differentiating between low-grade and high-grade lesions. But we know as a fact, that it is very difficult for human observers to identify the cells that are Ki-67 positive in a dual-stained tissue, in which the cells are also p16-positive; this is due to the strong p16 stain intensity. Prior to starting our work on this project, a preliminary study had been carried at BCCRC to quantify the proliferation characteristics of the epithelium by manual labeling. Two non-expert persons were first asked to mark the Ki-67 positive cells in single-stained tissues (using Mib stain). Then they were asked to mark the Ki-67 positive cells in dual-stained tissues (using CINtec). Their assessments are shown in Fig. 4.1. 81 Figure 4.1. Percentage of Ki-67 positive cells in the different epithelial layers of a Ki-67- stained CIN3 lesion (Mib) and of a dual-stained CIN 3 lesion (CINtec). Figure 4.1 shows that the assessment done on dual-stained tissue was much worse than the assessment done on single-stained tissue. The percentage of Ki-67 positive cells was under-estimated (by a factor of 50%) in the dual-stained tissue for all epithelial layers. In normal epithelium, the estimations were identical. This study suggested that manual labeling could under-estimate the true proliferating characteristics of the epithelium. We propose a deep learning method to detect and locate different biomarkers present in cervical intra-epithelial lesions and quantify the percentage of proliferating cells in different layers of a p16 positive epithelium. The tissue could be a normal cervix tissue, low grade CIN, or high grade CIN. The deep learning network that we propose herein is a fully convolutional neural network that can take an IHC image as an input, and detect and locate the different biomarkers present in every cell of the image. This network is an extension of a convolutional neural CIN3 – CINtec stainCIN3 –Mib stainNormal epitheliumCINtec stainNormal epitheliumMib stainKi‐67 positive cells %706010020304050Layers1 2 3 4 5 6 7 8 9 1082 network that we trained using RGB images of nuclei, to classify a nucleus according to its expressed biomarker. The benefit of using the deep learning approach (over image processing methods that rely on extracting pre-defined features of nuclei or cell appearance) is that the feature descriptors are automatically learned in the convolution kernels (LeCun et al. 2015) (during the training of the network). Therefore, there is no need to find and quantify sophisticated features for every group of cells expressing a specific biomarker. We believe that by eliminating the need to find and quantify advanced features for each cell expressing a biomarker, we could develop more reproducible methods for biomarker localization in IHC images. In addition, in our proposed approach, there is no need for pre-processing the RGB images before feeding them to the network. This is in contrast to most methods in the literature, as these methods rely on complicated processing like un-mixing techniques (e.g. un-mixing multi-spectral images and color un-mixing in RGB images). We should keep in mind that there is a drawback in employing deep neural networks. They performed very well and won grand challenges across different practical areas (LeCun et al. 2015; Russakovsky et al. 2015), and of course in the field of medical imaging (Jain et al. 2007), but the good performance of deep neural networks comes at a cost; the need for large data sets, computational complexity, and being black box predictors which can make their performances unpredictable. 83 In this chapter, we first explain how we have collected our dataset which is composed of IHC images selected on whole slide images of cervical biopsies. Then we discuss the methods we have used to detect and localize the different biomarkers in each of these images. After that, we explain the experimental design, results and validation metrics we have used to evaluate our proposed approach. 4.2 Dataset The study presented in this chapter included 69 cervical biopsy sections in total. 4.2.1 Imaging Dual-Immunostained Specimens As described in Chapter 2, Section 2.1.2, from each biopsy we obtained 9 histological sections (slides). In this chapter, we used 4 out of the 9 slides cut from each biopsy. Out of the 9 slides cut from each biopsy, three slides were stained with H&E. The H&E-stained slides were considered diagnostic slides and reviewed by an expert pathologist to establish disease grade as either: normal, reactive atypia, CIN1, CIN2, and CIN3. A fourth slide, which we refer to as IHC slide, was double immunostained for p16/Ki-67, using the CINtec PLUS kit (Roche mtm laboratories, Mannheim, Germany) according to the manufacturer’s instruction, and then counterstained with Hematoxylin. The IHC slide was used for immunohistochemical analysis. Our study included 69 slides in total. We randomly selected slides from biopsies with different disease grade to make sure our dataset includes all disease grades. To ensure that our analysis was not biased by our staining or tissue processing procedure, the slides were randomly assigned to two pathology cyto-technicians. The slides were split into seven batches (six batches of 10 slides and one batch of 9 slides). One of the two cyto-technicians stained three batches and the 84 other technician stained four batches; one batch per week. For all the slides, one reagent lot was used and the staining was performed manually. Each immunostained tissue sample (IHC slide) contained three labels: 1) a counterstain that labeled every nucleus (Hematoxylin [H]), 2) a chromogen (fast red) that labeled a nucleus-bound IHC biomarker (Ki-67 antigen), and 3) a chromogen (DAB) that enabled the visualization of the other IHC biomarker (p16 antigen), which is localized in the nuclei and cytoplasm (Reuschenbach et al. 2012). The Ki-67 antigen is one of the most commonly used markers of cell proliferation, and is routinely applied to biopsies of cancerous or pre-cancerous cervical tissues (Keating et al. 2001). The p16 protein is an important tumor suppressor protein that regulates the cell cycle, and is also commonly used as a diagnostic biomarker of cervical neoplasia (Van Niekerk et al. 2007). For each of the obtained IHC slides, we reviewed its three corresponding H&E slides using Pannoramic Viewer (3DHISTECH Ltd.) at 4X magnification and located the regions exhibiting the worst diagnosis on the H&E slides. Then we scanned each of the IHC slides with a Pannoramic MIDI whole slide scanner (3DHISTECH Ltd., Budapest, Hungary) at 20X magnification. Again, using Pannoramic Viewer at 4X magnification we marked the same diagnostic areas that we located before by reviewing three H&E slides, on the corresponding IHC slide. Then, the marked areas were exported to RGB images in TIFF format (with the total magnification of 80X and the resolution of 96 dpi). A total of 69 images, varying in size from 1024×1024 up to 17408×17408 pixels, were used in our analysis. For simplicity, we will refer to these images as IHC images in this thesis. Figure 4.2 shows example IHC images. 85 Figure 4.2. Sample IHC images taken from CINtec-stained cervical tissues: (A) a normal tissue, (B) CIN1, (C) CIN2, and (D) CIN3 lesions. 4.3 Biomarker Labeling Method The biomarker labeling, outlined in Fig. 4.3 was done in two main steps: i) training a convolutional neural network which we will refer to as Nuclei(N)-Net to label the expressed 86 biomarkers in one nucleus, and ii) transforming the N-Net to a fully convolutional network to locate all cells expressing each of the different biomarkers in an IHC image. We will refer to this network as Whole Image (WI)-Net. Figure 4.3. A two-step approach to biomarker labeling in IHC imaging. Step one (top): N-Net was trained with nucleus images segmented from the IHC images in the training set, where each image had one nucleus only. This network learned to classify each nucleus image according to the expressed biomarker(s) in its respective nucleus. Step two (bottom): the trained classifier (N-Net) was extended to WI-Net. This end-to-end, pixel-to-pixel network localized the biomarkers expressed in each cell/nucleus in an IHC image. The input to the WI-Net is an IHC image with a large number of cells and the output is a heat-map of the two biomarker expression. In the first step, the nuclei in the IHC images were segmented (see Sec. 4.1.1 for details). The resulting nuclei images were labeled (as p16 positive, Ki-67 positive, both Ki-67 and p16 positive, and both p16 and Ki-67 negative). We then used these nuclei images to train the N-Net to detect expressed biomarkers in a nucleus image. We would like to clarify that in our P16 positive Ki67 positiveP16 & Ki67 pos. Output Convolutional Neural Network Fully Convolutional Network P16 & Ki67 neg.(Selected ROI in an IHC image) (Nuclei images)Input(Labels)Nuclei segmentation(Immunostained image) (Biomarker heatmap)Output Input87 segmentation, we did not aim to delineate the exact border of the nucleus or the cytoplasm (which was not necessary for our application), and therefore, each nucleus image was represented by a square image consisting of the nucleus and its surrounding cytoplasm. In the training process, the nuclei images were fed as inputs to the N-Net. The outputs of the N-Net were the labels that identified the expressed biomarker in the nucleus/ cytoplasm of each input image (Fig. 4.3 top). In the second step, a whole IHC image containing a large number of cells was fed into the WI-Net. This image was of arbitrary size. For each pixel, the WI-Net calculates the probability of each biomarker being expressed. These probability values were then used to produce a color heat-map of the different regions expressing different biomarkers in the IHC image (Fig. 4.3 bottom). 4.3.1 Nuclei Classification Using N-Net The N-Net was the classifier we built for labeling the expressed biomarkers in nuclei images. Figure 4.4 shows the schematic representation of the N-Net configuration, which is a convolutional neural network (Lecun et al. 1998) consisting of five layers. The input to the first layer is an RGB image containing one nucleus only. The first layer is a convolution layer that convolves the input image with K filters; each of these is 3-dimentional with a kernel size of F1× F1×3. The values of each kernel are learned by the N-Net. The second layer is a max-pooling layer with a kernel size of F2 by F2, which is a sub-sampling layer that reduces the size of the image. The third and forth layers are fully connected layers, each containing M neurons. The output layer consists of N neurons, each with a Gaussian function. This layer generates labels, showing the probability of each biomarker being present in the image. 88 Figure 4.4. Schematic representation of the architecture of the N-Net. N-Net consists of one convolution layer, followed by a max-pooling layer and two fully connected layers. The last layer is the output layer. This network takes an RGB image of a nucleus as input and generates a label as the output. 4.3.2 Biomarker Localization in Whole Slide Images Using WI-Net In the above section, we explained how N-Net was trained as a classifier for images, each containing one nucleus (see Sec. 3.1). In order to employ this classifier for localizing all biomarkers in a whole IHC image (containing thousands of nuclei), a straightforward approach would be to employ a sliding window (whose size is equal to the N-Net input size) on each pixel of the IHC image. Taking the window region as the input of the N-Net and obtaining the N-Net output, could generate a label for each pixel in the IHC image. The main disadvantage of this approach is that it could get computationally intensive especially for large input images. Therefore, we used a more computationally optimized approach in this study, and took advantage of the nature of convolution which could be considered as a sliding window (Shelhamer et al. 2016). By transforming the trained N-Net to WI-Net which is a fully convolutional network, a classification map of the IHC image is obtained (rather than segmenting the images and generating a single classification of a nucleus image by the N-Net). 0100Convolution layerDepth of 3 (R, G, and B channels)Depth of KMax pooling OutputLabelsFully‐connected layersInputImage of a nucleusDepth of KF1F1M MN neurons89 This transformation is shown in Fig. 4.5. The fully connected layers of the N-Net were converted to convolutional layers; this was done for the purpose of modifying these layers to make them independent of the spatial size (Shelhamer et al. 2016). Figure 4.5. Transforming the fully connected layers in the N-Net to convolution layers. All other layers remain the same. This enables the new network (WI-Net) to produce a pixel-wise output for prediction. The input to the WI-Net is a whole IHC image, and the output is a heat-map of the present biomarker(s) in the IHC image. 4.4 Experiments and Results In this section we present the experimental design, results and validation metrics that we have used to evaluate our proposed approach for IHC analysis. 4.4.1 Biomarker Localization The biomarker localization approach that we developed herein, as described in the previous section, consisted of two steps. First, we trained the N-Net for labeling different biomarkers in nuclei images and then we transformed the Network into WI-Net for biomarker labeling in a whole IHC image. 90 4.4.1.1 Preparing Nuclei Images for Training N-Net To train N-Net, we must first prepare nuclei images from the whole slide IHC images. To do so, we selected different regions of interest (ROI) delineating the basal membrane and the superficial membrane in six whole slide images. Then we coarsely segmented the nuclei in each ROI. Figure 4.6 shows the ROI selection, the nuclei images and the label corresponding to each of four different protein expressions (p16 positive, Ki-67 positive, p16 and Ki-67 positive, and p16 and Ki-67 negative). Figure 4.6. Sample of nuclei images obtained from an IHC image. ROI (red enclosure) selected in an IHC image, within which the nuclei were segmented in order to obtain nuclei images for training N-Net (left). Nuclei images expressing different proteins (right); p16 positive, Ki-67 positive, p16 and Ki-67 positive, and p16 and Ki-67 negative. To select the ROIs we used our in-house image analysis software, Getafics (Kamalov et al. 2005), and manually delineated the basal membrane and the superficial membrane within each image. Then, we detected the nuclei and their centers of gravity within each ROI at magnification of 20X. This procedure was semi-automated, requiring only minor manual adjustments (described in (Guillaud et al. 2004)). p16 positiveKi‐67 positivep16 and Ki‐67 pos.p16 and Ki67 neg.91 Digital RGB images, each containing one nucleus, were rescaled from their original sizes to 32 by 32 pixel images and stored in a gallery. We then manually labeled the nuclei into four groups: i) p16 positive, ii) Ki-67 positive, iii) both Ki-67 and p16 positive, and iv) both p16 and Ki-67 negative. These nuclei images along with their group number, were considered as the ground truth and used for training the N-Net. We had a total of 4679 labeled nuclei RGB images obtained within the ROIs from the six IHC images. The IHC images were acquired from six biopsy sections (obtained from six different patients): two CIN3, one CIN2, two CIN1 and one negative (normal cervical tissue) cases. Of these 4679 nuclei images, 156 nuclei images belonging to one of the six IHC images were used for testing the performance of N-Net, and the remaining 4523 nuclei images obtained from the other five IHC images were used for training. Table 4.1 illustrates the distribution of the nuclei in the training and test sets of the different groups. Number of nuclei in the different groups p16 positive Ki-67 positive Ki-67 & p16 positive p16 and Ki-67 negative Total Train Set 918 866 1572 1167 4523 Test Set 62 25 52 17 156 Table 4.1. Distribution of nuclei in the training and test sets. 4.4.1.2 Training and Testing N-Net We used the following network configuration. As mentioned above, the first layer of N-Net was a convolutional layer containing 96 filters with a kernel size of 11×11×3 and stride of 4 through 92 which each filter was slid. The second layer was a max-pooling layer with a kernel size of 3×3, and stride of 2. To decide on the architecture of the neural network, we created a synthetic dataset to train and validate the network. This dataset consisted of 1100 images of 32×32 pixels, each containing a disc in a white background. Figure 4.7 shows example images from this dataset. The discs had different sizes and were in three different colors (blue, brown and red). Using different parameters (number of layers and filters) and starting from a simple network, we trained the network to classify discs in to three groups based on their colors, and chose the simplest architecture that was trained successfully. Out of these 1100 images, 1000 images were used as the training set and 100 images were used as the validation set. The discs with three different colours were evenly (and randomly) distributed in both training and validation sets. Figure 4.7. Example images of synthetic dataset used for determining the architecture of the N-Net. The dataset consisted of discs with random sizes and random locations on a white square. The discs were in three colours: (A) blue, (B) brown, and (C) red. The N-Net learned to classify each disk to one of the three groups based on the disk colour. In order to define the kernel sizes, we used a validation set consisting of 161 nuclei images (randomly selected from the training set), and trained the N-Net with different kernel sizes. The kernel size corresponding to the lower error rate was selected. The third and forth layers were A B C93 fully connected layers, each containing 2048 neurons. The output layer consisted of four neurons. The N-Net was implemented by using the open source, deep learning framework Caffe (Jia et al. 2014), and a system with an Intel I5-6600 3.3GHZ Quad core CPU and a GTX750TI GP. For training, we used the back-propagation algorithm whereby 4523 labeled nuclei images were the inputs to the N-Net. The Softmax function was used in the output layer of the N-Net. We considered four labels (representing the four groups) as the output of the N-Net. The trained network learned the features corresponding to the presence of following biomarkers in each nucleus: i) H and p16 (label: 0 0 0 1), ii) H and Ki-67 (label: 0 0 1 0), iii) H and both Ki-67 and p16 (label: 0 1 0 0), iv) H only (absence of Ki-67 and p16) (label: 1 0 0 0). Training was stopped after 8000 iterations (with an error rate of 0.04 on the training set). Following the training, we performed the classification on the test set using N-Net. Table 4.2 shows the confusion matrix of the results of the training and test sets. According to this confusion matrix, the classification error rate for N-Net was 0.07 on the test set. 94 Training Set p16 pos. (Predicted) Ki-67 pos. (Predicted) Ki-67 & p16 pos. (Predicted) p16 & Ki-67 neg. (Predicted) Accuracy p16 pos. (Actual) 869 0 45 4 94.6% Ki-67 pos. (Actual) 0 835 30 1 96.4% Ki-67 & p16 pos. (Actual) 79 10 1483 0 94.3% p16 & Ki-67 neg. (Actual) 1 6 0 1160 99.4% Overall Accuracy 96.1% Test Set p16 pos. (Predicted) Ki-67 pos. (Predicted) Ki-67 & p16 pos. (Predicted) p16 & Ki-67 neg. (Predicted) Accuracy p16 pos. (Actual) 59 0 1 2 95.1% Ki-67 pos. (Actual) 0 25 0 0 100% Ki-67 & p16 pos. (Actual) 8 0 44 0 84.6% p16 & Ki-67 neg. (Actual) 0 0 0 17 100% Overall Accuracy 92.9% Table 4.2. Confusion matrices of the classification results on the training set (top) and test set (bottom) using N-Net. Figure 4.8 illustrates the value of each of the 96 filters in the convolution layer in the trained N-Net. 95 Figure 4.8. Illustration of the 96 filters in the convolution layer of the trained N-Net. Figure 4.9 shows the output of the convolution layer for different nuclei images as the input. By looking at the output of convolution filters, obtained from different nuclei images expressing different biomarkers, we could not find a meaningful pattern suggesting that the convolution layer extracts any nucleus morphology information. It should be noted that the p16 protein is located in both cytoplasm and nuclei of the infected cells, and not only in the nuclei. Each of the nuclei images we used in the training of the N-Net, consisted of a nucleus and part of its surrounding cytoplasm. We trained the N-Net to locate different protein expressions in both nucleus and cytoplasm of the cells. However, it is possible to train the network for picking morphology features as well; we have not trained the network for that purpose as such information was not of interest in our application. 96 Figure 4.9. The output of 96 filters in the convolution layer obtained from different nucleus images expressing different biomarker as the input of the N-Net. (A) p16 and Ki-67 negative nucleus, (B) Ki-67 positive nucleus, (C, D) p16 positive nuclei, and (E, F) both p16 and Ki-67 positive nuclei. A BC DE F97 To estimate the accuracy of N-Net classification, we also performed a 10-fold cross-validation analysis. We split the training set (4523 nuclei images) in to 10 subsets (nine sets of 452 nuclei and one set of 455 nuclei). The subsets were approximately balanced in different labels. We used the same network architecture and training parameters as explained above and trained the network 10 different times. At each training round, we trained the N-Net by using 9 subsets and then test the network performance on the left out subset. In 10 training rounds, we obtained the average accuracy of 0.94 (SD=0.01) on classifying the test subsets. The obtained accuracy is consistent with the results obtained from training the N-Net with the train set of 4523 nuclei images and testing its performance on an independent set of 156 nuclei images. The results suggest that the performance of N-Net is robust to the way we split our dataset into training and test data. 4.4.1.3 Labeling Biomarkers in IHC Images Using WI-Net After training the N-Net in labeling the expressed biomarker(s) in nuclei images, we transformed the N-Net into WI-Net for analyzing the whole IHC image. The WI-Net produced a classification map that covers an input IHC image of arbitrary size. We transformed the N-Net to WI-Net based on the method presented by Shelhamer et al (Shelhamer et al. 2016). The kernel sizes for WI-Net are the same as N-Net. The only difference is the last three layers, which are fully connected layer in N-Net and we transfer them to convolutional layers (with the kernel sizes of 4, 1, and 1 respectively) in WI-Net. Next, we fed each IHC image as an input to the WI-Net, and obtained a heat-map for each of its biomarker. We used interpolation to up-sample WI-Net output to the size of the input image. In total, four heat-maps were produced for each IHC image, each illustrating the regions in the IHC image that are: i) p16 positive, ii) Ki-67 positive, 98 iii) both p16 and Ki-67 positive, and iv) both p16 and Ki-67 negative (only expressing H). Figure 4.10 shows an example of an IHC image (Fig. 4.10A) and its corresponding heat-map obtained as the output from the WI-Net (Fig 4.10F). Figure 4.10. Heat-maps resulting from the biomarker labeling on an example IHC image. (A) The IHC image (size 2048×2048 pixels) belonging to a whole slide image obtained from the scanner was fed to the WI-Net. (B) The outputs of the WI-Net were the four heat-maps (each with the size of 2048 × 2048 pixels) marking p16 positive regions, (C) Ki-67 positive regions, (D) p16 and Ki-67 positive regions, (E) p16 and Ki-67 negative regions. (F) The four heat-maps were combined to produce an overall biomarker heat-map. Visual comparison of the original IHC image and the labeling results from WI-Net (Fig. 4.10A and Fig 4.10F) shows that the WI-Net performance in localizing p16 and Ki-67 in the dual-stained tissue was qualitatively acceptable. To obtain a better perception of the performance of WI-Net, we encourage the readers to check all the IHC images that we assessed in this thesis and their corresponding heat-maps generated by WI-Net (which we made publically available in: 99 https://osf.io/c74wu/). In addition, in Sec. 4.4. 2 we provide more validation for our method. The IHC images we used for validation were completely unseen to N-Net and WI-Net (i.e. the images were never been used as whole images or nuclei images to train or test N-Net or WI-Net). We would like to emphasize that in our validations, we have excluded the 6 patients whose IHC sections were used for preparing nuclei images for training and testing the N-Net (see Sec. 4.4.1.1). This was done to prevent the bias that might be introduced to our validation by using IHC images obtained from a same biopsy/patient in both training dataset and validation dataset. 4.4.2 Validation To evaluate the performance of our method, we compared the results obtained from WI-Net with those obtained from a color deconvolution approach (Ruifrok and Johnston 2001). We employed the color deconvolution presented by Ruifrok et al. (using ImageJ, Fiji implementation (Schindelin et al. 2012)) which is one of the most widely used color un-mixing approaches in digital pathology. Figure 4.11 shows the results of locating p16 and Ki-67 on two IHC images acquired from two dual-stained tissue specimens (one CIN3, and one normal tissue) by using color deconvolution and WI-Net. 100 Figure 4. 11. Comparison between color deconvolution approach and WI-Net approach for locating p16 and Ki-67 positive pixels in two IHC images. The IHC image on the left two columns corresponds to a high-grade cervical lesion. The IHC image on the right two columns corresponds to a normal cervical epithelium. The first row shows the RGB images. The second row shows the regions marked as p16 positive by the two methods. The third row shows the regions marked as Ki-67 positive by the two methods. In locating p16 and Ki-67 in the CIN3 tissue (where both p16 and Ki-67 were present), we observed that the color deconvolution method performed very well. The closer look at the results reveals that the presence of p16 in the CIN3 tissue was slightly under-estimated by the color deconvolution method (in central areas of the tissue), while it was slightly over-estimated by our WI-Net. On the other hand, some regions that are Ki-67 negative are marked as Ki-67 positive by color deconvolution method. It should be noted that the knowledge of the exact border of a IHC image(RGB)p16-positiveKi-67-positive Color Deconvolution WI-Net WI-NetColor Deconvolution101 nucleus or a cell in immunostain quantification is not critical, and the main concern is to detect the presence of a biomarker in a cell. Hence, in general, we could say that the performance of color deconvolution and WI-Net in localizing p16 and Ki-67 in CIN3 tissue were comparable. In the normal tissue (where none of p16 or Ki-67 were present), we observed that our WI-Net method did not classify any pixel as p16 positive or Ki-67 positive. Unfortunately, the color deconvolution method mistakenly marked some regions on the tissue as being p16 or Ki-67-positive. This lack of robustness over different tissue samples with different disease stages, could be due to difficulty of finding the optimal parameter/threshold setting in the color deconvolution method. It is possible that with a set of parameters the color deconvolution would perform better in high-grade tissues, but would fail in low-grade tissues. In addition, to evaluate the efficacy of our proposed biomarker labeling method, we performed two separate analyses. First, we compared the performance of our fully automatic algorithm to the human performance. Second, we investigated the utility of our immunohistochemical analysis to quantify the proliferation characteristics of the epithelium to aid with delineating between different pathology grades. 4.4.2.1 Comparison with Manual Biomarker Labeling We manually labeled the cells expressing different biomarkers in two IHC images, and then used the labeled cells as the ground truth. Obtaining a biomarker heat-map for each IHC image in a manual fashion is not easy; it is extremely laborious, requiring manual delineation of the membrane around each nucleus and cell and labeling each pixel in an IHC image. However, the 102 knowledge of the exact border of a nucleus or a cell for immunostain quantification in the application presented in this thesis is not critical, because the main concern is to detect the presence of a biomarker in a cell. Hence, we only marked the center of the nucleus in each cell and assigned a label to the center based on the expressing protein(s) in that nucleus. To obtain the best possible manual labeling (in the sense that it could be regarded with confidence as the ground truth for our comparisons), we asked two experienced cyto-technicians to assign these labels. Basically, the cyto-technicians worked together and reached to an agreement on the protein expression for each cell. We then considered the label assigned to the center of a nucleus as the ground truth of the expressed protein(s) in its corresponding cell. We compared this ground truth to the biomarker heat-map generated by the outlined proposed method. For each biomarker, we defined the following metrics:  Number of correctly labeled nuclei (true positive): this is the number of nuclei centers in the ground truth which are marked positive for that biomarker and are classified as positive for the biomarker heat-map.  Number of missed nuclei (false negative): this is the number of nuclei centers in the ground truth which are marked positive for that biomarker and are classified as negative in the biomarker heat-map.  Number of falsely labeled nuclei (false positive): this is the number of nuclei centers in the ground truth which are not marked positive for that biomarker but are classified as positive for the biomarker heat-map. 103 Table 4.3 shows the validation metrics for the WI-Net performance compared to the human performance (the ground truth). From the above metrics, we calculate the precision (number of true positives divided by the sum of true positives and false positives ), recall (number of true positives divided by total number of positive cases), and the F-score(van Rijsbergen 1979). We selected two IHC images from our data set and using the proposed method, we generated their biomarker heat-maps. These images were never seen before by the network; they were not used (neither were their nucleus images) for training or testing the N-Net or WI-Net. IHC image 1 (high-grade CIN) IHC image 2 (normal epithelium) Image size 2048×2048 pixels 3072×3072 pixels Process time 166 sec 328 sec Biomarker p16 pos. Ki-67 pos. p16 & Ki-67 neg. p16 pos. Ki-67 pos. p16 & Ki-67 neg. True positive 271 157 0 0 18 253 False positive 1 12 0 0 2 28 False negative 0 0 0 0 0 2 Precision 0.996 0.929 - - 0.900 0.900 Recall 1 1 - - 1 0.992 F-score 0.998 0.963 - - 0.947 0.943 Table 4.3. WI-Net performance evaluation of two IHC images taken from two WSIs: based on biomarker heat-maps generated by WI-Net. The ground truth is taken as the labels obtained manually by humans. (Note that in this table, the p16 positive and Ki-67 positive include dual p16 and Ki-67 positive) 104 4.4.2.2 Performance Evaluation in Quantification of Cell Proliferation The true validation of any algorithm in medical imaging requires the evaluation of how it performs in accordance with biological behavior/ outcome. In this section, we investigate the performance of our biomarker labeling algorithm in quantifying the proliferation characteristics of the epithelium for discriminating between different CIN grades. As previously described in Sec. 4.2.1, three H&E-stained slides and one CINtec-stained slide were cut from each biopsy and used in this study. The study pathologist determined the CIN grade for each biopsy and selected the diagnostic areas in the H&E-stained slide; two trained pathology technicians drew ROIs on the image of CINtec-stained slide, marking those areas. More than one region could be indicative of worst diagnosis, therefore the technicians drew as many ROIs of any size as needed on each image (the ROIs do not overlap). The coordinates of these ROIs were saved and used in validation. The proposed biomarker labeling algorithm in Sec. 4.4.1.3 was performed on all CINtec-stained images, and heat-maps of biomarkers (Ki-67 and P16, in this study) were obtained for each image. As mentioned before, the input to the algorithm were raw RGB images of arbitrary sizes selected from immunostained whole slide images obtained from scanner. A total of 61 IHC images, varying in size from 2048×1024 up to 17408×17408 pixels, were analyzed. The entire process was fully automated. It was fast (approximately 30 seconds for analyzing an image of size 1024×1024 pixels). For each case, the coordinates of the ROIs drawn by technicians were used to obtain the equivalent ROIs on each heat-map. Each ROI was then divided into n layers, all of which were parallel to the basal layer and evenly distributed from the basal membrane to the top of the epithelium (Fig. 4.12). 105 Figure 4.12. (A) The ROI selected on an IHC image, and (B) evenly distributed layers in the ROI parallel to the basal layer. Using these layers, we obtained the percentage of positive pixels for each stain in each layer. The graphs in Fig. 4.13 show the average percentage of positive pixels in different layers of epithelium. A B106 Figure 4.13. The average percentage of pixels classified as (A) Ki-67 positive, (B) p16 positive, and (C) both Ki-67 and p16 positive in different layers of images of cervical epithelium as assessed for each pathology grade. Layer 1 is the first layer (i.e. basal layer) and layer 16 is the superficial layer. ABC%107 The results shown in Fig. 4.13 were obtained by analyzing a total of 94 ROIs marked on 61 IHC images. Out of these 61 slides, 16 were CIN3 (31 ROIs), 13 were CIN2 (19 ROIs), 15 were CIN1 (24 ROIs), and 17 were negative (20 ROIs). The graphs in Fig. 4.14-17 show the percentage of positive pixels in different epithelial layers within different pathology grades. Testing was done to determine how many layers to use. For different number of layers n from 10 to 20, no significant variation was observed in the results, therefore we chose to set n to be 16. Figure. 4.14. Expression of Ki-67 per layers in normal squamous cervical epithelia. The box plot of the percentage of pixels classified as Ki-67-positive. The percentage of pixels classified as p16-positive, and both Ki-67 and p16-positive in the different layers of normal squamous cervical epithelia were negligible (<1% for all the cases), therefore are not illustrated in this figure. Layer 1 corresponds to the first layer (i.e. basal layer) and layer 16 corresponds to the most superficial layer. (Central point median; box first and third quartiles; whiskers most extreme data points not considered outliers, + outliers) 108 Figure 4.15. Expression of p16 and Ki-67 per layers in CIN1 lesions. The box plot of the percentage of pixels classified as Ki-67-positive (top left), p16-positive (top right), and both Ki-67 and p16-positive (bottom) in the different layers of CIN1 lesions. Layer 1 corresponds to the first layer (i.e. basal layer) and layer 16 corresponds to the most superficial layer. (Central point median; box first and third quartiles; whiskers most extreme data points not considered outliers, + outliers) Ki67 positive pixels %p16 positive pixels %Dual p16 and Ki67 positive pixels %109 Figure 4.16. Expression of p16 and Ki-67 per layers in CIN2 lesions. The box plot of the percentage of pixels classified as Ki-67-positive (top left), p16-positive (top right), and both Ki-67 and p16-positive (bottom) in the different layers of CIN2 lesions. Layer 1 corresponds to the first layer (i.e. basal layer) and layer 16 corresponds to the most superficial layer. (Central point median; box first and third quartiles; whiskers most extreme data points not considered outliers, + outliers) 110 Figure 4.17. Expression of p16 and Ki-67 per layers in CIN3 lesions. The box plot of the percentage of pixels classified as Ki-67-positive (top left), p16-positive (top right), and both Ki-67 and p16-positive (bottom) in the different layers of CIN3 lesions. Layer 1 corresponds to the first layer (i.e. basal layer) and layer 16 corresponds to the most superficial layer. (Central point median; box first and third quartiles; whiskers most extreme data points not considered outliers, + outliers) Ki67 positive pixels %p16 positive pixels %Dual p16 and Ki67 positive pixels %111 As shown in Fig. 4.13, the relative proliferation pattern obtained from the results is in good accordance with the patterns reported in the literature (Guillaud et al. 2014). In normal epithelia, proliferation mainly occurred in first layers (closer to basal membrane) and dropped rapidly in the upper layers (Fig. 4.13 and Fig. 4.14). The patterns observed in CIN1, CIN2 and CIN3 epithelia were similar to each other in the first layers (up to one fourth of the thickness of the epithelium). Interestingly, the proliferation rate was notably higher in the upper layers of CIN3 and CIN2 epithelia comparing to CIN1 epithelia (Fig. 4.13 and Fig. 4.15-17). We observed negligible p16 positive pixels in the layers of normal cervical tissue images. For all the layers, the average percentage of p16-positive pixels in low-grade CIN was noticeably lower than the average of positive pixels in high-grade CIN (Fig. 4.13). The results show that the layer-based co-expression of p16 and Ki-67 as a metric has the potential to be employed for differentiating between normal, low-grade, and high-grade CIN. The dual positive pixels were not seen in any layer of normal tissue. Although the percentage of dual positive pixels were similar in the two first layers of low-grade and high-grade CINs, this percentage in low-grade CIN dropped to half of the percentage in high-grade CINs in upper layers (starting from the half of the thickness of the epithelium). 4.5 Discussion Immunohistochemical analysis is commonly employed for detecting dysplastic cells in pre-neoplastic lesions and for localizing the expression of one or more proteins in a tissue. In IHC images, it is important to localize the cells that express each of the different labels and visualize the different biomarkers. In this chapter, we propose a deep learning-based algorithm for the 112 immunohistochemical analysis of IHC images. The IHC images used in this study were images of selected regions in whole slide images, acquired by whole slide scanners. However, it is worth noting that, if required in an application, the proposed algorithm can be easily employed to process the whole slide image as well. Considering that the input size is not fixed for this algorithm, it can process images of arbitrary sizes using the proper hardware. The results from our method are in accordance with the expert human observers’ classification and compare well with the labeling performed manually by experienced cyto-technicians (See Table 4.3). We also compared the performance of our WI-Net method for localizing p16 and Ki-67 on dual-stained tissues with the performance of the color deconvolution method presented by Ruifrok et al. (Ruifrok and Johnston 2001). We observed in high-grade tissues, their performances were comparable, but in low-grade tissues, where p16 and Ki-67 were missing, our method performed superior to the color deconvolution method (Fig. 4. 11). This could be a result of manual parameter/threshold setting in color deconvolution method. One set of parameters may result in very good labeling in high-grade tissue, while failing in labeling biomarkers in low-grade tissues. In general, adjusting threshold/parameters in such un-mixing techniques could be very subjective. In the approach presented in this paper, there is no need to change any parameters manually. Even for employing our method on unseen data from another institution, there is no need to change the architecture of the neural network or the training procedure. The network could tune its parameters during the training with a small dataset from the new institution. The effort needed to fine-tuning a pre-trained network for each institute is much less than the effort than needed for training the network from scratch. 113 One main challenge that our method successfully tackled was to automatically quantify the proliferation characteristics of the epithelium on dual-stained tissues. Ki-67 and p16 presentation can look different in each slide or even different regions in the same slide. It is very difficult for human observers to identify the cells that are Ki-67 positive in a dual-stained tissue, when the cells are also p16 positive. To human observers, the color appearance of these cells could be very similar to the color appearance of the cells that are p16 positive and Ki-67 negative. We know that most of the time, the cells that are both p16 and Ki-67-positive are labeled as Ki-67-negative cells. This results in under-assessment of Ki-67-positive cells by humans (see Fig. 4.1). Even the expert cyto-technicians that have prepared our dataset for training and testing our N-Net had difficulties to agree on the expression of Ki-67 protein in p16 positive cells. Therefore, we decided to use nuclei images as basic elements for our training. We trained the network with 4523 nuclei images that were p16 and Ki-67 positive or negative, so that the network got exposed to real examples of a p16 or ki-67 positive cell appearance, and learned the features that were related to these protein expressions. The color and spatial information of each pixel in an IHC image (which is an RGB image) were used to identify the different biomarkers and localize the cells expressing each biomarker (or a combination of them) in one integrated step. In this approach, the color information was not disregarded in training the WI-Net, as color plays an important role in immunohistochemical analysis. There is no need to perform any pre-processing on the color RGB images. The color IHC images were fed directly to the network. In contrast, most methods in the literature include complicated processing like an un-mixing technique (e.g. un-mixing in multi-spectral images and color un-mixing in RGB images). For example, in the study by Chen et al.(Chen and 114 Chefd’hotel 2014) (in which convolutional neural networks was employed in immunohistochemical analysis), a sparse color un-mixing algorithm to un-mix the RGB image into different channels representing different markers was used. Then, a convolutional neural network was used as a cell detection tool in one channel. The real ground truth is the biological behavior or lesions outcome, and one can only validate the performance of the IHC biomarker quantification by correlating the results with these biological behavior. Therefore, in addition to visual evaluation of the classification results and comparison against manually-performed labeling, and visual comparison between our method and a color deconvolution method, we used our method to automatically obtain different immuno-stain expression patterns in cervical epithelia with different pathology grades. We have shown that there is a meaningful correlation between these patterns and the pathology grades (Fig. 4.13). Based on these results, it can be concluded that the proposed method can be used for discriminating between different grades. More importantly, it may aid pathologists to distinguish between CIN1 and CIN2 in difficult cases. Since the diagnosis of CIN1 or CIN2 may eventually result in different therapeutic management, any novel method (including imaging biomarker-based method) that can help pathological decision should be carefully assessed. We should also consider the fact that deep learning-based algorithms are black box predictors which could make them especially risky in a clinical setting. Now that such algorithms and tools are being developed, extensive studies over multiple institutions should be done to examine the reliability of these algorithms. 115 4.6 Summary and Conclusion In this chapter, we trained an end-to-end network for pixel-wise prediction of molecular biomarkers in immunostained whole slide images. The results demonstrate that the proposed automatic labeling method compare well (average F-score of 0.96) with the labeling (p16 positive, Ki-67 positive, p16 and Ki-67 positive, and p16 and Ki-67 negative) performed manually by expert cyto-technicians. The advantage of this method is that it eliminates the need for finding and quantifying sophisticated features of those cells that express every specific biomarker (p16 and Ki-67 in our application). Therefore, this approach is potentially more reproducible than methods that rely on extracting pre-defined or hand-crafted features of nuclei or cell appearance. In addition, we have shown that it provides an accurate assessment of epithelial proliferation characteristics that could be helpful in delineating disease grades. It should be noted that this approach could be easily applied for different tissue types and different stain combinations and other research questions in modern pathology. 116 Chapter 5: Conclusion s and Future Work 5.1 Summary In this thesis, we presented approaches to improve the accuracy and effectiveness of cervical cancer screening and diagnosis procedure. Currently, patients with abnormal Pap-smears are referred to the colposcopy clinic. In the clinic, a gynecologist examines the cervix by a colposcope and acquires biopsies from abnormal areas and sends them to the pathology lab for further diagnosis. In the pathology lab, a pathologist cuts the biopsies into histological sections and reviews them under an optical microscope to provide a diagnosis. In this thesis, we showed that the accuracy and effectiveness of the procedure in the clinic could be enhanced by implementing in vivo confocal imaging in the clinic. We showed that confocal imaging has the capability to replace colposcopy examination that suffers from a relatively low specificity for detecting precancerous lesions. In addition, we developed automatic algorithms for histological and IHC analysis that could empower the pathologists and improve the speed and accuracy of identifying high-risk precancerous lesions in the pathology lab. The data set required for the studies presented in this thesis was not available at the time I started my PhD work. In order to build the data set, we collected normal and abnormal cervical biopsies (over a period of 22 months) from 50 patients with abnormal Pap-smear test who were referred to colposcopy clinic. In the clinic, a gynecologist examined the cervix and provided a dysplasia grading (normal, low-grade, and high-grade CIN) based on the cervix appearance under colposcopy, and then obtained normal and abnormal cervical biopsies. For each biopsy, I acquired ex vivo confocal fluorescence images. Confocal images were en face (imaging plane 117 parallel to the surface of epithelium) and collected at multiple epithelial depths. The biopsies were then processed in the pathology lab and histological slides were obtained from each of them. For each biopsy, different slides were stained with H&E, Thionin-Feulgen, and immuno-stain. The sections that were stained with H&E were reviewed by a pathologist to provide pathological diagnosis for the corresponding biopsies. Then, I acquired transverse images (imaging plane perpendicular to the surface of epithelium) of histological sections cut from biopsies, including Thionin-Feulgen stained and immune-stained slides. We used CI-Slide (which is an imaging calibration slide (Sheikhzadeh et al. 2014)) for calibrating the imaging systems that were used to collect images. The obtained comprehensive data set enabled me to simultaneously investigate two approaches for improving cervical neoplasia diagnosis procedure: using in vivo confocal imaging, and employing enhanced quantitative digital pathology methods. To study the potential of in vivo confocal imaging for detecting and grading cervical dysplasia, and utilize this technology in the clinic (to replace colposcopy examination), we analyzed the ex vivo confocal imaging results against the gold standard of histopathological interpretation (provided by the pathologist) (Sheikhzadeh, Ward, et al. 2015; Sheikhzadeh et al. 2015). Also, we identified four features for quantifying nuclei phenotype and tissue architecture in confocal images, which were comparable to the similar features calculated from histological images. The features obtained from confocal images were also consistent with classical descriptions for grading of cervical dysplasia based on histopathological criterion of H&E-stained slides (e.g. increased nuclear-cytoplasmic ratio or decreased cell-cell distances). The results of our analyses verified the potential of confocal fluorescence microscopy as an effective quantitative tool for delineating normal cervical tissues, low-grade dysplasia, and high-grade dysplasia. I also 118 demonstrated that the obtained results were comparable when confocal images were collected at different epithelial depths. In addition, we confirmed that the accuracy of high-grade dysplasia detection significantly increases by using confocal microscopy rather than using colposcopic examination which is currently being used in clinical settings for the diagnosis of cervical neoplasia. The obtained findings and results play a critical role for utilizing confocal approaches in an in vivo clinical application. Following our study, a group of researchers at BCCRC started to implement the in vivo confocal microscopy in the clinic (Schlosser et al. 2016), based on our findings and results. Utilizing powerful optical technologies (such as confocal microscopy) in the clinic is very promising. These technologies will enable the clinicians to detect and grade cervical lesions with high sensitivity and specificity in a non-invasive manner. This will decrease the number of unnecessary biopsies that are taken in the current diagnosis procedure. More importantly, these enhanced technologies could provide real time diagnosis in the clinic, with the possibility of immediate treatment by less experienced medical practitioners. In addition to employing more powerful optical technologies in the clinic rather than colposcopy, we can improve the effectiveness of the current procedure for cervical neoplasia diagnosis by employing advanced techniques for quantitative digital pathology. We should enhance the approaches that are currently being used for extracting diagnostic information from histological sections. Such enhancements can lead to less labor/time consuming, and more accurate pathological reviews. As part of this PhD work, I developed a fast and fully automatic deep learning-based framework for labeling and locating two biomarkers (p16 and Ki-67) in images of multi immune-stained cervical tissues (Sheikhzadeh et al. 2016; Sheikhzadeh, Guillaud, and Ward 2016; Sheikhzadeh et al. 2018). These two biomarkers are very challenging to be 119 differentiated and sometimes almost impossible for human observers, as their colour appearances are very similar in IHC images. I demonstrated that the developed framework accurately detects the two biomarkers in these images. It should be noted that by employing deep learning, I eliminated the need for finding sophisticated features of those cells that express every specific biomarker. As a result, the reproducibility of our approach is higher than the approaches that rely on pre-defined or hand-crafted features of nuclei or cell appearance. By comparing the results of biomarker detection and localization obtained from our method, and the results obtained from a colour deconvoluition method that is commonly being used in digital pathology, I demonstrated that our approach was more robust and reproducible. Finally, by defining a layer based analysis of IHC images, I have shown that the developed framework could provide an automatic and accurate assessment of epithelial proliferation characteristics. This assessment plays a significant role in delineating disease grades, but it is very time-consuming for the pathologists to perform it manually. I believe that the proposed IHC analysis method could be used in the pathology lab to increase the accuracy and effectiveness of detection of high risk precancerous lesions. More importantly, this approach could be easily applied for different tissue types and different stain combinations and other research questions in modern pathology. 5.2 Contributions The following contributions were made in this work:  During the course of my PhD, I collected a thorough data set containing different image modalities from cervical biopsies. For each biopsy, I acquired en-face ex vivo confocal fluorescence images of the biopsies at different epithelial depths, transverse images of 120 histological sections cut from biopsies, including Thionin-Feulgen-stained and immuno-stained sections and IHC-stained sections. This data set could act as a benchmark to assess the power of different image modalities in cervical neoplasia diagnosis, in relation to each other.  I characterized the properties of CI-Slide (which is a quality control imaging slide) and explained how it can be used as a calibration tool for microscopy imaging systems. Quality control is very important for validating the accuracy and reliability of the imaging results, especially when studies are conducted using different imaging systems in different labs/hospitals or during a long period of time.  I identified and quantified four features related to the morphology of nuclei and tissue architecture in confocal fluorescence microscopic images obtained from the cervical biopsies. I demonstrated that these features play key roles in delineating normal cervical tissues, low-grade dysplasia, and high-grade dysplasia.  For the first time, I demonstrated that the effectiveness of confocal imaging for delineating dysplasia grade was consistent even when confocal images were collected at different epithelial depths. This finding is very crucial in utilizing in vivo confocal microscopy for dysplasia diagnosis because it is almost impossible to accurately determine the epithelial depth in which images were acquired in in vivo imaging. 121  I verified that the accuracy of high-grade dysplasia detection significantly increases by using confocal microscopy for the diagnosis of cervical neoplasia rather than using the current colposcopic examination in the clinical settings.  I developed a fast and fully automatic deep learning-based framework for labeling and localizing different biomarkers in images of multi immunostained-cervical tissues.  The framework I developed, detects different biomarkers with high accuracy in multi-stained tissues, even when it is very difficult for human observers to do so. In addition, I demonstrated that the performance of my algorithm was more robust and reproducible comparing to the performance of the state of the art algorithms that are currently being used for this purpose in digital pathology.  By defining a layer based analysis of IHC images, I showed that the developed framework can automatically extract useful information (such as cell proliferation in the tissue) which could be employed for automatic prediction of cervical neoplasia grade. 5.3 Future Areas of Research Many interesting areas for further research can be proposed based on this work. A few of these research areas are outlined below:  It is an inevitable fact that all the data in the medical field is transforming from analog format to digital format. This is resulting in huge amount of complicated digital data that 122 needs to be translated into meaningful biologic/diagnostic outcome; which makes powerful techniques like deep learning an inseparable part of the future of medical technologies. Based on the needs of the medical field, comprehensive studies should be performed to develop meaningful performance evaluation for such techniques before applying them in clinical settings. For example, in the case of digital pathology, the true strength of these techniques in different pathology applications could be evaluated by performing sufficient reader agreement studies to compare the performance of pathologists to that of developed algorithms.  The fact that deep learning-based algorithms are black box predictors, make them especially risky for being used in a clinical setting. Now that such algorithms and tools are being developed for clinical applications, extensive studies over multiple institutions should be done to examine the reliability of these algorithms. Also, to evaluate the true strength of such algorithms in different pathology applications, sufficient human reader studies should be done to compare the performance of pathologists to that of those algorithms.  The IHC analysis approach developed in this thesis could be easily applied for diagnosis of cancer in other organs with different tissue types and different stain combinations and other research questions in modern pathology.  We demonstrated that the deep learning network we developed as part of this thesis performs very well for our application in detecting and locating biomarkers in IHC 123 images. The architecture or parameters of the network could be customized for future needed applications. For example, if delineating nuclei borders is required in an application, the developed networked could be trained by nucleus masks (rather than square images we used, that contained a nucleus and part of cytoplasm).  In the future, we believe that a new generation of pathologists/clinicians should become skilled in selecting and identifying the precancerous lesions on confocal images, rather than colposcopy images or histological images. This will enable real-time diagnosis of high-risk cervical lesions in the clinic. Future studies could be done to provide detailed and organized instructions for interpretation of confocal images.  We observed that confocal microscopy is not always performing well in differentiating normal cervical tissues from low-grade cervical lesions. We believe that new experiments involving deeper tissue examinations by improved staining techniques will help to improve the accuracy of confocal microscopy in this application.  We presented the CI-Slide and analyzed its characteristics. CI-slide could be used for developing Statistical Process Control protocols for real clinical settings. We need to extend the study presented in this thesis to more patterns on CI-Slide and also investigate other whole-slide scanners and imaging systems to enable such developments. 124 Bibliography Adelman, Marisa Rachel. 2014. “Novel Advancements in Colposcopy: Historical Perspectives and a Systematic Review of Future Developments.” Journal of Lower Genital Tract Disease 18 (3): 246–60. doi:10.1097/LGT.0b013e3182a72170. Anderson, G, C MacAulay, J Matisic, D Garner, and B Palcic. 1997. “The Use of an Automated Image Cytometer for Screening and Quantitative Assessment of Cervical Lesions in the British Columbia Cervical Smear Screening Programme.” Cytopathology : Official Journal of the British Society for Clinical Cytology 8 (5): 298–312. doi:10.1111/j.1365-2303.1997.tb00554.x. Araujo, T, G Aresta, E Castro, J Rouco, P Aguiar, C Eloy, A Polonia, and A Campilho. 2017. “Classification of Breast Cancer Histology Images Using Convolutional Neural Networks.” PLoS One 12 (6): 1–14. doi:10.1371/journal.pone.0177544. Berg, M De, O Cheong, M Van Kreveld, and M Overmars. 2008. Computational Geometry: Algorithms and Applications. Computational Geometry. Vol. 17. doi:10.2307/3620533. Blaustein, A, and R J Kurman. 2002. Blaustein’s Pathology of the Female Genital Tract. Blaustein’s Pathology of the Female Genital Tract. Springer. https://books.google.ca/books?id=tuKGMxGRKa8C. Bosch, F. Xavier, M. Michele Manos, Nubia Muñoz, Mark Sherman, Angela M. Jansen, Julian Peto, Mark H. Schiffman, Victor Moreno, Robert Kurman, and Keerti V. Shan. 1995. “Prevalence of Human Papillomavirus in Cervical Cancer: A Worldwide Perspective.” Journal of the National Cancer Institute 87 (11): 796–802. doi:10.1093/jnci/87.11.796. Buys, Timon P H, Scott B. Cantor, Martial Guillaud, Karen Adler-Storthz, Dennis D. Cox, Clement Okolo, Oyedunni Arulogon, et al. 2012. “Optical Technologies and Molecular Imaging for Cervical Neoplasia: A Program Project Update.” Gender Medicine 9 (1 SUPPL.): 1–18. doi:10.1016/j.genm.2011.08.002. Canny, J. 1986. “A Computational Approach to Edge Detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (6): 679–98. doi:10.1109/TPAMI.1986.4767851. Cantor, Scott B, Marylou Cárdenas-Turanzas, Dennis D Cox, E Neely Atkinson, Graciela M 125 Nogueras-Gonzalez, J Robert Beck, Michele Follen, and J L Benedet. 2008. “Accuracy of Colposcopy in the Diagnostic Setting Compared with the Screening Setting.” Obstetrics and Gynecology 111 (1): 7–14. doi:10.1097/01.AOG.0000295870.67752.b4. Cantor, Scott B, Jose-Miguel Yamal, Martial Guillaud, Dennis D Cox, E Neely Atkinson, John L Benedet, Dianne Miller, et al. 2011. “Accuracy of Optical Spectroscopy for the Detection of Cervical Intraepithelial Neoplasia: Testing a Device as an Adjunct to Colposcopy.” International Journal of Cancer. Journal International Du Cancer 128 (5): 1151–68. doi:10.1002/ijc.25667. Carignan, Charles S, and Yukako Yagi. 2012. “Optical Endomicroscopy and the Road to Real-Time, in Vivo Pathology: Present and Future.” Diagnostic Pathology 7 (1): 98. doi:10.1186/1746-1596-7-98. Carlson, Alicia L., Lezlee G. Coghlan, Ann M. Gillenwater, and Rebecca R. Richards-Kortum. 2007. “Dual-Mode Reflectance and Fluorescence near-Video-Rate Confocal Microscope for Architectural, Morphological and Molecular Imaging of Tissue.” Journal of Microscopy 228 (1): 11–24. doi:10.1111/j.1365-2818.2007.01818.x. Carlson, K, I Pavlova, T Collier, M Descour, M Follen, and R Richards-Kortum. 2005. “Confocal Microscopy: Imaging Cervical Precancerous Lesions.” Gynecol Oncol 99 (3 Suppl 1): S84-8. doi:S0090-8258(05)00530-5 [pii]\r10.1016/j.ygyno.2005.07.049. Chen, Ting, and Christophe Chefd’hotel. 2014. “Deep Learning Based Automatic Immune Cell Detection for Immunohistochemistry Images.” In Machine Learning in Medical Imaging, 2014, LNCS 8679:17–24. doi:10.1007/978-3-319-10581-9_3. Chiu, David, Martial Guillaud, Dennis Cox, Michele Follen, and Calum MacAulay. 2004. “Quality Assurance System Using Statistical Process Control: An Implementation for Image Cytometry.” Cellular Oncology : The Official Journal of the International Society for Cellular Oncology. http://www.ncbi.nlm.nih.gov/pubmed/15371646. Cireşan, Dan C., Alessandro Giusti, Luca M. Gambardella, and Jürgen Schmidhuber. 2013. “Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks.” In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8150 LNCS:411–18. doi:10.1007/978-3-642-40763-5_51. 126 Cireşan, Dan, Ueli Meier, and Juergen Schmidhuber. 2012. “Multi-Column Deep Neural Networks for Image Classification.” International Conference of Pattern Recognition, no. February: 3642–49. doi:10.1109/CVPR.2012.6248110. Collier, Tom, Martial Guillaud, Michele Follen, Anais Malpica, and Rebecca Richards-Kortum. 2007. “Real-Time Reflectance Confocal Microscopy: Comparison of Two-Dimensional Images and Three-Dimensional Image Stacks for Detection of Cervical Precancer.” Journal of Biomedical Optics 12 (2): 24021. doi:10.1117/1.2717899. Collier, Tom, Alicia Lacy, Rebecca Richards-Kortum, Anais Malpica, and Michele Follen. 2002. “Near Real-Time Confocal Microscopy of Amelanotic Tissue: Detection of Dysplasia in Ex Vivo Cervical Tissue.” In Academic Radiology, 9:504–12. doi:10.1016/S1076-6332(03)80326-4. Degueldre, M, J Vandromme, Wind A de, and F Feoli. 2015. “Real-Time in-Vivo Microscopic Imaging of the Cervix Using Confocal Laser Endomicroscopy: Preliminary Observations and Feasibility Study.” Eur.J.Cancer Prev., no. 1473–5709 (Electronic): 335–43. doi:10.1097/CEJ.0000000000000188. Doudkine, A., C. Macaulay, N. Poulin, and B. Palcic. 1995. “Nuclear Texture Measurements in Image Cytometry.” Pathologica. Fitzmaurice, Christina, Christine Allen, Ryan M Barber, Lars Barregard, Zulfiqar A Bhutta, Hermann Brenner, Daniel J Dicker, et al. 2017. “Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived with Disability, and Disability-Adjusted Life-Years for 32 Cancer Groups, 1990 to 2015: A Systematic Analysis for the Global Burden of Disease Study.” JAMA Oncology 3 (4). American Medical Association: 524–48. Gallas, Brandon D., Marios A. Gavrielides, Catherine M. Conway, Adam Ivansky, Tyler C. Keay, Wei-Chung Cheng, Jason Hipp, and Stephen M. Hewitt. 2014. “Evaluation Environment for Digital and Analog Pathology: A Platform for Validation Studies.” Journal of Medical Imaging 1 (3): 37501. doi:10.1117/1.JMI.1.3.037501. Gallwas, Julia K S, Lisa Turk, Herbert Stepp, Susanna Mueller, Robert Ochsenkuehn, Klaus Friese, and Christian Dannecker. 2011. “Optical Coherence Tomography for the Diagnosis of Cervical Intraepithelial Neoplasia.” Lasers in Surgery and Medicine 43 (3): 206–12. 127 doi:10.1002/lsm.21030. Garner, David. 1994. “The Cyto-Savant^< TM> System.” The Automation of Cervical Cancer Screening. Igaku-Shoin Medical Publishers. Inc. Gavrielides, M A, C Conway, N O’Flaherty, B D Gallas, and S M Hewitt. 2014. “Observer Performance in the Use of Digital and Optical Microscopy for the Interpretation of Tissue-Based Biomarkers.” Analytical Cellular Pathology 2014 (2): 157308. doi:http://dx.doi.org/10.1155/2014/157308. Gavrielides, Marios A, Brandon D Gallas, and Stephen M Hewitt. 2015. “Uncertainty in the Assessment of Immunohistochemical Staining with Optical and Digital Microscopy: Lessons from a Reader Study.” In Medical Imaging 2015: Digital Pathology, 9420:94200V. Giroud, F, G Haroske, A Reith, and A Böcking. 1998. “Part II: Specific Recommendations for Quality Assurance.” Analytical Cellular Pathology 17 (4). Hindawi Publishing Corporation: 201–8. GLOBOCAN. http://globocan.iarc.fr/Default.aspx. Guillaud, M, J B Matthews, A Harrison, C MacAulay, and K Skov. 1997. “A Novel Image Cytometric Method for Quantitation of Immunohistochemical Staining of Cytoplasmic Antigens.” Analytical Cellular Pathology : The Journal of the European Society for Analytical Cellular Pathology 14 (2): 87–99. Guillaud, Martial, Timon P H Buys, Anita Carraro, Jagoda Korbelik, Michele Follen, Michael Scheurer, Karen Adler Storthz, Dirk Van Niekerk, and Calum E. MacAulay. 2014. “Evaluation of HPV Infection and Smoking Status Impacts on Cell Proliferation in Epithelial Layers of Cervical Neoplasia.” PLoS ONE 9 (9). doi:10.1371/journal.pone.0107088. Guillaud, Martial, Dennis Cox, Karen Adler-Storthz, Anais Malpica, Gregg Staerkel, Jasenka Matisic, Dirk Van Niekerk, Neal Poulin, Michele Follen, and Calum MacAulay. 2004. “Exploratory Analysis of Quantitative Histopathology of Cervical Intraepithelial Neoplasia: Objectivity, Reproducibility, Malignancy-Associated Changes, and Human Papillomavirus.” Cytometry 60A (1): 81–89. doi:10.1002/cyto.a.20034. Guillaud, Martial, Dennis Cox, Anais Malpica, Gregg Staerkel, Jasenka Matisic, Dirk Van Niekirk, Karen Adler-Storthz, Neal Poulin, Michele Follen, and Calum MacAulay. 2004. 128 “Quantitative Histopathological Analysis of Cervical Intra-Epithelial Neoplasia Sections: Methodological Issues.” Cellular Oncology : The Official Journal of the International Society for Cellular Oncology 26: 31–43. Guillaud, Martial, Rebecca Richards-Kortum, and Michele Follen. 2007. “Paradigm Shift: A New Breed of Pathologist.” Gynecologic Oncology 107 (1 Suppl 1). United States: S46-9. doi:10.1016/j.ygyno.2007.07.036. Gurcan, M.N., L.E. Boucheron, Ali Can, Anant Madabhushi, N.M. Rajpoot, and B. Yener. 2009. “Histopathological Image Analysis: A Review.” IEEE Reviews in Biomedical Engineering 2: 147–71. doi:10.1109/RBME.2009.2034865. Hallani, S. El, C. F. Poh, C. E. Macaulay, M. Follen, M. Guillaud, and P. Lane. 2013. “Ex Vivo Confocal Imaging with Contrast Agents for the Detection of Oral Potentially Malignant Lesions.” Oral Oncology 49 (6): 582–90. doi:10.1016/j.oraloncology.2013.01.009. Han, Zhongyi, Benzheng Wei, Yuanjie Zheng, Yilong Yin, Kejian Li, and Shuo Li. 2017. “Breast Cancer Multi-Classification from Histopathological Images with Structured Deep Learning Model.” Scientific Reports 7 (1). doi:10.1038/s41598-017-04075-z. Hipp, Jason, Jerome Cheng, Stephanie Daignault, Jefferey Sica, Michael C. Dugan, David Lucas, Yukako Yagi, Stephen Hewitt, and Ulysses J. Balis. 2011. “Automated Area Calculation of Histopathologic Features Using SIVQ.” Analytical Cellular Pathology 34 (5): 265–75. doi:10.3233/ACP-2011-0025. Hocking, R R. 2005. Methods and Applications of Linear Models: Regression and the Analysis of Variance. Wiley Series in Probability and Statistics. Wiley. https://books.google.ca/books?id=zVy0So2UAxMC. Honda, Hisao. 1978. “Description of Cellular Patterns by Dirichlet Domains: The Two-Dimensional Case.” Journal of Theoretical Biology 72 (3). doi:10.1016/0022-5193(78)90315-6. Irshad, Humayun, Antoine Veillard, Ludovic Roux, and Daniel Racoceanu. 2014. “Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review&#x2014;Current Status and Future Potential.” IEEE Reviews in Biomedical Engineering 7: 97–114. doi:10.1109/RBME.2013.2295804. Jain, Viren, Joseph F. Murray, Fabian Roth, Srinivas Turaga, Valentin Zhigulin, Kevin L. 129 Briggman, Moritz N. Helmstaedter, Winfried Denk, and H. Sebastian Seung. 2007. “Supervised Learning of Image Restoration with Convolutional Networks.” In 2007 IEEE 11th International Conference on Computer Vision, 1–8. IEEE. doi:10.1109/ICCV.2007.4408909. Janowczyk, Andrew, and Anant Madabhushi. 2016. “Deep Learning for Digital Pathology Image Analysis: A Comprehensive Tutorial with Selected Use Cases.” Journal of Pathology Informatics 7 (1): 29. doi:10.4103/2153-3539.186902. Jester, James V., Peter M. Andrews, W. Matthew Petroll, Michael A. Lemp, and H. Dwight Cavanagh. 1991. “In Vivo, Realtime Confocal Imaging.” Journal of Electron Microscopy Technique 18 (1): 50–60. doi:10.1002/jemt.1060180108. Jia, Yangqing, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. “Caffe: Convolutional Architecture for Fast Feature Embedding.” In Proceedings of the 22Nd ACM International Conference on Multimedia, 675–78. MM ’14. New York, NY, USA: ACM. doi:10.1145/2647868.2654889. Johnson, Fred L, Bojana Turic, Roger Kemp, Branko Palcic, Robert Sussman, Kirk G Voelker, and Emory Robinette. 2004. “Improved Diagnostic Sensitivity for Lung Cancer Using an Automated Quantitative Cytology System and Uridine 5’-triphosphate-Induced Sputum Specimens.” Chest 125 (5). Elsevier: 157S--158S. Kamalov, R., M. Guillaud, D. Haskins, A. Harrison, R. Kemp, D. Chiu, M. Follen, and C. MacAulay. 2005a. “A Java Application for Tissue Section Image Analysis.” Computer Methods and Programs in Biomedicine 77 (2): 99–113. doi:10.1016/j.cmpb.2004.04.003. Kayser, K., and H. Stute. 1989. “Minimum Spanning Tree - Voronoi’s Tesselation - Johnson-Mehl Diagrams - Human Lung Carcinoma.” Pathology Research and Practice 185 (5): 729–34. doi:10.1016/S0344-0338(89)80228-6. Keating, Jeffrey T, Aida Cviko, Sabine Riethdorf, Lutz Riethdorf, Bradley J Quade, Deqin Sun, Stefan Duensing, Ellen E Sheets, Karl Munger, and Christopher P Crum. 2001. “Ki-67, Cyclin E, and p16 INK4 Are Complimentary Surrogate Biomarkers for Human Papilloma Virus-Related Cervical Neoplasia.” The American Journal of Surgical Pathology 25 (7): 884–91. doi:10.1097/00000478-200107000-00006. Khojasteh-Lakelayeh, Mehrnoush. 2012. “Methods for Improved Imaging and Analysis of 130 Tissue-Based Biomarkers.” University of British Columbia. Klaes, Ruediger, Tibor Friedrich, Dimitry Spitkovsky, Ruediger Ridder, Wolfgang Rudy, Ulrich Petry, Gisela Dallenbach-Hellweg, Dietmar Schmidt, and Magnus Knebel Von Doeberitz. 2001. “Overexpression of p16ink4a as a Specific Marker for Dysplastic and Neoplastic Epithelial Cells of the Cervix Uteri.” International Journal of Cancer 92 (2): 276–84. doi:10.1002/ijc.1174. Krizhevsky, A, I Sutskever, and G E Hinton. 2012. “ImageNet Classification with Deep Convolutional Neural Networks.” In Advances in Neural Information Processing Systems, 1097–1105. Lecun, Y., L??on Bottou, Yoshua Bengio, and Patrick Haffner. 1998. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86 (11): 2278–2324. doi:10.1109/5.726791. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521 (7553): 436–44. doi:10.1038/nature14539. Litjens, Geert, Clara I. Sánchez, Nadya Timofeeva, Meyke Hermsen, Iris Nagtegaal, Iringo Kovacs, Christina Hulsbergen - van de Kaa, Peter Bult, Bram van Ginneken, and Jeroen van der Laak. 2016. “Deep Learning as a Tool for Increased Accuracy and Efficiency of Histopathological Diagnosis.” Scientific Reports 6 (May): 26286. doi:10.1038/srep26286. MacAulay, C., and B. Palcic. 1990. “An Edge Relocation Segmentation Algorithm.” Analytical and Quantitative Cytology and Histology 12 (3): 165–71. Marcelpoil, R., and Y. Usson. 1992. “Methods for the Study of Cellular Sociology: Voronoi Diagrams and Parametrization of the Spatial Relationships.” Journal of Theoretical Biology 154 (3): 359–69. doi:10.1016/S0022-5193(05)80176-6. Miralpeix, Ester, Jordi Genovés, Josep Maria SoléSedeño, Gemma Mancebo, Belen Lloveras, Beatriz Bellosillo, Francesc Alameda, and Ramon Carreras. 2016. “Usefulness of p16 INK4a Staining for Managing Histological Highgrade Squamous Intraepithelial Cervical Lesions.” Modern Pathology Advance Online Publication doi (10): 304–10. doi:10.1038/modpathol.2016.168. Mitchell, Michele Follen, David Schottenfeld, Guillermo Tortolero-Luna, Scott B. Cantor, and Rebecca Richards-Kortum. 1998. “Colposcopy for the Diagnosis of Squamous 131 Intraepithelial Lesions: A Meta-Analysis.” Obstetrics and Gynecology 91 (4): 626–31. doi:10.1016/S0029-7844(98)00006-4. Montalto, MichaelC, RobertJ Filkins, and RichardR McKay. 2011. “Autofocus Methods of Whole Slide Imaging Systems and the Introduction of a Second-Generation Independent Dual Sensor Scanning Method.” Journal of Pathology Informatics 2 (1): 44. doi:10.4103/2153-3539.86282. Niekerk, Dirk Van, Martial Guillaud, Jasenka Matisic, John L. Benedet, J. Adrian Freeberg, Michele Follen, and Calum MacAulay. 2007. “p16 and MIB1 Improve the Sensitivity and Specificity of the Diagnosis of High Grade Squamous Intraepithelial Lesions: Methodological Issues in a Report of 447 Biopsies with Consensus Diagnosis and HPV HCII Testing.” Gynecologic Oncology 107 (1): S233–40. doi:10.1016/j.ygyno.2007.07.064. Pierce, Mark C., Yao Yao Guan, Mary Kate Quinn, Xun Zhang, Wen Hua Zhang, You Lin Qiao, Philip Castle, and Rebecca Richards-Kortum. 2012. “A Pilot Study of Low-Cost, High-Resolution Microendoscopy as a Tool for Identifying Women with Cervical Precancer.” Cancer Prevention Research 5 (11): 1273–79. doi:10.1158/1940-6207.CAPR-12-0221. Quinn, Mary K., Tefo C. Bubi, Mark C. Pierce, Mukendi K. Kayembe, Doreen Ramogola-Masire, and Rebecca Richards-Kortum. 2012. “High-Resolution Microendoscopy for the Detection of Cervical Neoplasia in Low-Resource Settings.” PLoS ONE 7 (9). doi:10.1371/journal.pone.0044924. Rajadhyaksha, M, S González, J M Zavislan, R R Anderson, and R H Webb. 1999. “In Vivo Confocal Scanning Laser Microscopy of Human Skin II: Advances in Instrumentation and Comparison with Histology.” The Journal of Investigative Dermatology 113 (3): 293–303. doi:10.1046/j.1523-1747.1999.00690.x. Rashid, Nosheen, Haq Nawaz, Kelvin W C Poon, Franck Bonnier, Salih Bakhiet, Cara Martin, John J. O’Leary, Hugh J. Byrne, and Fiona M. Lyng. 2014. “Raman Microspectroscopy for the Early Detection of Pre-Malignant Changes in Cervical Tissue.” Experimental and Molecular Pathology 97 (3): 554–64. doi:10.1016/j.yexmp.2014.10.013. Reuschenbach, Miriam, Mirjam Seiz, Christina Von Knebel Doeberitz, Svetlana Vinokurova, Alexander Duwe, Ruediger Ridder, Heike Sartor, Friedrich Kommoss, Dietmar Schmidt, and Magnus Von Knebel Doeberitz. 2012. “Evaluation of Cervical Cone Biopsies for 132 Coexpression of p16INK4a and Ki-67 in Epithelial Cells.” International Journal of Cancer 130 (2): 388–94. doi:10.1002/ijc.26017. Richart, R M. 1990. “A Modified Terminology for Cervical Intraepithelial Neoplasia.” Obstetrics and Gynecology 75 (1): 131–33. http://www.ncbi.nlm.nih.gov/pubmed/2296409. Rijsbergen, C J van. 1979. “Information Retrieval. 1979.” Butterworth. Ruifrok, A. C., and D. A. Johnston. 2001. “Quantification of Histochemical Staining by Color Deconvolution.” Analytical and Quantitative Cytology and Histology 23 (4): 291–99. doi:10.1097/00129039-200303000-00014. Russakovsky, Olga, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, et al. 2015. “ImageNet Large Scale Visual Recognition Challenge.” International Journal of Computer Vision 115 (3): 211–52. doi:10.1007/s11263-015-0816-y. Schindelin, Johannes, Ignacio Arganda-Carreras, Erwin Frise, Verena Kaynig, Mark Longair, Tobias Pietzsch, Stephan Preibisch, et al. 2012. “Fiji: An Open Source Platform for Biological Image Analysis.” Nature Methods 9 (7): 676–82. doi:10.1038/nmeth.2019.Fiji. Schlosser, Colin, Nico Bodenschatz, Sylvia Lam, Marette Lee, Jessica N. McAlpine, Dianne M. Miller, Dirk J. T. Van Niekerk, et al. 2016. “Fluorescence Confocal Endomicroscopy of the Cervix: Pilot Study on the Potential and Limitations for Clinical Implementation.” Journal of Biomedical Optics 21 (12): 126011. doi:10.1117/1.JBO.21.12.126011. Sheikhzadeh, Fahime, Anita Carraro, Jagoda Korbelik, Calum MacAulay, Martial Guillaud, and Rabab K Ward. 2016. “Automatic Labeling of Molecular Biomarkers on a Cell-by-Cell Basis in Immunohistochemistry Images Using Convolutional Neural Networks.” In Medical Imaging 2016: Digital Pathology, 9791:97910R. Sheikhzadeh, Fahime, Martial Guillaud, and Rabab K Ward. 2016. “Automatic Labeling of Molecular Biomarkers of Whole Slide Immunohistochemistry Images Using Fully Convolutional Networks.” arXiv Preprint arXiv:1612.09420. Sheikhzadeh, Fahime, Rabab K Ward, Anita Carraro, Zhao Yang Chen, Dirk van Niekerk, Dianne Miller, Tom Ehlen, et al. 2015. “Quantification of Confocal Fluorescence Microscopy for the Detection of Cervical Intraepithelial Neoplasia.” Biomedical Engineering Online 14 (1). BioMed Central: 96. Sheikhzadeh, Fahime, Rabab K Ward, Anita Carraro, Zhaoyang Chen, Dirk van Niekerk, Calum 133 MacAulay, Michele Follen, Pierre Lane, and Martial Guillaud. 2015. “Confocal Fluorescence Microscopy for Detection of Cervical Preneoplastic Lesions.” In SPIE- Medical Imaging 2015: Digital Pathology, 9420:942009. Sheikhzadeh, Fahime, Rabab K Ward, Dirk van Niekerk, and Martial Guillaud. 2018. “Automatic Labeling of Molecular Biomarkers of Immunohistochemistry Images Using Fully Convolutional Networks.” PloS One 13 (1). Public Library of Science: e0190783. Sheikhzadeh, Fahime, Qian Ye, Nasir Zulkafly, Anita Carraro, Jagoda Korbelic, Zhaoyang Chen, Alan Harrison, et al. 2014. “CI Slide: Calibration Slide for Quantitative Microscopy Imaging in Absorbance.” In SPIE-Photonics 2014, 8945:894507. Shelhamer, Evan, Jonathan Long, and Trevor Darrell. 2016. “Fully Convolutional Networks for Semantic Segmentation,” May. IEEE. http://arxiv.org/abs/1605.06211. Soille, P. 2003. Morphological Image Analysis - Principles and Applications. Springer. doi:10.1007/978-3-662-05088-0. Solomon, Diane, Diane Davey, Robert Kurman, Ann Moriarty, Dennis O’Connor, Marianne Prey, Stephen Raab, et al. 2002. “The 2001 Bethesda System: Terminology for Reporting Results of Cervical Cytology.” Jama 287 (16): 2114–19. doi:10.1097/00006254-200208000-00015. Sudbø, Jon, Raphael Marcelpoil, and Albrecht Reith. 2000. “New Algorithms Based on the Voronoi Diagram Applied in a Pilot Study on Normal Mucosa and Carcinomas.” Analytical Cellular Pathology : The Journal of the European Society for Analytical Cellular Pathology 21 (2): 71–86. http://www.ncbi.nlm.nih.gov/pubmed/11310643. Tan, J., M. A. Quinn, J. M. Pyman, P. M. Delaney, and W. J. McLaren. 2009. “Detection of Cervical Intraepithelial Neoplasia in Vivo Using Confocal Endomicroscopy.” BJOG: An International Journal of Obstetrics and Gynaecology 116 (12): 1663–70. doi:10.1111/j.1471-0528.2009.02261.x. Thun, Michael J., John Oliver DeLancey, Melissa M. Center, Ahmedin Jemal, and Elizabeth M. Ward. 2009. “The Global Burden of Cancer: Priorities for Prevention.” Carcinogenesis 31 (1): 100–110. doi:10.1093/carcin/bgp263. 134 Treanor, Darren, Brandon D Gallas, Marios A Gavrielides, and Stephen M Hewitt. 2015. “Evaluating Whole Slide Imaging: A Working Group Opportunity.” Journal of Pathology Informatics 6: 4. V., Mace, Ahluwalia A., Coron E., Le Rhun M., Boureille A., Bossard C., Mosnier J.-F., Matysiak-Budnik T., and Tarnawski A.S. 2015. “Confocal Laser Endomicroscopy: A New Gold Standard for the Assessment of Mucosal Healing in Ulcerative Colitis.” Journal of Gastroenterology and Hepatology (Australia) 30 (S1): 85–92. doi:10.1111/jgh.12748. Wang, Dayong, Aditya Khosla, Rishab Gargeya, Humayun Irshad, and Andrew H. Beck. 2016. “Deep Learning for Identifying Metastatic Breast Cancer,” June. http://arxiv.org/abs/1606.05718. Wang, Haibo, Angel Cruz-Roa, Ajay Basavanhally, Hannah Gilmore, Natalie Shih, Mike Feldman, John Tomaszewski, Fabio Gonzalez, and Anant Madabhushi. 2014. “Mitosis Detection in Breast Cancer Pathology Images by Combining Handcrafted and Convolutional Neural Network Features.” Journal of Medical Imaging 1 (3): 34003. doi:10.1117/1.JMI.1.3.034003. Webster, J. D., and R. W. Dunstan. 2014. “Whole-Slide Imaging and Automated Image Analysis: Considerations and Opportunities in the Practice of Pathology.” Veterinary Pathology 51 (1): 211–23. doi:10.1177/0300985813503570. White, W M, M Rajadhyaksha, S González, R L Fabian, and R R Anderson. 1999. “Noninvasive Imaging of Human Oral Mucosa in Vivo by Confocal Reflectance Microscopy.” The Laryngoscope 109 (10): 1709–17. doi:10.1097/00005537-199910000-00029. Wright, TC, JT Cox, LS Massad, LB Twiggs, and EJ Wilkinson. 2002. “2001 Consensus Guidelines for the Management of Women with Cervical Cytological Abnormalities.” JAMA : The Journal of the American Medical Association 287 (16): 2120–29. http://www.ncbi.nlm.nih.gov/pubmed/11966387. Wright, Thomas C, Brigitte M Ronnett, Robert J Kurman, and Alex Ferenczy. 2011. “Precancerous Lesions of the Cervix.” In Blaustein’s Pathology of the Female Genital Tract, edited by Robert J Kurman, Lora Hedrick Ellenson, and Brigitte M Ronnett, 193–252. Boston, MA: Springer US. doi:10.1007/978-1-4419-0489-8_5. Xu, Jun, Xiaofei Luo, Guanhao Wang, Hannah Gilmore, and Anant Madabhushi. 2016. “A Deep 135 Convolutional Neural Network for Segmenting and Classifying Epithelial and Stromal Regions in Histopathological Images.” Neurocomputing 191 (May): 214–23. doi:10.1016/j.neucom.2016.01.034.
Thesis/Dissertation
eng
Biomedical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Improving cervical neoplasia diagnosis via novel in vivo imaging technologies and deep learning algorithms
Text