UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Excitation/emission spectroscopy with multi-channel imaging guidance for skin disease diagnosis Yu, Yingqiu 2010

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2011_spring_yu_yingqiu.pdf [ 25.12MB ]
Metadata
JSON: 24-1.0071505.json
JSON-LD: 24-1.0071505-ld.json
RDF/XML (Pretty): 24-1.0071505-rdf.xml
RDF/JSON: 24-1.0071505-rdf.json
Turtle: 24-1.0071505-turtle.txt
N-Triples: 24-1.0071505-rdf-ntriples.txt
Original Record: 24-1.0071505-source.json
Full Text
24-1.0071505-fulltext.txt
Citation
24-1.0071505.ris

Full Text

Excitation/Emission Spectroscopy with Multi-Channel Imaging Guidance for Skin Disease Diagnosis by Yingqiu Yu B.Eng., Huazhong University of Science and Technology, 2007 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF APPLIED SCIENCE in  The Faculty of Graduate Studies (Biomedical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) December 2010  © Yingqiu Yu, 2010  ABSTRACT Skin cancer is the most prevalent type of cancer and early stage diagnosis plays an important role in improving the cure rate. This project is to investigate non-invasive techniques for skin disease diagnosis. Nonlinear Excitation-Emission-Matrix (EEM) spectroscopy is chosen as the core technique in the project, which provides us with biochemical and physiological information for skin tissue characterization. To assist the EEM analysis, multiple imaging channels acquire the image signal in real-time, simultaneously with EEM acquisition. The image-guided EEM provides a more comprehensive set of data containing both biochemical and morphological information of the constituents of skin tissue. A nonlinear EEM system is built by adding a spectroscopy detection path to a multimodality microscopy system. Software programs are custom designed to realize signal synchronization of system components, real-time image acquisition and processing, and EEM measurements. Calibrations and measurements of key parameters are carried out to ensure the accuracy of imaging-guided nonlinear EEM measurements. Preliminary experiments are carried out on purified endogenous skin fluorophores as well as normal and diseased human skin tissue sections. Nonlinear EEMs of normal and seborrheic keratosis skin tissues are compared. Preliminary results from the study demonstrate great potential of using the image-guided nonlinear EEM for skin diseases diagnosis.  ii  PREFACE  The use of clinical materials involved in the research for this thesis was approved by the University of British Columbia Clinical Research Ethics Board (Certificate # H96-70499).  iii  TABLE OF CONTENTS Abstract ................................................................................................................. ii Preface .................................................................................................................. iii Table of Contents .................................................................................................. iv List of Tables ........................................................................................................ vi List of Figures ...................................................................................................... vii List of Abbreviations .......................................................................................... xiii Acknowledgements ............................................................................................. xiv Dedication ............................................................................................................xv 1 Introduction ........................................................................................................ 1 1.1 Basics of Skin Anatomy and Skin Cancer ..................................................... 1 1.1.1 Skin Anatomy ......................................................................................... 1 1.1.2 Skin Cancer............................................................................................. 3 1.2 Optical Imaging Techniques for Diagnosis ................................................... 4 1.2.1 Confocal Laser Scanning Microscopy ..................................................... 4 1.2.2 Two-Photon Excitation Fluorescence (TPEF) ......................................... 7 1.2.3 Second Harmonic Generation (SHG) ...................................................... 9 1.3 Excitation-Emission-Matrix for Diagnosis ...................................................10 1.3.1 Spectral Diagnosis .................................................................................10 1.3.2 Excitation-Emission-Matrix ...................................................................13 1.4 Personal Contribution ..................................................................................18 2 System Development .........................................................................................19 2.1 System Optical Setup ...................................................................................19 2.1.1 Multimodality Microscopy and Spectroscopy System ............................19 2.1.2 EEM Sub-System ...................................................................................21 2.2 Signal Synchronization for Scanner and Detection Unit ...............................23 2.3 Image Acquisition and Processing ...............................................................27 iv  2.3.1 Attenuator Module for Optimization of PMT Video Output...................27 2.3.2 Linearization Processing to Correct Image Distortion ............................31 2.4 EEM Acquisition .........................................................................................35 2.5 System Performance ....................................................................................38 3 Calibration and Measurement of System Parameters..........................................39 3.1 Excitation Beam Path ...................................................................................39 3.1.1 Excitation Power Calibration .................................................................39 3.1.2 Laser Pulse Width Measurement ............................................................43 3.1.3 Laser Spectral Bandwidth Measurement ................................................49 3.1.4 Imaging Resolution Measurements ........................................................50 3.2 Emission Beam Path ....................................................................................53 3.2.1 Spectrometer Wavelength Calibration ....................................................53 3.2.2 Emission Beam Path Intensity Response Calibration .............................56 4 Experiments on Biological Samples ...................................................................58 4.1 Three Imaging Channels to Guide EEM Acquisition ...................................58 4.2 Imaging Guided Nonlinear Excitation-Emission-Matrix ..............................61 4.2.1 Experiments on Purified Fluorophores ...................................................61 4.2.2 Experiments on Human Skin Tissue ......................................................70 5 Conclusions and Future Work ............................................................................78 Bibliography .........................................................................................................79 Appendices ...........................................................................................................85 Appendix A: Details of driving boards for the X-Y scanners .............................85 Appendix B: Labview Program for Signal Synchronization ...............................88 Appendix C: Automated EEM Acquisition Program..........................................92 Appendix D: Program code for real-time imaging processing and display .........98 Appendix E: Attenuator Box Design ................................................................104  v  LIST OF TABLES  Table  2.1  Performance  of  the  multimodality  microscopy  and  spectroscopy  system…………………………………………………………………....................……38 Table A.1 – CRS Driver Board Connection Pins .............................................................85 Table A.2 – MiniSAX II Driver Board Connection Pins .................................................87  vi  LIST OF FIGURES Figure 1.1 Anatomical structure of skin .……...………………..………………………...2 Figure 1.2 Schematic picture of basic setup of confocal microscope. Illumination beams are shown in green color and emission beams are shown in red color [5].........................................................................................................................................6 Figure 1.3 Schematic drawing illustrating the processes of fluorescence (A) single-photon fluorescence (B) two-photon fluorescence [6]................................……………………….7 Figure 1.4 Two-photon images of various skin layers obtained after excitation at 760 nm and observed through different emission filters. The emission spectra of each layer for different excitation wavelengths are also shown (a) stratum corneum, (b) stratum spinosum and (c) stratum basal layer. Each layer is excited at three different excitation wavelengths: 730 (line), 780 (line with triangle node) and 830 (line with square node) [54] ............................................…………………………………………………………11 Figure 1.5 Tissue collagen and NAD(P)H intrinsic fluorescence EEMs are extracted from variceal asphyxiation measurements. a and b, extracted intrinsic fluorescence EEMs of two components that changes observed during tissue asphyxiation (suffocation). Different colors represent different fluorescence intensities, as indicated by the adjacent color bar scales. c, intrinsic fluorescence EEM of collagen I in powder form. d, NAD(P)H fluorescence from isolated epithelium of freshly excised cervical tissue [64] .........................................…………………………………………………………...14 Figure 1.6 The raw EEM of mouse skin with the fluorescence emission characteristics of 1-tryptophan; 2-collagen; 3-NADH; 4-FAD, and 5-porphyrin [65] ................................................................................................................................... 15 Figure 1.7 Three-dimensional excitation-emission-matrices plot of (a) a purified collagen sample, (b) a purified elastin sample, and (c) an excised human skin dermis. (Laser was tuned from 750 nm to 950 nm in increments of 20 nm.) [66] ….......................…..…….17 Figure 2.1 Optics layout of the multimodality microscopy and spectroscopy system. Beam paths for four channels are shown including confocal, TPEF, SHG and spectroscopy. (In the actual 3-D optics setup of the system, several mirrors are installed to direct the light beam, which are not drawn in this figure.) …………………………..20 Figure 2.2 Optics layout of the EEM sub-system. Cross sections of two ends of the fiber bundle are shown in the zoomed in figures. …………………………………………….22 Figure 2.3 Figure 2.3 Principle of signal synchronization for scanners and detection unit. Yellow dotted line shows the synchronized edges……..………………………………..24  vii  Figure 2.4 Physical connections of the electrical components for signal synchronization.(PMT: Photomultiplier Tube; Sync: Synchronization Signal; TTL: Transistor-Transistor Logic; LPF: Line Per Frame)…..…………………………………25 Figure 2.5 Labview program for signal synchronization (A) Program operation process (B) Labview Program Interface (Program code is shown in the Appendix 2.)……………………………...…………………………………………………………26 Figure 2.6 Principle for setting blanking level on the video signal with the ADC clamp circuit of frame grabber. (A) Standard RS-170A video signal (B) PMT video signal……….…………………………………………………………………………….28 Figure 2.7 Block diagram of the attenuator and multiplexer box to adapt the frame grabber to non-standard video signal generated by PMT. (OPA: Operational Amplifier; MUX: Multiplexer; H.Sync: Horizontal Synchronization Signal.)….…………………..29 Figure 2.8 Effectiveness of the attenuator box. (A) Image acquired without the attenuator box (B) Image acquired with the attenuator box installed. (Bovine Collagen Sample, 30 mw @780 nm excitation, 100 µm × 100 µm Field of View, Each image includes the signals collected with forward and backward scans of the resonant scanner, which are two symmetric parts.)…………………………………………………………………...…….30 Figure 2.9 Resonant scanner working mechanism and sample image to show the distortion caused by the sinusoidal moving pattern of resonant scanner. (TPEF image of fluorescence beads, 30 mw @780 nm excitation, 100 µm × 100 µm Field of View. Each image includes the signals collected with forward and backward scans of the resonant scanner, which are two symmetric parts.)………………………………………………..32 Figure 2.10 Algorithm to correct image distortion. Blue curve represents the actual sinusoidal scanning pattern of the resonant scanner; Red curve represents the ideal linear scanning pattern. (θ(t): variable representing the actual scanning angle of X-axis scanner at a certain time point (t); A: a certain value of θ(t); L(t): variable representing the corrected angle of X-axis scanner for the ideal linear scan.) A pixel positioned at P is reassigned to position PC to correct the distortion. ..…………....………………………33 Figure 2.11 Experiments to verify the effectiveness of the linearization algorithm for correction of image distortion. (a) Before processing (b) After processing (TPEF image of fluorescence beads)……………………………………………………………...…….34 Figure 2.12 Labview program for automatic EEM acquisition (A) Program operation process (B) Program interface in Labview .……………………………………………..37 Figure 3.1 Optical setup of the power attenuator kit ….…….…………………………..40 Figure 3.2 Relation of excitation power versus rotation angle of ½ wave-plate. (Excitation power is measured at the exit of power attenuator. ½ wave-plate is rotated viii  from 0 to 180 o for each excitation wavelength. Four excitation wavelengths are investigated including 750 nm, 800 nm, 850 nm, 900 nm.) …………………………….40 Figure 3.3 Experiment to test laser power fluctuations. (Excitation power is measured after the objective.)………………………………............................................................41 Figure 3.4 Optical setup of autocorrelator for pulse width measurement………………..44 Figure 3.5 Pulse width measurements with the autocorrelator at different locations in the system. (Two types of PBS and two types of objectives have been investigated. Laser wavelength is tuned from 720 nm to 950 nm with an increment of 10 nm.)………….....45 Figure 3.6 TPEF images of bovine elastin powder taken under various excitation wavelengths. (A) Taken with conventional dielectric mirrors, 30 mW excitation power, 200 µm × 200 µm Field of View. (B) Taken with new dielectrically enhanced Ag mirrors, 20 mW excitation power, 200 µm × 200 µm Field of View. .......……………………….46 Figure 3.7 TPEF image intensity versus excitation wavelength for comparison of two optical setups with different types of reflectance mirrors...……………………………...47 Figure 3.8 Comparison of EEMs of pure keratin sample measured with two setups respectively (A) EEM measured with the old setup. (B) EEM measured with the new setup. ………………………………………………………………………………….....48 Figure 3.9 Excitation beam bandwidth (FWHM) measurement result. There are 3 groups with forward and backward tuning of laser in each group...…………………………….49 Figure 3.10 PSF of a single fluorescent bead (A) Image taken with excitation wavelength of 820 nm and FOV of 60 µm × 60 µm. (Each pixel is approximately related to the actual size of 0.1 µm × 0.1 µm on the sample.) (B) Intensity distribution of the center vertical line of the bead image (Background has been subtracted from the original data for Gaussian fitting. For a standard Gaussian curve: = 2√2 2 × ), where σ is a coefficient depending on the shape of the curve ..............................................................51 Figure 3.11 Resolution measurement result for excitation wavelengths from 730 nm to 920 nm. Experimental and theoretical result of two-photon resolution and the linear fitting to the experimental result are shown in the picture for comparison. Theoretical curve of one-photon resolution is also shown here. ………………………………….....52 Figure 3.12 Spectrum of Hg(Ar) lamp. The x-axis represents the pixel location on the spectrometer CCD array.....……………………………………………………………...53 Figure 3.13 Relation between wavelength and CCD pixel number of the spectrometer. Four curves are drawn including: theoretical fitting of lamp peaks, measured lamp peaks, quadratic fitting of lamp peaks and polynomial fitting of fs laser peaks.…………….....55  ix  Figure 3.14 Setup for intensity calibration for the emission path.……………………....56 Figure 3.15 Intensity calibration of emission beam path. (A) Specified spectrum of the lamp (blue) and measured spectrum of the lamp (red); (B) Measured spectrum of the calibration ratio (r = Iblue/Ired) with fully opened integration sphere entrance aperture and 4 seconds integration time of the spectrometer; (C) Measured spectrum of the calibration ratio (r = Iblue/Ired) with partly closed integration sphere entrance aperture and 1 second integration time of the spectrometer; (D) Stitched spectrum of the intensity calibration ratio..........…....................................................................................................57 Figure 4.1 Cross section of normal human scalp skin tissue section, dermis layer (thickness - 20 µm, FOV - 150 µm × 150 µm, excitation wavelength-785 nm, excitation power - 30 mw, 30 image frames were averaged for TPEF and SHG channels). In the combined false color image, we used red for confocal channel, blue for SHG channel, and green for TPEF channel.…………………………………………….........................59 Figure 4.2 Cross section of normal human scalp skin tissue section, dermis layer (thickness-20µm, FOV-150µm x 150µm, excitation wavelength-785nm, excitation power-30 mw, 30 image frames were averaged for TPEF and SHG channels). In the combined false color image, we used red for confocal channel, blue for SHG channel, and green for TPEF channel.…………………………………………….........................60 Figure 4.3 Pure Human Elastin (A) Two-photon EEM (excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 2 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Single-photon EEM (excitation wavelength: 250 nm – 600 nm)…………………………………………….....................62 Figure 4.4 Pure Human keratin (1mg/ml solution in urea) (A) Two-photon EEM (Excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 2 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Single-photon EEM (excitation wavelength: 250 nm – 600 nm)……........................63 Figure 4.5 Purified human collagen (powder) (A) Two-photon EEM (excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 3 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Singlephoton EEM (excitation wavelength: 250 nm – 600 nm) ......…………………………..65 Figure 4.6 SHG excitation spectrum of purified human collagen fiber (Type I) (Excitation wavelength: 730 nm – 920 nm, 20 mw excitation power, FOV: 100 µm × 100 µm, Exposure time: 3 s)……………………………...............................................................66 Figure 4.7 SHG excitation spectra (A) Fish scale sample (B) Rat tail tendon sample. (Excitation wavelength: 730 nm – 920 nm, Excitation power: 20 mw, FOV: 100 µm × 100 µm, Exposure time: 1 s)…………………..................................................................67  x  Figure 4.8 Nonlinear EEM of more fluorophores (Excitation wavelength: 730 nm–920 nm, Excitation power: 20 mw, Exposure time: 3 s. Both of 1-photon and 2-photon EEM for NADH are measured.)………………………………..................................................69 Figure 4.9 Image-guided EEM of normal human temple skin cross section. (Excitation wavelength: 730 nm – 920 nm, excitation power: 30 mw, FOV: 100µm × 100µm, Exposure time: 3 s) (A) Pseudocolor image of a combination of three channels including confocal (red), TPEF (green) and SHG (blue) (Excitation Wavelength: 790 nm, Averaged image of 50 frames) (B) Histology image related to the image in A (H&E staining) (C) Nonlinear EEM (white arrow shows the SHG signal)…………………………………...71 Figure 4.10 Image-guided nonlinear EEM of normal human temple skin. Left column shows the pseudocolor image (TPEF: green, SHG: red). Right column shows the EEMs related to the imaging area in the same row. Each row represents a different imaging depth (10 µm, 20 µm, 30 µm, 40 µm, 60 µm) (FOV: 100 µm × 100 µm; Scale bar: 20 µm; Excitation Power: 30 mW; Excitation Wavelength: 790 nm, Exposure time: 2 s, 50 frames averaged.) ………....…………………………………………………………….74 Figure 4.11 Cross section images of human skin tissue with Seborrheic Keratosis. (A) Combined image of TPEF and SHG channels by pseudocolor (TPEF: Green, SHG: Red). (B) Histology image with H&E staining. (Field of view: 200 µm × 200 µm; Blue square: 20 µm × 20 µm; Excitation wavelength: 790 nm; Excitation power: 30 mW)……..…...75 Figure 4.12 EEM for human skin tissue with Seborrheic Keratosis (a)-(e) is related individually to the blue square areas in Fig. 4.11. (Excitation wavelength: 730 nm – 920 nm; Excitation power: 30 mW, Exposure time: 2 s)…………..………………………...76 Figure A.1 MiniSAX II driver board electronic connections ...........................................86 Figure A.2 Analog output channel setup.………………………………………………..88 Figure A.3 Analog output waveform initialization..……………………………………..89 Figure A.4 Writing loop for analog output and Stop of writing ………………………...90 Figure A.5 Digital Output...……………………………………………………………...91 Figure A.6 Imaging System Initialization …………………………………………….....92 Figure A.7 Spectrometer Initialization ………………………………………………….93 Figure A.8 Loop of Spectrum Acquisition (Frame 1) …………………………………..94 Figure A.9 Loop of Spectrum Acquisition (Frame 2) …………………………………..95 xi  Figure A.10 Loop of Spectrum Acquisition (Frame 3) …………………………………96 Figure A.11 Stop Each Module and Write Average Emission Spectra to .TXT Files ….97 Figure A.12 Attenuator electronic design ……………………………………………...104  xii  LIST OF ABBREVIATIONS ADC – Analog to Digital Converter BCC – Basal Cell Carcinoma CLSM – Confocal Laser Scanning Microscopy DAQ – Data Acquisition EEM – Excitation Emission Matrix FCM – Fluorescence Confocal Microscopy FS – Femto-second FWHM – Full Width Half Maximum H.Sync – Horizontal Synchronization Pulse MFC – Microsoft Foundation of Classes NA – Numerical Aperture NIR – Near Infrared OCT – Optical Coherence Tomography OPA – Operational Amplifier PBS – Polarization Beam Splitter PMT – Photomultiplier Tube PSF – Point Spread Function PSL – Pigmented Skin Lesions RCM – Reflectance Confocal Microscopy SCC – Squamous Cell Carcinoma SHG – Second harmonic generation SK – Seborrheic Keratosis TPEF – Two-photon excited fluorescence V.Sync – Vertical Synchronization Pulse xiii  ACKNOWLEDGEMENTS I owe special thanks to Dr. S. Tang and Dr. H. Zeng, who offered me valuable guidance and support for me to continue working in this field. I thank Dr. H. Lui for providing clinical support to the project. I thank Dr. A. Lee and Dr. J. Zhao for sharing their rich research experience with me. I also thank the faculty, staff and my fellow students at UBC and BC Cancer Research Center for their assistance in different aspects. I owe particular thanks to my parents who provided me with both moral inspiration and financial support.  xiv  DEDICATION  To my parents  xv  Chapter 1 Introduction Basic knowledge of skin anatomy and related biological terms are introduced first in this chapter. Second, the imaging techniques utilized to guide EEM acquisition are introduced, including reflectance confocal, two-photon excited fluorescence (TPEF), and second harmonic generation (SHG). Third, the spectral and EEM applications on skin are discussed based on the theory and available results published by other groups. A project outline is included at the end of this chapter.  1.1.  Basics of Skin Anatomy and Skin cancer  1.1.1. Skin Anatomy Skin plays an important role in protecting human body from damage and water loss. It has a multi-layered structure as shown in Fig. 1.1. It comprises two compartments, epidermis and dermis. The top layer of skin is the epidermis. It is a thin layer, averaging around 0.1 mm thick. There are five sub-layers in the epidermis layer including stratum corneum, stratum lucidum, stratum granulosum, stratum spinosum, and stratum basale. Stratum corneum is the outermost layer of epidermis and it is mainly composed of dead cells. These dead cells are continuously sloughed off and replaced by new cells. Cells in stratum corneum layer mainly contain keratin protein to keep the skin moisturized. Keratin has strong fluorescence signal. Beneath the corneum sub-layer, there are stratum lucidum, granulosum and spinosum sub-layers containing live cells, which originate from stratum basale, keep migrating up and eventually become dead cells in stratum corneum. These cells mainly contain Nicotinamide adenine dinucleotide – reduced form (NADH) and Flavin adenine dinucleotide (FAD), which are coenzymes involved in several important redox reactions in metabolism. These two compositions also have strong fluorescence emissions. Cells in the stratum basale sub-layer keep dividing to generate new keratinocytes. Stratum basale also contains melanocytes, which produce the pigment melanin. Melanin can protect underlying tissues from UV damage by dissipating most of the absorbed UV radiation as heat. It is another typical fluorophore which can be used for fluorescence imaging and spectral analysis of skin tissues.  1  In the dermis layer of skin tissue, there are mainly extracellular matrix structures consisting of collagen, elastin and reticular fibers. These protein fibers provide dermis layer with the characteristics of high strength, elasticity and extensibility so that it works to cushion the body from strain and stress. Elastin fiber can have strong fluorescence, while collagen fiber can have strong emission of second harmonic generation signal, which is a nonlinear optical process and will be introduced later in this chapter.  Figure 1.1 Anatomical structure of skin.  2  1.1.2. Skin Cancer There are mainly three types of skin cancers including Basal Cell Carcinoma (BCC), Squamous Cell Carcinoma (SCC) and Melanoma, which are named based on the type of cells where these cancers originates respectively. Melanoma is a kind of malignant tumour of melanocytes in the stratum basale sub-layer of epidermis. BCC and SCC are more common but cause a less death rate than melanoma. BCC also develops in the stratum basale sub-layer of epidermis while SCC develops from squamous cells which are contained in stratum corneum, lucidum and granulosum sub-layers of epidermis. They usually happen in sun-exposed areas over human body. If these three types of carcinomas are diagnosed at early stage, they can be treated by surgical excision. The procedure involves surgically removing the tumour and a certain amount of normalappearing skin surrounding it. Cure rates following excision can be up to 99% for both BCC and SCC for low-risk lesions, and up to 98% for melanoma [1 – 3]. Therefore early stage diagnosis plays a significant role in skin cancer treatment. Histology examination is the gold standard for skin cancer diagnosis. However, complicated procedures involved in the examination leads to a high cost to the health care system and long waiting time. Hence, non-invasive diagnostic techniques are developed as screening procedures over skin lesions to extract the suspect cases for further examinations. Optical techniques are widely used to realize the non-invasive diagnosis of skin diseases, which will be introduced in the next section.  3  1.2.  Optical Imaging Techniques for Diagnosis  There are various optical diagnostic techniques. Dermoscopy is well developed for clinical application and only needs simple device and implementation procedure, which is good for screening of Pigmented Skin Lesions (PSL). However, it cannot differentiate signals emitted from different depths of the tissue sample, which will influence the accuracy of diagnosis. Optical Coherence Tomography (OCT) has been investigated for possibility of skin cancer diagnosis and there are already commercial products available. OCT has a large penetration depth for skin tissue imaging. However, the resolution of OCT image is comparatively low (usually in the range of 3-15 µm), which will not suffice cellular structure visualization [4]. In view of the drawbacks of these techniques, some advanced techniques are developed to increase imaging performance such as the following three to be introduced. 1.2.1. Confocal Laser Scanning Microscopy (CLSM) Figure 1.2 shows a basic confocal microscope setup. A laser beam becomes a point light source after a light source pinhole. The point source light is reflected by a dichromatic mirror and focused by objective onto a specific plane in the specimen to form point illumination as a voxel. Emitted light passes through the dichromatic mirror and reaches a detector pinhole aperture. Because the light source pinhole, the focal voxel and the detector pinhole are lying on the optically conjugate focal planes, only light emitted from the focal voxel can pass through the detector pinhole and be collected by the detector as image signal. Light emitted from out-of-focus plane are all rejected by the detector pinhole [5]. By scanning the laser beam in x-y directions, 2-D images can be acquired. The elimination of out-of-focus light leads to a high resolution and optical sectioning capability for non-invasive imaging without physical tissue sectioning. Confocal can have a resolution of around 0.5-1.0 µm laterally and 3-5 µm axially, which makes it possible for in vivo imaging of cellular structures down to the upper dermis layer [6]. There have already been some results reported for applying confocal microscopy on skin cancer diagnosis [7 – 12]. Moreover, possibility has been investigated to utilize CLSM for microscopy-guided surgery for skin cancer [13, 14]. There are two modes for confocal microscopy, reflectance and fluorescence modes. Reflectance confocal 4  microscopy (RCM) applies the differences of refractive indices of tissue structures to produce image contrast while fluorescence confocal microscopy (FCM) utilizes the differences of fluorescence characteristics of exogenous or endogenous fluorophores to form image contrast [15]. Skin tissues contain several typical endogenous fluorophores such as flavins, porphyrins and collagen fiber, which can be applied for FCM. However, fluorescence from such endogenous fluorophores reaches the maximum with excitation in the UV range. UV light will encounter severe scattering when entering thick tissue, which is not suitable for deep tissue imaging. Moreover, UV light is carcinogenic, which is not safe for skin imaging. Therefore, exogenous fluorescent dyes have to be used for FCM to provide sufficient signals [16]. Near infrared (NIR) laser sources are usually used for RCM, which provides a large penetration depth because the NIR light encounters much less scattering than UV light. However, RCM can only supply morphological information of tissue samples which is not related to any specific fluorophore or biochemical composition. In two-photon excited fluorescence (TPEF), skin endogenous fluorophores will have the excitation maxima in the near infrared (NIR) range due to the difference of imaging mechanisms which will be introduced in the next section. Thus TPEF can directly use the intrinsic fluorescence signal to analyze skin tissue composition and morphological characteristics.  5  Figure 1.2 Schematic picture of basic setup of confocal microscope. Illumination beams are shown in green color and emission beams are shown in red color [5].  6  1.2.2. Two-Photon Excitation Fluorescence (TPEF) TPEF is based on a nonlinear optical process, in which the fluorophore absorbs two photons simultaneously and emit one photon. The difference of single-photon and twophoton fluorescence mechanism is compared in Fig. 1.3. Because the energy of excitation photons has to match the energy gap between two electronic states of the fluorophore, in two-photon process excitation photon energy will be almost half of that in single-photon process. Thus two-photon process can use NIR light source for excitation, which offers it the ability to image much deeper than single-photon process [6, 17 – 19].  Figure 1.3 Schematic drawing illustrating the processes of fluorescence (A) single-photon fluorescence (B) two-photon fluorescence [6].  Furthermore, the two-photon process requires two photons interacting with fluorophore almost simultaneously (~10-16 s) and thus the emission intensity has a quadratic dependence on the excitation intensity. TPEF probabilities will be very small except for the focal point, where photons are focused by the objective lens.  This special  characteristic enables TPEF to intrinsically exclude the signal from regions other than the focal point, which is similar as the optical sectioning ability of confocal microscopy [20 – 23]. This means two-photon fluorescence can be used for in vivo diagnosis with large imaging depth (1 W average power allows imaging depths of about 600-800 µm in the neocortex and up to 1 mm with a regenerative amplifier) [24 – 27], high resolution (usually ~0.5 µm laterally, 1-2 µm axially, which is even better than CLSM) [6, 28, 29] and less photo-bleaching and photo-damage [30].  7  In addition to advantage in imaging performance, two-photon fluorescence can also provide information of chemical compositions of skin tissue which is important for imaging and spectral analysis. There are several important intrinsic fluorophores in human skin including: NADH, FAD, collagen, elastin, keratin, melanin and flavins [31, 32]. Example applications of two-photon microscopy on human skin studies include quantifying skin aging [33]; distinguishing normal, precancerous and cancerous epithelial tissues [34]; in vivo diagnosis of nodular BCC [35], etc [36 – 38]. When two-photon imaging is utilized to guide two-photon fluorescence spectral analysis, both of the structural and spectral information of endogenous fluorophores can be obtained simultaneously so that we can better differentiate and characterize these fluorophores for skin tissue analysis.  8  1.2.3. Second Harmonic Generation (SHG) SHG is also produced by a nonlinear optical process. However, it has different characteristics from TPEF. SHG emission occurs at the wavelength exactly half of the excitation wavelength while TPEF emission maximum happens at a wavelength longer than half of excitation wavelength. Therefore, we can differentiate SHG spectrum from TPEF spectrum. SHG will only happen in materials with non-centrosymmetric structures [39]. Such structure is available in collagen fiber, which is a very important constituent of connective tissues in the dermis compartment of skin. Compared to confocal and TPEF, SHG has exclusive contrast from collagen fiber which makes it promising for study of collagen related skin diseases. There have already been some studies of using SHG imaging to analyze ex vivo skin lesions [40, 41]. Moreover, a method of quantitative analysis of the ratio between SHG and TPEF signals has been introduced to characterize the alterations of dermal content related to tumour invasion [42]. The above optical imaging techniques are based on different contrast mechanisms and a combination of these three will supply morphological and composition information of skin tissues. However, some tissue constituents will have similar structures and it will be difficult to differentiate one from another by using only imaging. Different fluorophores have specific spectral signatures which can be used to analyze complex tissue compositions. Therefore an integration of imaging and spectral detections will provide us more comprehensive information to assist diagnosis. Moreover, using real-time imaging to guide spectra acquisition will help to obtain spectral information from regions of specific interest on tissue samples.  9  1.3.  Excitation-Emission-Matrix for Diagnosis  1.3.1. Spectral Diagnosis Spectroscopy can be applied to extract chemical composition information of skin tissue to analyze physiological and biochemical states of tissues. Leveraging the large variety of fluorophores in skin tissues including keratin, collagen, elastin, NADH and FAD, spectroscopic study has a great potential in studying skin diseases including aging [43], psoriasis [44] and skin cancer [45]. Single-photon excited fluorescence spectroscopy has been investigated on skin and other epithelial tissues [46 – 48]. A number of endogenous skin fluorophores have also been identified using single-photon spectroscopy [49]. Certain characteristic differences between normal and diseased tissues can be found from those results. Compared with single-photon spectroscopy, two-photon spectroscopy has irreplaceable advantages, including the intrinsic optical sectioning ability, large penetration depth and addition of SHG signal. Two-photon spectroscopy can be potentially used to provide important information for skin disease diagnosis such as metabolism and physiological states of the cellular epidermis layer, and distribution of collagen fiber in the dermis layer. Numerous studies have been done on applying two-photon spectroscopy on skin tissue analysis [50 – 53]. As shown in Fig. 1.4, skin tissues from 3 sub-layers in the epidermis compartment are investigated with two-photon images and spectra at 3 excitation wavelengths [54]. The author claimed that all of the three sub-layers (a, b, c) have peaks at 475 nm and 550 nm. The 475 nm peak shows agreement with NAD(P)H emission spectra measured by other groups. The 550 nm peak originates from two-photon fluorescence of melanin in the skin tissue and the peak intensity keeps increasing while it goes deeper into the tissue until the stratum basal layer. This result shows the potential of using two-photon spectroscopy to study the biochemical characteristics of ex vivo skin tissues with the layer-resolving capability. Furthermore, the morphological information provided by two-photon images can assist in identifying the endogenous fluorophores in the skin tissue [54].  10  Figure 1.4 Two-photon images of various skin layers obtained after excitation at 760 nm and observed through different emission filters. The emission spectra of each layer for different excitation wavelengths are also shown (a) stratum corneum, (b) stratum spinosum and (c) stratum basal layer. Each layer is excited at three different excitation wavelengths: 730 (line), 780 (line with triangle node) and 830 (line with square node) [54].  In view of the published results, spectroscopy has significant potential to provide biochemical and physiological information for comprehensive and accurate diagnosis, which cannot be realized by imaging methods. Hence, a combination of image and spectra will generate both morphological and biochemical information of tissues, and further improve the performance in diagnosis. Moreover, two-photon spectroscopy has significant advantages over single-photon spectroscopy as discussed above. Based on these conclusions, our project will use multiple imaging channels to guide the acquisition of two-photon excited spectral signal (TPEF & SHG). In addition to applications of 11  emission spectrum at certain excitation wavelength, applications of excitation spectrum with fixed emission wavelength have also been investigated in research on skin tissues [55 – 57]. Together with other considerations which will be discussed in the following section, it leads us to the point of using imaging guided nonlinear Excitation-EmissionMatrix (EEM) to provide comprehensive information of skin tissues to assist diagnosis.  12  1.3.2. Excitation-Emission-Matrix (EEM) Excitation-Emission-Matrix (EEM) is a two-dimensional matrix whose element, I (λex, λem), is the fluorescence intensity as a function of the excitation and emission wavelengths. Normally it is acquired by scanning the excitation source over a range of wavelengths to collect data of emission spectrum for each excitation wavelength, and then plot the data in a 2-D or 3-D contour map. For ease of operation, fluorescence excitation/emission spectra are usually acquired at one or limited number of emission/excitation wavelengths in various analytical and diagnostic applications as introduced in the last section. EEM is a comprehensive set including all of these spectra. Thus firstly, EEM can assist in choosing the optimal excitation/emission wavelength for the best performance of spectral analysis of specific tissue samples. Secondly, EEM of purified fluorophore can serve as a unique signature. Based on these signatures, EEM of complex tissues can be used to identify the fluorophores and their relative distributions in the tissue in a reliable manner. Thirdly, the variations of emission from one intrinsic fluorophore with the change of excitation wavelengths can be used to analyze the biochemical state of cells and tissues. EEM can display these variations within a single contour plot and thus facilitate fast and reliable diagnosis. There have already been some studies applying single-photon EEM on tissue analysis [58 – 62]. A group has collected single-photon EEM data from pure endogenous fluorophores of skin tissue including: collagen (type I, type IV-type VII), elastin, FAD, Keratin, NADP, Tryptophan, PpIX and so on. This collection can work as a database for decomposition and analysis of complex tissue EEMs [63].  13  Figure 1.5 Tissue collagen and NAD(P)H intrinsic fluorescence EEMs are extracted from variceal asphyxiation measurements. a and b, extracted intrinsic fluorescence EEMs of two components that changes observed during tissue asphyxiation (suffocation). Different colors represent different fluorescence intensities, as indicated by the adjacent color bar scales. c, intrinsic fluorescence EEM of collagen I in powder form. d, NAD(P)H fluorescence from isolated epithelium of freshly excised cervical tissue [64].  Another group has applied single-photon EEM on analysis of precancerous changes of epithelial tissues [64]. They first extracted EEM signatures of NAD(P)H and collagen from EEM of normal bulk tissue. Second, they decomposed EEMs of a group of various tissues into linear combinations of NAD(P)H and collagen EEM signatures. Finally, they used the relative distribution of NAD(P)H and collagen to characterize normal and dysplastic epithelial tissues. Fig. 1.5 shows the extracted EEMs of NAD(P)H and collagen. For comparison, it also shows EEM results of purified collagen powder and isolated epithelium of freshly excised cervical tissue [64]. Another group has done research on utilizing single-photon EEM for early detection of the neoplastic changes of mouse skin tumour [65]. Fig. 1.6 is a typical EEM from the mouse skin exhibiting the fluorescence characteristics of tryptophan, collagen, NADH and porphyrin. Compared with single emission/excitation spectra, the EEM includes much more comprehensive information of these fluorophores and a relative distribution of fluorescence originating from each of them. The authors used special treatments to induce tumour promotion on the mouse skin and measured EEMs of the mouse skin while the tumour developed into various stages. Their results suggest that tryptophan and 14  endogenous porphyrin can be used as tumour markers for early detection of neoplastic changes [65].  Figure 1.6 The raw EEM of mouse skin with the fluorescence emission characteristics of 1tryptophan; 2-collagen; 3-NADH; 4-FAD, and 5-porphyrin [65].  The published results mentioned above are all on single-photon fluorescence EEM which has no depth resolution. When conventional single-photon EEM is applied to bulk tissue, the result will be a mixture of spectra emitted from different tissue depths, which is not practical for analysis of tissue compositions. In two-photon EEM study, using the optical sectioning capability of TPEF and SHG, EEM can be acquired from a specific thin layer and even a specific small region-of-interest. Spectral analysis can then be accurately targeted to particular locations in skin tissues with minimal interference from the surrounding tissues. Furthermore, SHG is a new contrast in two-photon EEM, which can provide extra information of collagen fibers in skin dermis. Moreover, as introduced in the last section, two-photon emissions (TPEF and SHG) in the UV-Visible range are excited by a laser beam in the NIR range. Hence the excitation beam will incur less absorption and scattering during transmission in tissue samples and have larger penetration depths than single-photon fluorescence.  15  There have been numerous studies of applying nonlinear (TPEF & SHG) emission/excitation spectra for skin tissue analysis. However, there are few investigations of nonlinear (TPEF & SHG) EEM applications. One study was focused on nonlinear EEM of purified collagen and elastin, and ex vivo human skin dermis sample [66]. Fig. 1.7 shows the three-dimensional EEM plots for these samples. Comparing (b) and (c), it was found that EEM of the excised human skin dermis have a different pattern with pure collagen sample, while it is similar as that of pure elastin sample. This may indicate that the TPEF signals of excised dermis tissue mainly originate from elastin fiber. This group has done imaging experiments to confirm this conclusion [66]. We can see great potential of applying nonlinear EEM in skin tissue characterization, especially when it is combined with image analysis. Nevertheless, as in the above study, corresponding images were taken after completing all of the EEM experiments. It is not guaranteed that the same region on tissue sample is investigated for imaging and spectral experiments, which may cause errors in analysis of complex tissues. Therefore, we developed a system with the function of simultaneous video-rate imaging acquisition and EEM measurement. It will first facilitate us in searching interesting regions on the tissue for EEM research and then guarantee that spectral and imaging signals are acquired from exactly the same region on the tissue sample. This system can provide us with a comprehensive set of information with both morphological and biochemical characteristics of skin tissues. To the best of our knowledge, no published result has done nonlinear EEM experiment for specific sub-layers in the epidermis compartment, which plays a significant role in skin diseases diagnosis. Taking advantage of the optical sectioning ability of TPEF and SHG, we have done layer-resolved characterizations of both normal and diseased human skin tissues using the multiple-channel-imaging guided nonlinear EEM.  16  Figure 1.7 Three-dimensional excitation-emission-matrices plot of (a) a purified collagen sample, (b) a purified elastin sample, and (c) an excised human skin dermis. (Laser was tuned from 750 nm to 950 nm in increments of 20 nm.) [66]  17  1.4.  Personal Contribution  A multimodality system was first built with three imaging channels (Confocal, TPEF and SHG), and a spectroscopy channel (nonlinear EEM) by Anthony Lee (Postdoctoral Fellow), myself, and Tracy Wang (PhD Candidate) at the BC Cancer Research Centre. This multimodality system is developed for various applications and it includes several sub-projects. Imaging-guided nonlinear EEM is one of them. My contribution to the multimodality system and the EEM sub-project includes the following: 1. System Development (1) Designed the electronic module for computer control and electrical communications within the system and developed software programs to facilitate operations. (2) Calibrated major optical parameters for accurate EEM measurements. 2. Experiments and Analysis (1) Completed EEM measurements on purified fluorophores and analyzed the exclusive EEM characteristics of individual fluorophores, which were used for tissue analysis. (2) Completed image-guided EEM measurements on normal and diseased human skin tissue samples and interpreted the complex tissue EEMs based on the results of pure fluorophores. (3) Compared EEMs of normal and diseased skin tissue and demonstrated the great potential of using this image-guided EEM technique for skin disease diagnosis.  18  Chapter 2 System Development The large versatile multimodality system is designed for various clinical applications including in-vivo clinical imaging diagnosis and the image-guided nonlinear EEM. I have participated in part of the multimodality system development and completed the EEM sub-project all on my own. This chapter will introduce the whole multimodality system setup and more specifically the EEM sub-system setup. Several units will be explained in detail including scanner unit, image acquisition unit and EEM acquisition unit, which are the most important three units for the image-guided nonlinear EEM sub-system. A list of specifications of key components will be included at the end.  2.1.  System Optical Setup  2.1.1. Multimodality Microscopy and Spectroscopy System Figure 2.1 shows the optical layout of the multimodality system. A femto-second (fs) laser (Chameleon Series, Coherent) is used as the excitation source. The maximum output power of the laser is specified as 1 W. A computer controlled attenuator, including a ½ wave-plate (10RP52-2, Newport) and a polarization beam splitter (PBS 052 AR600-1000 nm, Thorlabs), is installed directly at the beam exit of the laser. The power is reduced to be less than ~200 mW by the attenuator in order to get a power level of around 30 mW at the sample after taking into account of the transmission and reflection loss of other optical components in the path. As shown in Fig. 2.1, excitation beam originates from the laser and transmits through the attenuator module, gets expanded, reflected by the scanner and focused by the objective (LUMPLFLN60XW, Olympus) onto tissue sample. The emitted beam, including reflection, SHG and fluorescence, originates from the sample and passes through the objective and divides into two ways. The reflection beam transmits through the dichroic mirror (FF665 Di02 25 × 36, Semrock), gets descanned by the scanner unit and collected by the photomultiplier tube (PMT, R3896, Hamamatsu) for reflectance confocal imaging. The multi-photon signal is reflected by the dichroic mirror, and is separated into another two beams with one collected by the spectrometer (SpectraPro-150, Roper Scientific) and the other transmitted to the SHG/TPEF Dichroic Mirror (FF409 Di03 25 × 36, Semrock). Finally the beam, which reaches the SHG/TPEF  19  dichroic mirror, divides into another two beams and is collected by two PMTs (H9433MOD, Hamamatsu) for Second Harmonic Generation (SHG) imaging and TwoPhoton Excitation Fluorescence (TPEF) imaging. 2-D scanning of the excitation beam is realized by the X-Y scanner set. Data can be collected from different depth of sample tissues by adjusting the micrometer to move the sample stage up and down. This elaborate design realizes high performance for simultaneous three-channel image acquisition including the TPEF, SHG and Confocal. Anthony Lee and Tracy Wang have been responsible for building this optical setup for the multimodality system.  Figure 2.1 Optics layout of the multimodality microscopy and spectroscopy system. Beam paths for four channels are shown including confocal imaging, TPEF imaging, SHG imaging and spectroscopy. (In the actual 3-D optics setup of the system, several mirrors are installed to direct the light beam, which are not drawn in this figure.)  20  2.1.2. EEM Sub-System Fig. 2.2 shows the optical layout of the EEM sub-system, which is extracted from the main multimodality setup. The excitation beam path for EEM is the same as the imaging channels. Because the laser power and the transmission efficiency of optical components will change due to the variation of excitation wavelengths, we need to control the excitation power applied on the sample in EEM measurements so that it will be constant for all excitation wavelengths. The attenuator kit plays an important role in realizing the excitation power calibration. It comprises a ½ wave-plate (λ/2 in Fig.2.2) and a polarization beam splitter (PBS). Rotation of the ½ wave-plate will change the polarization of laser beam. The PBS will attenuate laser beam power at a variable ratio according to the beam polarization. Therefore we can adjust excitation power applied on the tissue sample by controlling the rotation angle of the ½ wave-plate. Automated adjustment is realized by using a computer-controlled motor (PRM1-Z7, Thorlabs) to rotate the ½ wave-plate. Details about the attenuator kit and excitation power calibration will be discussed in Chapter 3. In Fig. 2.2, the emitted light is collimated by the objective and part of it is reflected by the 50/50 beam splitter. The reflected light is coupled into a customized fibre bundle (Fiberguide Industries) by a fibre coupler lens, transmitted through the fibre bundle and finally collected by the spectrometer (SpectraPro-150, Roper Scientific). The fibre bundle is applied here to increase the collection area of emitted photons for a high signal-tonoise ratio in the spectral acquisition. The fibre bundle has two different patterns at the input and output sides respectively as shown in the zoom-in pictures in Fig. 2.2. Grey circles denote that the outside fibers at the input side of the fiber bundle are mapped to the ends of the line pattern of the fiber bundle at the output side. The fiber bundle has 90 small fibres (single fiber diameter: 100 µm; numerical aperture (NA): 0.12) arranged in a hexagon pattern at the input end to provide a much larger collection area than a single fibre. The output end has all of the 90 fibres arranged as 2 straight lines with 45 fibres in each line so that most of the collected light can be coupled into the narrow entrance slit of the spectrometer. The width of this line-shape bundle of fibers is 200 µm. The f-number (f/#) of the spectrograph system (f/4) has been matched with the numerical aperture (NA) of the fiber (0.12) as: f/# = 1 / (2 × NA) for optimum performance. 21  Figure 2.2 Optics layout of the EEM sub-system. Cross sections of two ends of the fiber bundle are shown in the zoomed in insertions. (Grey circles denote that outside fibers at the input side of fiber bundle are mapped to ends of line at the output side.)  Careful alignment of the two ends of the fibre bundle has been completed for optimized spectra detection. In addition to optics design, the multimodality system and the EEM sub-system both need computer control and electrical signal synchronization for automated operation. Hence electronics hardware and computer control software has an irreplaceable significant role in the system development, which will be described in the next section.  22  2.2.  Signal Synchronization for Scanners and Detection Unit  There are two parts in the system that need electronic control: scanner unit and detection unit. Scanners are used to scan the excitation beam over a certain area on the sample. Detection unit includes PMT and frame grabber for image acquisition, and spectrometer for EEM measurement. We need to coordinate image acquisition with movement of scanners for accurate image display. Therefore, synchronization signals need to be generated for proper operation of the system. There are various designs for XY scanning in two-photon microscope systems [67]. Most commonly used one is the non-resonant galvanometer scanner, which can be positioned at any specific angle in the scanning range but can only scan slowly. An alternative is the resonant scanner which operates in a self-oscillating mode at fast speed [68, 69]. The resonant scanner has to be in constant motion and it can’t stop at a specific angle. It will move back to its default position once it stops scanning. Our system is designed for clinical applications and imaging speed is important for in vivo diagnosis. Hence the resonant scanner is chosen for the fast axis of this system. The X-axis scanning is implemented with resonance scanner (CRS Series, General ScanningTM) and the Y-axis scanning is realized by non-resonant galvanometer scanner (VM Series, General ScanningTM). Both scanners are controlled by computer. The resonant scanner has a fixed scanning frequency of ~8 kHz, which can realize a frame rate of ~12 FPS for the frame size of 512 × 512 pixels. This scanning rate is suitable for real-time in vivo diagnosis. A data acquisition (DAQ) board (USB 6211, National Instrument) is used to generate the synchronization signals to coordinate the scanner and the detection units. The resonant scanner is self-oscillating and only needs a DC signal to set the scanning amplitude. The galvanometer scanner needs a voltage signal in a saw-tooth pattern to drive each step of its scanning. DAQ board generates these two scanner-movement-control signals as analog outputs. The resonant scanner outputs the horizontal synchronization pulse (H.Sync.), which has a rising edge at the beginning of each line scan. When the DAQ board receives this pulse, it generates the voltage output to move Y-axis scanner to the next line. Upon finishing of scanning for one frame, the DAQ board will generate the 23  vertical synchronization pulse (V.Sync.) to inform Y-axis scanner to move to the beginning line for another frame. In our experiments, the frame size is set as 512 x 512 pixels. As in Fig. 2.3, for each frame there are 512 H.Sync pulses representing 512 lines and 1 V.Sync pulse representing 1 frame. There is also a fly back duration for the galvanometer scanner (Y-axis) to safely move back to the starting line of a frame. All the signal edges, which overlap with the same vertical dotted line, are synchronized together.  Figure 2.3 Principle of signal synchronization for scanners and detection unit. Vertical dotted line shows the synchronized edges.  Fig. 2.4 shows the physical connections between the scanner and the detection unit. DAQ board is the core component. It first receives H.Sync from the resonant scanner driving board. Then it generates V.Sync based on the H.Sync and transfers both of the synchronization signals to the frame grabber for display of video signals detected by PMTs. The DAQ board also transmits analog outputs to the scanner driving boards to control scanner movement. The two scanner driving boards also need a +/- 15v power supply to work properly. (Details about the two driving boards are supplied in the Appendix.) To make the system more compact, we have the scanner driving boards and the power supply integrated into a small electronic box.  Thanks Anthony Lee for  designing and machining this electronic box.  24  Figure 2.4 Physical connections of the electrical components for signal synchronization.(PMT: Photomultiplier Tube; Sync: Synchronization Signal; TTL: Transistor-Transistor Logic; LPF: Line Per Frame)  A Labview program is developed to control the DAQ board for signal reception and generation. (Program code is attached in the Appendix 2) Fig. 2.5 shows the operation process and user-interface of the program. As shown in Fig. 2.5 A, the DAQ board receives H.Sync from the resonant scanner driving board and applies it as the sampling clock for generation of other digital and analog signals. The V.Sync is generated by frequency division using H.Sync as the base frequency. To generate the analog outputs, two waveform samples are first written into the buffers. Then the two buffers are triggered by H.Sync and V.Sync respectively to start outputting the analog waveforms to drive the two scanners (DC waveform for resonant scanner and saw-tooth waveform for galvanometer scanner). Then they will continuously output the analog signals until the STOP button is clicked. Fig. 2.5 B is the interface of the Labview program. The number of scanning lines for one frame can be set at the beginning. The amplitude of two scanning waveforms can be adjusted in real time using either input of an exact number or smooth sliding bar control. The analog output waveforms are displayed in two oscilloscope style windows. 25  A  B  Figure 2.5 Labview program for signal synchronization (A) Program operation process (B) Labview Program Interface (Program code is shown in the Appendix 2.)  26  2.3.  Image Acquisition and Processing  As introduced in Chapter 1, using TPEF and SHG images to guide the spectral acquisition will provide morphological information of endogenous fluorophores and assist in analyzing chemical composition and physiological state of the tissue. Therefore, imaging acquisition is an important part of this project. 2.3.1. Attenuator and Multiplexer Module for Optimization of PMT Video Output The detection unit of imaging channel includes PMT and frame grabber (Alta AN, BitFlow). PMT will generate electrons based on the incoming photons and collect these electrons as an output signal. As a photosensitive device, it provides extremely high sensitivity and fast response. However, there are some problems coordinating the operations of PMT and frame grabber. Firstly, we need to increase the gain of PMT for a high sensitivity of photon detection. However, when we reach the ideal level of PMT gain, the voltage of output signal will be around 3 V, which is more than twice of the maximum input voltage (1.4 V) of the frame grabber. Therefore, we need to attenuate the PMT output signal into an acceptable range for the frame grabber. Secondly, the 8-bit ADC (Analog to Digital Converter) module of the frame grabber will digitize incoming signal into 256 levels based on two reference levels: white (high) and black (low). A clamp circuit will assume the blanking level embedded in the video signal as zero, and set the high and low levels of ADC accordingly. In standard video signal, there is an H.Sync pulse and a blanking level embedded inherently for clamping. Clamp circuit will choose the voltage level right after the H.Sync as blanking level. (Fig. 2.6 A) However PMT output is continuous signal without any blanking level set. Thus the clamping circuit will clamp to an arbitrary voltage level on the PMT signal according to its relative position with the external H.Sync. In the case that the level chosen by the clamp circuit is relatively high, all the pixels in that line which are weaker than that level will be digitized as zero and displayed as black. (Fig. 2.6 B) Lots of information will be lost due to this false setting of reference level.  27  Figure 2.6 Principle for setting blanking level on the video signal with the ADC clamp circuit of frame grabber. (A) Standard RS-170A video signal (B) PMT video signal  To solve the above problems, we contracted a company to make a custom attenuator box based on our design as shown in Fig. 2.7. The first unit functions to attenuate the PMT output signal with an operational amplifier (OPA). In the second unit, a multiplexer will choose either the PMT output or a ground signal as the input signal to the frame grabber. The external H.Sync (with a duty cycle « 50%) generated by DAQ board is input into the multiplexer as a digital reference signal. When the multiplexer detects a rising edge of H.Sync, it will choose to transmit the ground signal. Then the clamp circuit will assume this signal as blanking level signal and set it as zero for ADC. When the multiplexer detects a falling edge of H.Sync, it will transmit PMT video output to the frame grabber 28  for image formation. In this way the clamp circuit can work properly with correct setting of high and low reference levels of frame grabber ADC.  Figure 2.7 Block diagram of the attenuator and multiplexer box to adapt the frame grabber to non-standard video signal generated by PMT. (OPA: Operational Amplifier; MUX: Multiplexer; H.Sync: Horizontal Synchronization Signal.)  Experiments have been done to verify the effectiveness of this solution. As Fig. 2.8 shows, it is obviously different between the images acquired with and without the attenuator box installed. More signals have been captured after the attenuator and multiplexer box is implemented.  29  Figure 2.8 Effectiveness of the attenuator box. (A) Image acquired without the attenuator box (B) Image acquired with the attenuator box installed. (Bovine Collagen Sample, 30 mw @780 nm excitation, 100 µm × 100 µm Field of View, Each image includes the signals collected with forward and backward scans of the resonant scanner, which are two symmetric parts.)  30  2.3.2. Linearization Processing to Correct Image Distortion As mentioned above, the resonant scanner is self-oscillating in a sinusoidal way with varying speed. It scans faster in the middle of the scanning region and slower at the edges. However, the frame grabber will presume that it is scanning at a constant speed and then the displayed images will be stretched at two edges. Hence further processing is needed to correct the image distortion. Assuming focal length of the objective is D, the scanning angle of resonant scanner is θ and the X-axis displacement of excitation beam on the focal plane is S. Then we can calculate the displacement as: S = tan(θ) × D. According to specification of the resonant scanner, it has a typical scanning angle of ± 15 degrees. Then we can use the relation tan(θ) ≈ θ to simplify the above equation as: S ≈ θ × D. D is a constant and S will be linearly proportional to the scanning angle θ. Thus to simplify the theoretical calculation and processing procedure, we use the scanning angle θ of resonant scanner to represent the X-axis displacement of excitation beam on the focal plane. Fig. 2.9 shows the scanning angle of resonant scanner. One period includes forward and backward scans over the same scanning range. Thus the backward scan just generates a mirror image of the forward scan. As seen in the image of fluorescence beads, the beads at two edges of the scanning area are stretched along X-axis. A linearization algorithm has been developed to correct this distortion [70].  31  Figure 2.9 Resonant scanner working mechanism and sample image to show the distortion caused by the sinusoidal moving pattern of resonant scanner. (TPEF image of fluorescence beads, 30 mw @780 nm excitation, 100 µm × 100 µm Field of View. Each image includes the signals collected with forward and backward scans of the resonant scanner, which are two symmetric parts.)  The algorithm is to relocate each pixel to the position as linear scan so that it is not distorted from the true shape. We mainly focus on processing the forward scan part. First we need to calculate the scanning phase step size for each pixel as ∆φ = 2π / n, where n is the pixel number per line (e.g. n = 1024 for the image with 512 × 512 pixels because the forward scan and backward scan of X-axis scanner are symmetric) and 2π means the resonant scanner sweeps for one period per line. Then we can find the phase for each pixel location as φ = ∆φ × Pixel Number. The parameter pixel number is the pixel differences between the current location and the center pixel of the forward scan which is n/4. Then we can calculate the correction factor for relocation as C = φ / sin φ. [70]. As in Fig. 2.10, blue curve is the actual sinusoidal scanning pattern and red line is the ideal linear scanning pattern. Using the correction factor, pixels distributed along the actual pattern will be relocated onto the ideal linear pattern.  32  Figure 2.10 Algorithm to correct image distortion. Blue curve represents the actual sinusoidal scanning pattern of the resonant scanner; Red curve represents the ideal linear scanning pattern. (θ(t): variable representing the actual scanning angle of Xaxis scanner at a certain time point (t); A: a certain value of θ(t); L(t): variable representing the corrected angle of X-axis scanner for the ideal linear scan.) A pixel positioned at P is reassigned to position PC to correct the distortion.  In Fig. 2.11, images of fluorescence beads are compared before and after processing. There is almost no change for the beads in the center of the images before and after processing. While the beads at two edges in Fig. 2.11(a) are obviously stretched along Xaxis and returned to the round shape after processing as in (b).  33  Figure 2.11 Experiments to verify the effectiveness of the linearization algorithm for correction of image distortion. (a) Before processing (b) After processing (TPEF image of fluorescence beads)  To display the original and processed images in real-time, a program with user-friendly Graphical-User-Interface (GUI) has been composed using the Microsoft Foundation of Classes (MFC) in Visual Studio 2005. The program code for real-time image processing is supplied in the Appendix.  34  2.4.  EEM Acquisition  As introduced in Chapter 1, EEM is actually a matrix of emission intensity as a function of the excitation and emission wavelengths. In the EEM sub-system the laser excitation wavelength is tuned from 730 nm to 920 nm in a 10 nm step. (Specified tuning range of the fs laser is 720 nm – 950 nm. However, the laser cannot be constantly mode-locked at some extreme wavelengths of the tuning range. Hence we choose the range of 730 nm – 920 nm so that the laser can work properly at all the excitation wavelengths for accurate EEM acquisition.) The system collects the emission spectrum for each excitation wavelength with the spectrometer. These spectral data are processed and summarized into a matrix and plotted as a contour map for analysis. In order to make the calibration and measurement convenient and repeatable, an automatic tool is programmed in Labview. It uses a time sequence structure in Labview to coordinate the components including the laser, the attenuator rotation motor, and the spectrometer. Automatic EEM acquisition minimizes the time spent on adjustment of device settings for the next excitation step, which helps to decrease the chance of photo bleaching and photo damage of tissue sample. Furthermore, it is necessary to repeat the calibrations of EEM sub-system as long as there is any change on optics setup or alignment. Thus similar automatic programs have been developed for calibrations, especially for excitation power calibration, which should be carried out on a regular basis due to the laser power fluctuation. These automatic programs save a lot of time cost by the calibrations and measurements. Details about system calibrations and measurements will be discussed in the next chapter. Fig. 2.12 (A) shows the operation process of the automatic EEM acquisition program. The gray frame and red thick arrow set the timing order of the operation process. Firstly it will load data file of rotation angles for the optical power attenuator motor, which is acquired with the excitation power calibration and will maintain the excitation power at a constant level for different excitation wavelengths. Simultaneously, it will load Active X control module for rotation motor control and initialize electrical communication path for laser control. The program will wait until all these tasks have finished and then go to the next time frame. In the second frame, it acquires multiple emission spectra for each excitation wavelength, calculates the average spectrum, returns to the start of the frame to set a new excitation wavelength and new rotation angle of the optical power attenuator, 35  and acquires another group of emission spectra. The loop will continue until all the emission spectra have been acquired. Then the program will advance to the final unit to stop the control of system modules including laser, rotation motor and spectrometer, and save averaged spectra data to a .TXT file. Another separate program for single spectrum acquisition can be applied to measure the background spectrum either before or after the EEM measurement, which needs the operator to manually close the laser shutter. Fig. 2.12 (B) shows the user-interface of this automatic program. Basic settings can be changed directly on the interface such as laser scanning range and step width, motor rotating speed, and exposure time of spectrometer. It will display real-time settings of key parameters including the excitation wavelength of the laser, angle of optical power attenuator motor and emission spectrum under specific excitation wavelength. Operator can also know the number of groups of data that have already been acquired and have a record of the progress. Finally a standard “Save As” window will pop out to let the operator choose where and how to save the spectra data.  36  A  B  Figure 2.12 Labview program for automatic EEM acquisition (A) Program operation process (B) Program interface in Labview  37  2.5.  System Performance  Table 2.1 Performance of the multimodality microscopy and spectroscopy system Parameter  Description  Imaging Lateral Resolution  ~ 0.5 µm  Imaging Axial Resolution (Estimated value  ~ 1.5 µm  based on lateral resolution) Field of View (adjustable)  60 µm × 60 µm – 500 µm × 500 µm  Imaging Speed  ~12 FPS  Laser Tuning Range  720 nm – 950 nm  Spectral Resolution  ~4 nm  Spectrometer CCD Data Resolution  ~0.48 nm/pixel  Table 2.1 includes the major performance parameters of the imaging-guided EEM system. The lateral resolutions of the three imaging channels are all around 0.5 µm. The axial resolution is estimated based on the lateral resolution [71]. Accurate measurement results will be shown in Chapter 3. Images can be acquired with a field of view ranging from 60 µm × 60 µm to 500 µm × 500 µm. The spectrometer has a data resolution of ~0.48 nm/pixel and a spectral resolution of ~4 nm. The multimodality system and nonlinear EEM sub-system have been constructed and custom modules are installed to tune the system for optimum performance. Software programs have been developed to synchronize the components in the system and facilitate real-time multiple-channel image acquisition and EEM measurement. However, for accurate measurement of nonlinear EEM, we still need calibrations and measurements of some key parameters of the system, which will be discussed in the next chapter. 38  Chapter  3  Calibration  and  Measurement  of  System  Parameters Different from single-photon EEM, two-photon EEM uses fs laser as the illumination source for TPEF and SHG emissions. Some parameters related to the fs laser directly influence the emission efficiency. Hence there are certain measurements needed specifically for the ultrafast laser source such as measurements of excitation beam pulse width and spectral bandwidth. Basic calibrations are also needed for EEM acquisition, including excitation power calibration, spectrometer wavelength calibration and intensity response calibration for the detection beam path. This chapter will show the accomplishments on measurements and calibrations of these significant parameters.  3.1.  Excitation Beam Path  3.1.1. Excitation Power Calibration To compare the emission spectra under different excitation wavelengths, it is desirable to calibrate the excitation power to maintain it at a constant level over the whole excitation wavelength tuning range. As described in Chapter 2, a power attenuator unit is added right after the laser exit to adjust the excitation power over the sample. As shown in Fig. 3.1 (A), the unit consists of a ½ wave-plate and a polarization beam splitter (PBS). The intensity I of the polarized light that passes through the PBS is given by: I = I0 COS2 (θ)  (3-1)  Where I0 is the input intensity and θ is the angle between the beam’s polarization direction and the optical axis of the PBS. The ½ wave-plate is mounted on a motorized rotational stage (PRM1-Z7, Thorlabs Inc.). As the wave-plate is rotated, direction of the laser beam polarization varies continuously. Then the angle between the beam polarization and PBS optical axis will change continuously and cause the excitation power to be attenuated by a varying ratio according to Equation (3-1). Therefore, excitation power can be controlled by adjusting the rotation angle of the ½ wave-plate. When the ½ wave-plate is rotated by an angle ø, the laser beam polarization will be rotated by an angle 2ø. Thus the relation between output power of the attenuator and 39  rotation angle of the ½ wave-plate is: I = I0 COS2 (2ø + θ0) according to Equation 3-1, where θ0 is the initial angle between laser beam polarization and the PBS optical axis, and ø is the rotation angle of the ½ wave-plate. Two steps are completed to implement the excitation power calibration. Firstly, the ½ wave-plate is rotated from 0-180o and excitation power is measured after the power attenuator unit for different angles of rotation. The curve (Fig. 3.2) can provide us with a general idea how the excitation power changes (increase or decrease) with the increase of motor rotation angle so that the automated calibration program can be programmed to adjust the angle of rotation motor to attain the target value of excitation power. Four of these curves are shown in Fig. 3.2 for various excitation wavelengths (750 nm, 800 nm, 850 nm, 900 nm).  Figure 3.1 Optical setup of the power attenuator kit.  Figure 3.2. Relation of excitation power versus rotation angle of ½ wave-plate. (Excitation power is measured at the exit of power attenuator. ½ wave-plate is rotated from 0 to 180 o for each excitation wavelength. Four excitation wavelengths are investigated including 750 nm, 800 nm, 850 nm, 900 nm.)  40  In the second step, we will calibrate the rotation angle of attenuator for each excitation wavelength to maintain the excitation power after the objective at a constant level for all excitation wavelengths. Desired excitation power and an initial excitation wavelength are specified in the program, which then reads out the power after the objective and adjust rotation angle of the ½ wave-plate accordingly. The automatic program will keep reading current power and adjusting attenuator rotation angle until the excitation power reaches our goal. Then it will progress to the next excitation wavelength and repeat steps as the above. Finally it will record a group of calibrated rotation angles for each excitation wavelength (730 nm-920 nm, 10 nm step size). This set of angles will be input to the EEM acquisition program for automatic adjustment by the power attenuator to guarantee a constant level of excitation power for all excitation wavelengths. 22 21  Excitation Power (mw)  20 19 18 17 16 15 14 13 12 730 740 750 760 770 780 790 800 810 820 830 840 850 860 870 880 890 900 910 920 Excitation Wavelength (nm)  30 minutes  1 hour  5 hours  Figure 3.3. Experiment to test laser power fluctuations. (Excitation power is measured after the objective.)  Excitation power always has fluctuations due to inherent instability and operation noise of the mode-locked laser. A test is implemented to check the level of fluctuations to ensure the effectiveness of excitation power calibration. After the laser is turned on for a certain period of time, excitation power for different wavelengths is measured with the rotation angles of attenuator set as that recorded in the previous calibration. Three groups of data are collected at a different time after the laser has been turned on (30 minutes, 1 41  hour and 5 hours), as shown in Fig. 3.3. The fluctuations are all within ±5% of the desired power (20 mw), which will not have significant influence on EEM acquisition. Therefore it will suffice to implement power calibration at the commencement of every EEM experiment.  42  3.1.2. Laser Pulse Width Measurement The two-photon signal strongly depends on the spatial and temporal property of excitation light. The time-averaged emitted fluorescence photon flux can be expressed as the following equation [72]: 〈 ( )〉 ≈ ø  8 〈 ( )〉2  (3-2)  where C is the fluorophore concentration, δ is the two-photon absorption cross section, η2 is fluorescence quantum efficiency of the fluorophore, ø is the fluorescence collection efficiency of the measurement system, n is the refractive index of the sample medium, P is the incident power, f is pulse repetition rate, τ is the pulse width (FWHM), λ is the excitation wavelength and gp is a unit-less factor that depends on the temporal profile of the laser pulse (gp = 0.664 for Gaussian pulse shape, and gp = 0.588 for Hyperbolic-secant square pulse shape).  The factor ½ indicates that two photons are needed for each  excitation [72]. From the above equation, the fluorescence signal has approximately an inverse proportional dependence on the pulse width of fs laser pulses. As published in [73], twophoton excited emissions are found to increase proportionally to the inverse of pulse width in the range from ~400 fs to sub-20 fs. Therefore it is important to have a record of excitation pulse width for the whole wavelength tuning range to assist the analysis of nonlinear EEM. As listed in the specification, the fs laser has a pulse width less than 140 fs at the peak of wavelength tuning curve and less than 200 fs over the tuning range. An autocorrelator (FR-103MN, Femtochrome Research) is used to measure the pulse width.  43  Figure 3.4. Optical setup of autocorrelator for pulse width measurement.  The laser pulse duration cannot be easily measured by optoelectronic methods, since the response time of photodiodes and oscilloscopes are at best of the order of 200 fs, but laser pulses can be made as short as a few fs. Utilizing the autocorrelator, fs laser intensity is split into two pulse trains with a variable time delay τ between each other. When these two pulse trains are overlapped spatially in a second-harmonic-generation crystal, the crystal will generate SHG pulses. Intensity of the SHG pulse is proportional to the product of intensities of the two incident pulses. The detector can only measure autocorrelation of the intensity of the newly generated pulse. FWHM of the autocorrelation trace is used to calculate the width of the original pulse. The ratio between this FWHM value and the fs laser pulse width depends on the pulse shape. The ratio of 0.707 for Gaussian pulse is adopted in our experiments. Pulse width is measured at different positions in the optical path (after laser exit, after the polarization beam splitter (PBS), after the objective) as shown in Fig. 3.5. For measurements at the position after objective lens, large variation exists and the average level of pulse width is also largely increased compared with measurement at laser exit. It will result in variations and decrease of TPEF intensity across the laser wavelength tuning range due to the inverse proportional relation between TPEF intensity and laser pulse width. We have found that the main reason for the pulse-broadening is due to dispersion of the optical components used in the excitation beam path. The problem was solved by (1) replacing all the dielectric mirrors (totally 6 pieces) in the system with specially processed silver coated  44  mirrors (Dielectrically Enhanced Ag Mirror, Type I, FEMTOLASERS) and (2) replacing the 25 cm ×25 cm × 25 cm beam splitter cube with a smaller one (5 cm × 5 cm × 5 cm).  Figure 3.5. Pulse width measurement with the autocorrelator at different locations in the system. (Two types of PBS and two types of objectives have been investigated. Laser wavelength is tuned from 720 nm to 950 nm with an increment of 10 nm. )  Improvement of the system performance is tested using TPEF images of pure bovine elastin sample. The laser is tuned from 730 nm to 920 nm with a step of 10 nm and one frame of image is acquired for each excitation wavelength. In Fig. 3.6, a group of images are recorded for the old and new setup respectively. In Group A, images are taken with the old setup and under the excitation power of 30 mW while Group B is acquired with the new setup and under the excitation power of 20 mW. Group B has a much stronger TPEF intensity although less excitation power is applied compared with Group A. Average intensity of each image frame is calculated and a curve is drawn to show the relation between TPEF intensity and excitation wavelength. As seen in Fig. 3.7, TPEF intensity has severe fluctuations across the laser tuning range with the old setup, while the curve becomes much smoother with the new setup.  45  A  B  Figure 3.6 TPEF images of bovine elastin powder taken under various excitation wavelengths. (A) Taken with conventional dielectric mirrors, 30 mW excitation power, 200 µm × 200 µm Field of View. (B) Taken with new dielectrically enhanced Ag mirrors, 20 mW excitation power, 200 µm × 200 µm Field of View.  46  1 0.8 0.6 0.4 0.2 0 700  750  800  850  900  950  Previous Dielectric Mirrors New mirror-1hour after laser turned on_forward scan Figure 3.7. TPEF image intensity versus excitation wavelength for comparison of two optical setups with different types of reflectance mirrors.  It is further tested with EEM experiments of pure keratin sample. Fig. 3.8 shows the experiment results. As seen in the EEM contour map for the old setup, there are some fluctuations of TPEF intensity with the increase of excitation wavelength, which all disappeared in the EEM taken with the new setup. All of these comparisons show the improvement of pulse width handling capability of the new setup, which provides much better performance for nonlinear EEM measurements.  47  Figure 3.8 Comparison of EEMs of pure keratin sample measured with two setups respectively (A) EEM measured with the old setup. (B) EEM measured with the new setup.  48  3.1.3. Laser Spectral Bandwidth Measurement When the spectral bandwidth of laser excitation beam becomes wider than the fluorophore absorption window, two-photon absorption efficiency will be reduced, which will affect TPEF emission intensity [73]. Thus we also did measurements of spectral bandwidth of excitation beam over the laser tuning range from 730 nm to 950 nm. After transmitting through the excitation path and objective lens, the laser light is collected by an integration sphere. The collected light is sent to a spectrometer through a fiber bundle for spectral measurement. The spectral bandwidth is calculated as FWHM of the spectrum. There are 3 groups of measurements with forward and backward tuning of laser in each group, as shown in Fig. 3.9. All of the 6 sets of data show good agreement with each other. The bandwidth is in the range from ~6 nm to ~14 nm.  Laser Bandwidth 14  FWHM (Bandwidth) (nm)  12 10 8 6 4 2 0 700  750  800  850  900  950  Excitation Wavelength (nm) Group 1_Forward Scan  Group 1_Backward Scan  Group 2_Forward Scan  Group 2_Backward Scan  Group 3_Forward Scan  Group 3_Backward Scan  Figure 3.9 Excitation beam bandwidth (FWHM) measurement result. There are 3 groups with forward and backward tuning of laser in each group.  49  3.1.4. Imaging Resolution Measurements According to Equation 3-2, the number of two-photon fluorescence photons collected per unit time is proportional to the fluorescence collection efficiency ø of the measurement system, which is dependent on the numerical aperture (NA) of the objective lens (NA = 1; Magnification = 60. LUMPLFLN60XW, Olympus). As we know, the resolution is inversely proportional to NA of the objective lens. Thus the detected intensity of twophoton emission will be affected by the system resolution. It is necessary to measure the resolution of this system for different excitation wavelengths. Optical resolution is normally defined as the shortest distance between two points on a sample which can be differentiated from each other as individual entities. When emitted light from different points on a sample is collected by the objective lens and reconstructed as an image, light from each point will generate a pattern described as point spread function (PSF). Full width half maximum (FWHM) of the central bright region of the PSF is usually specified as the optical resolution. The FWHM of PSF in the x-y plane (lateral resolution) for twophoton microscope can be calculated as [71]: =  √  ×0.325 0.9  (3-3)  For Comparison, FWHM of the PSF for single-photon fluorescence microscope is expressed as: =  0.61  (3-4)  To measure the PSF, it is widely accepted to apply fluorescent micro beads with a diameter smaller than the FWHM of PSF [74]. The fluorescent bead used in this measurement has a diameter of Ф=116 nm (CAT: G0500, LOT: 34361, Duke Scientific Corp.), which is much smaller than the theoretical resolution of two-photon microscope (~400 nm). It was diluted with water and sonicated by an ultrasonic water-bath to further separate the particles. Then the bead suspension was blended with silicon gel. After the mixture dried out, it was placed on a plastic plate for measurement. Using a certain excitation wavelength, TPEF image of the beads is acquired and a line is drawn across the centre of the bead on the image. Intensity distribution along this line is plotted and fitted to a Gaussian curve using a Matlab program. FWHM of the curve is calculated as 50  the resolution value (Fig. 3.10). Several beads are measured and an average value of resolution is obtained for that specific excitation wavelength.  A  B  Figure 3.10 PSF of a single fluorescent bead (A) Image taken with excitation wavelength of 820 nm and FOV of 60 µm × 60 µm. (Each pixel is approximately related to the actual size of 0.1 µm × 0.1 µm on the sample.) (B) Intensity distribution of the center vertical line of the bead image (Background has been subtracted from the original data for Gaussian fitting. For a standard Gaussian curve: = 2√2 2 × ), where σ is a coefficient depending on the shape of the curve.  51  The relation between the measured resolutions with excitation wavelengths is plotted in Fig. 3.11. For comparison, theoretical resolutions of two-photon and single-photon fluorescence are also shown in the same figure, which are calculated based on Equation 3-3 and 3-4 respectively. There is also a linear fitting of the experimental data to assist our comparison. The average level of experimental resolution is higher than the theoretical value due to variations of refractive index and scattering effects in tissue samples [28]. However, the experimental resolution is still approximately proportional to the excitation wavelength, which is similar as the theoretical calculation. This result indicates that the resolution values of this system are in a reasonable range, which will not have significant influence on EEM analysis.  Theoretical FWHM of two-photon PSF Theoretical FWHM of one-photon PSF Measured FWHM of two-photon PSF Linear Fitting to the measurement  Figure 3.11 Resolution measurement result for excitation wavelengths from 730 nm to 920 nm. Experimental and theoretical result of two-photon resolution and the linear fitting to the experimental result are shown in the picture for comparison. Theoretical curve of one-photon resolution is also shown here.  52  3.2.  Emission Beam Path  This section mainly includes the wavelength calibration for the spectrometer and intensity response calibration for the emission beam path. 3.2.1. Spectrometer Wavelength Calibration  Figure 3.12 Spectrum of Hg(Ar) lamp. The x-axis represents the pixel location on the spectrometer CCD array.  Spectrometer applies a grating to separate and distribute the spectral components of the incoming signal along a CCD array. To acquire spectrum of the signal, we need to know the relation between actual wavelength of each spectral component and its location on the CCD array, which is obtained by wavelength calibration. The spectrometer in this system has two gratings to use. One is blazed at 550 nm, covering a spectral window from 300 nm to 800 nm while the other one is blazed at 800 nm, covering a spectral window from 560 nm to 1000 nm. The first one can be used for EEM measurement because the useful 53  signal is mainly in the range from 350 nm to 700 nm. The second one can be used for laser bandwidth calibration to detect laser beam in the range from 700 nm to 1000 nm. A standard Mercury-Argon lamp is used for the calibrations. A “spectrum” of the Hg(Ar) lamp is acquired with the CCD pixel number as x-axis. The actual wavelengths of the measured spectral peaks are read from the lamp specification as shown in Fig. 3.12. Finally the relation between wavelength of each spectral component and its pixel number on the CCD array is obtained by a polynomial fitting of the data points. Then we can use this relation to convert x-axis in pixel number into actual wavelengths. For the second grating (560 nm – 1000 nm), we need some special processing. The available Hg(Ar) Lamp only has signal below 920 nm and we need to have at least one typical data point close to 1000 nm for accurate polynomial fitting. Therefore, we applied a theoretical model to implement the fitting of available data points. For a diffractive grating-based spectrometer, the coordinate function is defined by the diffraction equation and the spectrometer design, and can be written as the following [75]: ( )=  1  2  −  −  1  2  −  (3-5)  0  Where x is the coordinate position on the CCD array, κ0 is wave number at the center of the first pixel where the coordinate equals zero, κc is the wave-number at the center of spectrum, µ is the groove density (grooves/mm) of the grating and f is the effective focal length of the focusing lens. This theoretical curve is then fitted to typical data points read from the lamp. The fitted curve is used to convert CCD pixel number to actual wavelengths. The comparison between the theoretical model fitting and basic polynomial fitting is shown in Fig. 3.13.  54  Figure 3.13 Relation between wavelength and CCD pixel number of the spectrometer. Four curves are drawn including: theoretical fitting of lamp peaks, measured lamp peaks, quadratic fitting of lamp peaks and polynomial fitting of fs laser peaks.  55  3.2.2. Emission Beam Path Intensity Response Calibration Intensity response calibration is used to correct the difference of the transmission efficiency of the emission beam path for different wavelengths. A standard tungsten lamp (RS-10A-1, GAMMA SCIENTIFIC) and the supplied spectral data are used for this calibration work. The lamp light beam transmits through an integration sphere and the objective lens, and gets collected by the spectrometer (Fig. 3.14). The measured spectrum will show some difference with a specified standard spectrum of the lamp, which is caused by the variation of the transmission efficiency for different wavelengths. Then a ratio between these two spectra is calculated to correct the EEM measurements through this emission beam path.  Figure 3.14 Setup for intensity calibration for the emission path.  The comparison of the two spectra is shown in Fig. 3.15 (A). The red curve is the standard spectrum and the blue curve is the measured spectrum. Then the ratio is calculated as r = Iblue/Ired. The lamp intensity is very weak in the UV region (below 400 nm), so SNR in that region is very low and the calibration ratio will be very noisy below 400 nm. To obtain accurate calibration ratio over the full wavelength range, two measurements are needed. First, the entrance aperture of the integration sphere is fully open and exposure time of the spectrometer is increased to 4 seconds, and then a ratio curve is acquired as in Fig 3.15 B. Second, the aperture is partly closed and the exposure time is set as 1 second, then the second group of ratio is acquired as in Fig. 3.15 C. Finally stitch the two ratio curves using the UV region (<400 nm) of the first one and visible region (400 nm-700 56  nm) of the second one. The stitched ratio curve is shown in Fig. 3.15 D. The stitching point is chosen as 400 nm because in the case of B, the lamp spectrum will be saturated above 400 nm, while in the case of C, the SNR of the spectrum is high enough above 400 nm. Therefore the result in D will not be affected by saturated spectrum and have a high SNR for the whole wavelength range. A  B  C  D  Figure 3.15. Intensity calibration of emission beam path. (A) Specified spectrum of the lamp (blue) and measured spectrum of the lamp (red); (B) Measured spectrum of the calibration ratio (r = Iblue/Ired) with fully opened integration sphere entrance aperture and 4 seconds integration time of the spectrometer; (C) Measured spectrum of the calibration ratio (r = Iblue/Ired) with partly closed integration sphere entrance aperture and 1 second integration time of the spectrometer; (D) Stitched spectrum of the intensity calibration ratio.  Based on the above calibrations and measurements, we made the EEM measurements more accurate and reliable. The accomplishments in Chapter 2 and 3 have prepared the image-guided nonlinear EEM sub-system for the preliminary experiments which will be discussed in the next chapter.  57  Chapter 4 Experiment on Biological Samples Previous chapters introduced the work done on optical and electrical setup of the whole multimodality system and EEM sub-system, and calibrations and measurements of key parameters for nonlinear EEM acquisition. Based on these work, imaging guided nonlinear EEM function is realized and optimized. This chapter mainly includes the work to verify this function through preliminary measurements on purified endogenous fluorophores as well as normal and diseased human skin tissue samples.  4.1.  Three Imaging Channels to Guide EEM Acquisition  As discussed in Chapter 1, the three imaging techniques, including TPEF, SHG and confocal, have different contrast mechanisms and can be applied to reveal various morphological characteristics of skin tissues. An integration of these three techniques can provide us with a useful preview of tissue sample to guide the acquisition of EEM, and more comprehensive information for tissue analysis. Fig. 4.1 and Fig. 4.2 show imaging results of a piece of normal human skin tissue sample from the scalp. The excised skin samples were obtained from the surgery unit of the VGH Skin Care Center, Vancouver, Canada. Using a microtome, 20 µm-thick cross sections were sliced from frozen skin tissues. The sections were sandwiched between microscope slide and cover glass and stored at 4 oC until used in experiments. The sample is illuminated by 785 nm fs laser pulses and under 30 mW excitation power with a field of view of 150 µm × 150 µm. The combined image is produced by using pseudocolor for each physical channel. We use red for confocal channel, blue for SHG channel, and green for TPEF channel. As shown in Fig. 4.1, clear fibrous structures are observed in all three channels. It indicates that most probably it is in the dermis compartment which mainly comprises of extracellular fiber structures. As introduced in Chapter 1, SHG signal (blue) mainly comes from collagen fiber and TPEF (green) may come from elastin and/or collagen fibers. As seen in the combined image, there is no overlap between green structure and blue structure. Thus the TPEF signal mainly originates from elastin fibers.  58  Figure 4.1 Cross section of normal human scalp skin tissue section, dermis layer (thickness - 20 µm, FOV - 150 µm × 150 µm, excitation wavelength-785 nm, excitation power - 30 mw, 30 image frames were averaged for TPEF and SHG channels). In the combined false color image, we used red for confocal channel, blue for SHG channel, and green for TPEF channel.  Fig. 4.2 is the result from another piece of normal human skin tissue section. It is imaged under the same conditions as the first one. The thick fiber bundle structure of collagen can be observed clearly with high resolution and contrast in the SHG channel. In TPEF channel, thin elastin fiber is visualized. The combined images in these results show various morphological structures of tissue compositions and relative distribution and density of these compositions. It can also assist in distinguishing TPEF of elastin fiber from that of collagen fiber, which is comparatively difficult using spectral analysis. Therefore, we can have a preview and brief understanding of the tissue sample in advance so that EEM can be acquired from  59  interesting regions of skin tissues. Moreover, much more comprehensive information can be obtained by using the imaging channels together with nonlinear EEM analysis.  Figure 4.2 Cross section of normal human scalp skin tissue section, dermis layer (thickness-20µm, FOV-150µm x 150µm, excitation wavelength-785nm, excitation power-30 mw, 30 image frames were averaged for TPEF and SHG channels). In the combined false color image, we used red for confocal channel, blue for SHG channel, and green for TPEF channel.  60  4.2.  Imaging-Guided Nonlinear Excitation-Emission-Matrix  4.2.1. Experiments on Purified Endogenous Fluorophores As introduced in Chapter 1, EEM is a unique fluorescence signature for purified fluorophore. It is important to acquire these signatures first for EEM analysis of complex tissue samples. Nonlinear EEM of collagen and elastin has been measured previously ([66]J.Chen 2009). However, to the best of our knowledge, there is no comprehensive study of nonlinear EEM of various purified fluorophores. To assist nonlinear EEM analysis of skin tissues, we completed the measurements of typical endogenous fluorophores of skin tissue including elastin, collagen, keratin, NADH, FAD and melanin. Fig. 4.3 is the results of pure human elastin. The purified elastin sample (GH421, Elastin from human skin, ELASTIN PRODUCTS CO., INC.) is light yellow powder and is stored at 4 oC until used for the EEM measurement. It was evenly placed on a microscope slide with a piece of cover glass on top. Imaging and single-photon & two-photon EEM data have been acquired for this sample. Single-photon EEMs were measured by tuning the excitation wavelength from 250 nm to 600 nm using a commercial system (FluoroLog3, Edison). For two-photon EEM, the fs laser is tuned from 730 nm to 920 nm. We have done measurement to verify the laser performance. The actual excitation wavelength will have some difference with the specification and the differences fluctuate from 3nm to 6nm. Excitation power is 20 mw (+/- 5%) for all excitation wavelengths. For every excitation wavelength, there is normally a peak in the emission spectrum. To facilitate analysis of EEM results, we use the parameter excitation-emission-pair to represent the emission wavelength of that peak and the excitation wavelength related to it. Fig. 4.3 A and B are two-photon and single-photon EEM respectively with the same range of excitation wavelength. In single-photon EEM, the maximum excitation emission pair is at (375 nm, 450 nm), while it is at (~730 nm, ~460 nm) for two-photon EEM. The maximum fluorescence occurs at similar emission wavelengths for single-photon and two-photon EEM, and the two EEMs have similar patterns over the whole range. As expected, the excitation wavelengths in two-photon EEM are roughly two times of that in single-photon EEM. This comparison shows similarity and difference of two-photon and single-photon EEM. As published in [66], purified elastin sample (GH421, Elastin from human skin, ELASTIN PRODUCTS CO., INC.) showed TPEF excitation-emission-pairs 61  at (750 nm, 457.9 ± 0.4 nm), (770 nm, 465.5 ± 0.4 nm), (810 nm, 479.8 ± 0.8 nm), (850 nm, 503.0 ± 0.2 nm) and (870 nm, 512.9 ± 0.4 nm), which are shown by the white dots in Fig. 4.3 A. These results match well with our measurements. Single-photon EEM with an excitation range from 250 nm to 600 nm is shown in C, which exhibits additional peaks in the UV excitation range.  Figure 4.3 Pure Human Elastin (A) Two-photon EEM (excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 2 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Singlephoton EEM (excitation wavelength: 250 nm – 600 nm)  62  A  B  C  Figure 4.4 Pure Human keratin (1mg/ml solution in urea) (A) Two-photon EEM (Excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 2 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Single-photon EEM (excitation wavelength: 250 nm – 600 nm)  63  Fig. 4.4 shows the EEM results for purified human keratin sample (Product K0253, Sigma-Aldrich, 1 mg/ml solution in urea) from human epidermis. 30 ml of the solution was instilled into a flat plastic vessel with a concave shape in the centre. A piece of cover class was placed right on top of the concave part of the vessel so that the water-immersed objective lens can be applied on the sample. The single-photon and two-photon EEM results have different patterns and different maximum excitation-emission pairs, because two-photon and single-photon fluorescence have different absorption coefficients which result from various selection rules for the transition of electronic states of the fluorophore molecules. The maximum excitation emission pair is at (~380 nm, ~450 nm) for singlephoton EEM, while it is at (~730 nm, ~475 nm) for two-photon EEM. Due to the limit of scanning range of the fs laser in our system, data is not acquired for excitation wavelengths below 730 nm. In Fig. 4.4 A, there are two excitation emission pairs at (760 nm. 490 nm) and (860 nm, 530 nm). These results match with the reported results in [76] as (760 nm, 4755 nm) and (860 nm, 5155 nm), which are shown as the two white dots on the EEM plot, for pure human keratin (K0253, Sigma-Aldrich, 30mg/ml solution in urea). Fig. 4.5 shows the EEM results of purified human collagen sample (Type I). The collagen sample (HS150 Collagen Type I, Acid soluble, from human skin, ELASTIN PRODUCTS CO., INC.) was supplied as a white sponge-like material and stored at 4 oC until used for EEM measurements. It was evenly placed between a microscope slide and a piece of cover glass. Fig. 4.5 A and B show two-photon and single-photon EEM respectively. As seen in A, there are a series of narrow peaks in addition to the TPEF signal. These narrow peaks originate from SHG emission from the purified collagen fiber because the emission wavelengths of those peaks are exactly half of the corresponding excitation wavelengths. This is a unique characteristic of collagen fiber. Since single-photon EEM does not have SHG signals, two-photon EEM has better performance in distinguishing epidermis from dermis, and elastin fibers from collagen fibers. As published in [66], there are several excitation emission pairs for TPEF of pure human skin collagen Type I (HS150, ELASTIN PRODUCTS CO., INC.) including: (770 nm, 472.5 nm), (810 nm, 487.6 nm), (850 nm, 503.0 nm) and (870 nm, 512.9 nm), which are shown by the white dots in Fig. 4.5 A. The published result shows agreement with the TPEF part in our EEM 64  measurement result as in Fig. 4.5 A. Single-photon EEM with an excitation range from 250 nm to 600 nm is shown in C, which indicates that the major peaks for single-photon fluorescence from human keratin are located in the UV excitation range.  A  B  C  Figure 4.5 Purified human collagen (powder) (A) Two-photon EEM (excitation wavelength: 730 nm – 920 nm, FOV: 100 µm × 100 µm, Power: 30 mW, Exposure time: 3 s) (B) Single-photon EEM (excitation wavelength: 365 nm – 460 nm) (C) Single-photon EEM (excitation: 250 nm – 600 nm)  65  The excitation spectrum of SHG signal (Peak value of SHG emission spectrum vs. excitation wavelength) is interesting for analysis because it can provide us with suggestions on choosing the optimal laser wavelength for two-photon research on tissues which are rich in collagen fibers. Fig. 4.6 shows three different SHG excitation spectra measured from three different locations on the purified human collagen sample. These excitation spectra showed a major peak at ~790 nm and a secondary peak at ~850 nm. These results match to a certain extent with the results in ([66]J.Chen 2009), which showed a major peak at ~810 nm and a secondary peak at ~850 nm.  SHG Peak Value (a.u.)  1.2 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 700  750  800 850 Excitation Wavelength (nm)  Position 1  Position 2  900  950  Position 3  Figure 4.6 SHG excitation spectrum of purified human collagen fiber (Type I) (Excitation wavelength: 730 nm – 920 nm, 20 mw excitation power, FOV: 100 µm × 100 µm, Exposure time: 3 s)  66  SHG Peak Value (a.u.)  A  1.2 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 700  750  800 850 Excitation Wavelength (nm)  Position 1  B  Position 2  900  950  Position 3  1.2  SHG Peak Value (a.u.)  1 0.8 0.6 0.4 0.2 0 700  750  800 850 900 Excitation Wavelength (nm) Position 1 Position 2 Position 3  950  Figure 4.7 SHG excitation spectra (A) Fish scale sample (B) Rat tail tendon sample. (Excitation wavelength: 730 nm – 920 nm, Excitation power: 20 mw, FOV: 100 µm × 100 µm, Exposure time: 1 s)  SHG excitation spectra of other samples were also measured including fish scale and rat tail tendon, which are rich in collagen fibers (Fig. 4.7). Comparing the excitation spectra in Fig. 4.6 and Fig. 4.7, different samples show different spectral characteristics in terms of peak positions and width of spectrum. These characteristics may be related to structure and chemical characteristics of collagen fibers, which are important for complex tissue analysis. In order to do this analysis, SHG excitation spectra of different tissues need to  67  be acquired and compared, which will be considered as a potential application of the nonlinear EEM sub-system. In addition to the fluorophores measured above, there are some other important endogenous fluorophores for two-photon study of skin, such as NADH & FAD which are significantly involved in cell metabolism activities, and melanin which is an excellent photoprotectant for viable skin tissues. The purified NADH (N6879-25MG, αNicotinamide adenine dinucleotide, reduced disodium salt, Analog of β-NADH, chemically reduced, SIGMA ALDRICH) and FAD (F6625-10MG, Flavin adenine dinucleotide disodium salt hydrate, SIGMA ALDRICH) are both powder and sandwiched between microscope slide and cover glass for inspection. Melanin powder (M8631250MG, Sigma Aldrich) is resolved in 1mol/l NH4OH and made into a 1mg/ml solution. EEM of melanin solution is measured in the same way as the keratin solution as introduced above. Two-photon EEM of these three fluorophores are summarized in Fig. 4.8. The excitation-emission pairs in those EEMs show agreement with the reported results as: (730 nm, 545 nm) and (900 nm, 540 nm) for FAD (solution, Sigma Aldrich), (800 nm, 565 nm) for melanin (Synthetic melanin prepared by oxidation of tyrosine with hydrogen peroxide, solution in KOH, Sigma Aldrich), and (730 nm, 460 nm) for NADH (solution, Sigma Aldrich) [77, 78]. Fig. 4.8 shows these published results as white dots on the contour map. The nonlinear EEM for each purified fluorophore measured above has a distinct pattern and it is very promising to use these EEMs as signatures for characterization of complex skin tissues.  68  A- NADH (2-photon)  B- NADH (1-photon)  C - FAD  D - Melanin  Figure 4.8 Nonlinear EEM of more fluorophores (Excitation wavelength: 730 nm–920 nm, Excitation power: 20 mw, Exposure time: 3 s. Both of 1-photon and 2-photon EEM for NADH are measured.)  69  4.2.2. Experiments on Human Skin Tissue The following experiment is performed on a piece of fresh normal skin (~2 mm thickness, ~1 cm × 1 cm area) excised from human temple. The bulk tissue without sectioning was applied for EEM measurement within 1 hour after excision. After all EEM measurements, we fixed the tissue in formalin and then applied H & E staining on it for histological image acquisition. As shown in Fig. 4.9 A, the cross section view of epidermis-dermis junction is measured with a field of view of 100 µm × 100 µm. In Fig. 4.9 A, the multichannel images are shown in pseudo color with confocal in red, TPEF in green, and SHG in blue. Fig. 4.9 B is the H&E stained histology image of a similar region as shown in Fig. 4.9 A. The yellow square represents an area of 100µm × 100µm which matches with the filed-of-view in Fig. 4.9 A. The papillary structure can be seen in both A and B, which is a typical structure in epidermis-dermis junction. EEM for this cross section contains two parts, wide TPEF curves and narrow SHG peak array. The emitted SHG peak is located at exactly half of the excitation wavelength as shown in Fig. 4.9 C, which originates from collagen fibers. TPEF part of EEM has similar pattern and excitation-emission-pairs as pure elastin sample, however, it is shifted a little to the longer wavelength side, which may be due to contribution of emission from keratin from cells in epidermis.  70  A  B  C  SHG  Figure 4.9 Image-guided EEM of normal human temple skin cross section. (Excitation wavelength: 730 nm – 920 nm, excitation power: 30 mw, FOV: 100µm × 100µm, Exposure time: 3 s) (A) Pseudocolor image of a combination of three channels including confocal (red), TPEF (green) and SHG (blue) (Excitation Wavelength: 790 nm, Averaged image of 50 frames) (B) Histology image related to the image in A (H&E staining) (C) Nonlinear EEM (white arrow shows the SHG signal)  Using the optical sectioning capability of the multi-modality system, five distinct layers are measured in the vertical direction from stratum corneum to the dermis layer. As shown in Fig. 4.10, 4 sub-layers of epidermis compartment and 1 layer from dermis  71  compartment are measured. For the epidermis compartment TPEF channel is mainly used for the imaging guidance purpose. TPEF channel is represented by the green pseudocolor and SHG channel is in red. Imaging depths of these four sub-layers are: 10 µm, 20 µm, 30 µm and 40 µm. The other layer with an imaging depth of 60 µm is in the dermis compartment. Field of view for these images is 100 µm × 100 µm and the scale bar in the figures is for 20 µm. All Images are taken under the excitation wavelength of 790 nm and the excitation power of 30 mW. As seen in the averaged image (50 frames) in Fig. 4.10 (A), stratum corneum cells have large size. The EEM maximum excitation emission pair is located at (730 nm, 470 nm), which is close to that of purified keratin as in our previous measurement (730 nm, 480 nm). As we image deeper, the number of cells keeps increasing while the size of cells decreases. EEMs for the first two sub-layers (A & B) indicate that most of the TPEF signals in these layers come from keratin, which mainly exists at the surface of skin. For the third and fourth sub-layers (C & D), the stratum spinosum and stratum basale, EEMs have the pattern and maximum excitation emission pairs resembling that of pure NADH as shown in the last section. It indicates that most of the TPEF signal in these layers originates from NADH. NADH is significantly involved in redox reactions for metabolism, hence these EEMs show that cells are more active in stratum spinosum and stratum basale layers (C & D), which matches with the point of view of skin biology. Because cells in epidermis originate from stratum basale layer and keep proliferating and migrating to the top layers, the cells at a deeper layer are more active for metabolism. In Fig. 4.10 (E), image and EEM data have been obtained for one layer in the dermis compartment. SHG channel is added and optimized for this layer for image-guidance purpose. The dotted array in the EEM plot comprises SHG peaks with each emission peak located at exactly half of the excitation wavelength respectively. The SHG signal originates from collagen fiber in the dermis layer. The peaks with much larger bandwidth represent TPEF signal. TPEF part of this EEM has a similar major excitation emission pair (730nm, 450nm) and overall pattern as pure elastin. Signal acquired in the imaging channels supports this analysis. TPEF imaging channel (green) mainly contains the thin elastic fiber structure while the fiber bundle structure of collagen can only be observed in SHG imaging channel (red). There is no overlap between SHG and TPEF signals, which 72  indicates that collagen fiber in fresh skin tissue doesn’t have strong TPEF emission at 790 nm excitation. SHG intensity in this EEM plot is attenuated 20 times so that the contour details of SHG and TPEF signal can be displayed on a single plot. The SHG/TPEF ratio for purified collagen is different from that of collagen fibers in skin tissue, which may be due to the fact that the purification process of collagen has changed its structure which is responsible for SHG emission.  73  10µm  20µm  A  B  30µm  C  40µm  60µm  D  E  Figure 4.10 Image-guided nonlinear EEM of normal human temple skin. Left column shows the pseudocolor image (TPEF: green, SHG: red). Right column shows the EEMs related to the imaging area in the same row. Each row represents a different imaging depth (10 µm, 20 µm, 30 µm, 40 µm, 60 µm) (FOV: 100 µm × 100 µm; Scale bar: 20 µm; Excitation Power: 30 mW; Excitation Wavelength: 790 nm, Exposure time: 2 s, 50 frames averaged.)  74  Diseased human skin tissue with SK is also investigated for comparison with normal skin tissue EEM. This type of tissue usually has histological characteristics such as basale cells mixed with squamous cells and keratin-filled invaginations of the epithelium. As shown in Fig. 4.11, a transverse slice of human shoulder skin with SK is imaged with a field of view of 200 µm × 200 µm for both two-photon images (TPEF & SHG) and the histology image.  This is a cross section view including epidermis and dermis  compartments. Fig. 4.11A shows the combined image of TPEF and SHG channels (TPEF in green, SHG in red), and B shows the H&E stained histology image of a location on the same piece of sample. We searched for a similar location on the histology imaging as that on the nonlinear microscopy imaging, which match with each other to a certain extent.  a  b  c  d e  a  b  c  d e  Figure 4.11 Cross section images of human skin tissue with Seborrheic Keratosis. (A) Combined image of TPEF and SHG channels by pseudocolor (TPEF: Green, SHG: Red). (B) Histology image with H&E staining. (Field of view: 200 µm × 200 µm; Blue square: 20 µm × 20 µm; Excitation wavelength: 790 nm; Excitation power: 30 mW)  EEM has been acquired from five small areas on this cross section as indicated by the blue squares in Fig. 4.11. The EEMs are shown in Fig. 4.12 with the same order as the blue squares in Fig. 4.11. In Fig. 4.12a, EEM has a similar pattern and maximum excitation-emission-pair as that of pure keratin sample. It is related to the blue square (a) in Fig. 4.11, where keratin structure can be observed clearly in the histological image. In Fig. 4.12 b & d, both of the EEMs resemble that of pure NADH, which shows agreement with the cellular structures as highlighted by the blue square (b) and (d) in Fig. 4.11. 75  a  b  c  d  e  Figure 4.12 EEM for human skin tissue with Seborrheic Keratosis (a)-(e) is related individually to the blue square areas in Fig. 4.11. (Excitation wavelength: 730 nm – 920 nm; Excitation power: 30 mW, Exposure time: 2 s)  In Fig. 4.12c, EEM seems to be a mixture of signals originating from various structures. The contour pattern and excitation-emission pairs below excitation of 760 nm (Black dotted line) are similar to pure keratin, while EEM above 760 nm shows similarity with that of purified FAD sample. This spectral characteristic matches with the image signal in the blue square (c) on Fig. 4.11B, which includes both keratin and cellular structures in it. This analysis shows the advantage of EEM compared with pure emission spectra or excitation spectra, since this tissue characteristic cannot be fully revealed using only one 76  excitation wavelength or one emission wavelength. More importantly, this EEM reveals the fact that keratin structure has intruded into the middle sub-layers of the epidermis part of this SK tissue. This result matches with the biological characteristic of SK tissues, which shows the capability and great potential of applying the image-guided EEM on diagnosis of skin diseases. Fig. 4.12e shows EEM for the last blue square (e), which is located in the dermis layer mainly containing fiber structures. The dotted array is related to SHG peaks from collagen. The TPEF spectra have a contour pattern similar as purified elastin fiber. Comparing the results from normal skin tissue with that from SK, EEMs show different characteristics at certain layers and show reasonable agreement with the biochemical characteristics of both tissues. These results suggest great potential of applying imagingguided nonlinear EEM for skin disease diagnosis.  77  Chapter 5 Conclusions and Future Work I have presented here a nonlinear EEM spectroscopy system with multi-modality imaging guidance. The system has been designed to optimize the imaging and spectroscopy performance in terms of imaging speed, resolution and emission collection efficiency. Calibrations and measurements of key parameters of the EEM sub-system are also completed to further guarantee the accuracy and performance of nonlinear EEM experiments. Preliminary experiments on purified skin molecules as well as fresh normal and diseased skin tissues have been carried out. Our preliminary results have demonstrated great potential for applying the imaging guided EEM spectroscopy for skin disease diagnosis. With the depth-resolving capability it revealed information in distinct skin layers inside the epidermis and dermis compartments of human skin tissues. Those EEMs show agreement with the characteristics observed in imaging channels. It has also revealed interesting differences between normal and diseased skin tissues. The imagingguided nonlinear EEM system is promising for clinical applications on skin disease diagnosis. In view of the great potential of this system, its performance can be improved and more applications can be developed in the future. Firstly, decomposition algorithms may be applied on EEMs to identify biochemical compositions of complex tissues and concentration of these compositions. Secondly, the EEM measurement can be applied to more ex vivo skin tissues including normal tissue from different body sites and tissues with various diseases and carcinomas. Thirdly, to realize in vivo diagnosis, the speed of EEM acquisition needs to be improved. Fourthly, SHG excitation spectrum extracted from the nonlinear EEM results can be investigated to assist analysis of collagen rich tissues.  78  Bibliography [1] G.D.Goldman. “Squamous Cell Cancer: A Practical Approach.” Semin. Cutan. Med. Surg. 17 (1998): 80-95. [2] D.M.Pariser, P.K.Philips. “Basal Cell Carcinoma: When to Treat It Yourself, and When to Refer.” Geriatrics 49 (1994): 39-44. [3] L.H.Goldberg. “Basal Cell Carcinoma.” Lancet 347 (1996): 663-667. [4] S.Astner, M.Ulrich. “Diagnosis of Skin Cancer.” Chap. 1 in Managing Skin Cancer, by E. Stockfleth. Springer-Verlag Berlin Heidelberg, 2010. [5] N.S.Claxton, T.J.Fellers, M.W.Davidson. “Laser Scanning Confocal Microscopy.” http://www.olympusconfocal.com/theory/LSCMIntro.pdf (accessed 2010 йил 13-10). [6] J.Paoli, M.Smedh, M.B.Ericson. “Multiphoton Laser Scanning Microscopy-A Novel Diagnostic Method for Superficial Skin Cancers.” Semin. Cutan. Med. Surg. 28(3) (2009): 190195. [7] S.Gonzalez, Z. Tannous. “Real-time, In vivo Confocal Reflectance Microscopy of Basal Cell Carcinoma.” J. Am. Acad. Dermatol. 47 (2002): 869-874. [8] G.Pellacani, P.Guitera, C.Longo. “The Impact of In vivo Reflectance Confocal Microscopy for The Diagnostic Accuracy of Melanoma and Equivocal Melanocytic Lesions.” J.Invest.Dermatol. 127 (2007): 2759-2765. [9] A.L.Agero, K.J.Busam. “Reflectance Confocal Microscopy of Pigmented Basal Cell Carcinoma.” J.Am.Acad.Dermatol. 54 (2006): 638-643. [10] A.Gerger, S.Koller. “Diagnostic Applicability of In vivo Confocal Laser Scanning Microscopy in Melanocytic Skin Tumours.” J.Invest.Dermatol. 24 (2005): 493-498. [11] C.C.Lewandrowski, C.Williams. “Use of In vivo Confocal Microscopy in Malignant Melanoma.” Arch.Dermatol. 140 (2004): 1127-1132. [12] G.Pellacani, A.M.Cesinaro, S.Seidenari. “Reflectance-mode Confocal Microscopy of Pigmented Skin Lesions-Improvement in Melanoma Diagnostic Specificity.” J.Am.Acad.Dermatol. 107 (2005): 979-985. [13] A.Gerger, M.Horn, S.Koller. “Confocal Examination of Untreated Fresh Specimens from Basal Cell Carcinoma: Implications for Microscopically Guided Surgery.” Arch.Dermatol. 141 (2005): 1269-1274. [14] M.Horn, A.Gerger, S.Koller. “The Use of Confocal Laser-Scanning Microscopy in Microsurgery for Invasive Squamous Cell Carcinoma.” Br.J.Dermatol. 156 (2007): 81-84.  79  [15] L.E.Meyer, N.Otberg, W.Sterry, J.Lademann. “In vivo Confocal Scanning Laser Microscopy: Comparison of the Reflectance and Fluorescence Mode by Imaging Human Skin.” J.Biomed.Opt. 11(4) (2006): 04412. [16] S.Astner, S.Dietterle, N.Otberg. “Clinical Applicability of In vivo Fluorescence Confocal Microscopy for Noninvasive Diagnosis and Therapeutic Monitoring of Nonmelanoma Skin Cancer.” J.Biomed.Opt. 13(1) (2008): 014003. [17] P.R.Callis. “Two-photon-induced Fluorescence.” Annu.Rev.Phys.Chem. 48 (1997): 271-297. [18] W.Denk, J.H.Strickler, W.W.Webb. “Two-photon Laser Scanning Fluorescence Microscopy.” Science 248 (1990): 73-76. [19] A.Diaspro, P.Bianchini, G.Vicidomini, M.Faretta, P.Ramoino, C.Usai. “Multi-photon Excitation Microscopy.” Biomed.Eng.Online 5: 36. [20] A.Diaspro, G. Chirico, M.Collini. “Two-photon Fluorescence Excitation and Related Techniques in Biological Microscopy.” Q.Rev.Biophys. 38 (2005): 97-166. [21] M.Oheim, D.J.Michael, M.Geisbauer, D.Madsen, R.H.Chow. “Principles of Two-photon Excitation Fluorescence Microscopy and Other Nonlinear Imaging Approaches.” Adv.Drug Deliv.Rev. 58 (2006): 788-808. [22] K.Svoboda, R.Yasuda. “Principles of Two-photon Excitation Microscopy and Its Applications to Neuroscience.” Neuron 50 (2006): 823-839. [23] M.D.Cahalan, I.Parker, S.H.Wei, M.J.Miller. “Two-photon Tissue Imaging: Seeing the Immune System In A Fresh Light.” Nat.Rev.Immunol. 2 (2002): 872-880. [24] F.Helmchen, W.Denk. “Deep Tissue Two-photon Microscopy.” Nature Methods 2 (2005): 932-940. [25] V.E.Centonze, J.G.White. “Multiphoton Excitation Provides Optical Sections From Deeper Within Scattering Specimens than Confocal Imaging.” Biophys.J. 75 (1998): 2015-2024. [26] M.J.Levene, D.A.Dombeck, K.A.Kasischke, R.P.Molloy, W.W.Webb. “In vivo Multiphoton Microscopy of Deep Brain Tissue.” J.Neurophysiol. 91 (2004): 1908-1912. [27] E.Beaurepaire, M.Oheim, J.Mertz. “Ultra-deep Two-photon Fluorescence Excitation in Turbid Media.” Opt.Comm. 188 (2001): 25-29. [28] S.P.Schilders, M.Gu. “Limiting Factors on Image Quality in Imaging through Turbid Media under Single-photon and Two-photon Excitation.” Microsc.Microanal. 6 (2000): 156-160. [29] F.Helmchen, S.M.Fee, D.W.Tank, W.Denk. “A Miniature Head-mounted Two-photon Microscope. High-resolution Brain Imaging in Freely Moving Animals.” Neuron 31 (2001): 903912.  80  [30] Y.Sako, A.Sekihata, Y.Yanagisawa. “Comparison of Two-photon Excitation Laser Scanning Microscopy with UV-Confocal Laser Scanning Microscopy in Three-dimensional Calcium Imaging Using the Fluorescence Indicator Indo-1.” J. of Microscopy 185 (1997): 9-20. [31] J.Paoli, M.Smedh, A.M.Wennberg, M.B.Ericson. “Multiphoton Laser Scanning Microscopy on non-melanoma skin cancer: morphologic features for future non-invasive diagnostics.” J.Invest.Dermatol. 17 (2008): 519-523. [32] N.Kollias, G.Zonios, G.N.Stamatas. “Fluorescence Spectroscopy of Skin.” Vib.Spectrosc. 28 (2002): 17-23. [33] S.J.Lin, R.Jr.Wu, H.Y.Tan, W.Lo. “Evaluating Cutaneous Photoaging by Use of Multiphoton Fluorescence and Second-Harmonic Generation Microscopy.” Opt.Lett. 30 (2005): 2275-2277. [34] M.C.Skala, J.M.Squirrell, K.M.Vrotsos, J.C.Eickhoff. “Multiphoton Microscopy of Endogenous Fluorescence Differentiates Normal, Precancerous, and Cancerous Squamous Epithelial Tissues.” Cancer Res. 65 (2005): 1180-1186. [35] S.J.Lin, S.H.Jee, C.J.Kuo. “Discrimination of Basal Cell Carcinoma from Normal Dermal Stroma by Quantitative Multiphoton Imaging.” Opt.Lett. 31 (2006): 2756-2758. [36] E.Dimitrow, M.Ziemer, M.J.Koehler. “Sensitivity and Specificity of Multiphoton Laser Tomography for in vivo and ex vivo Diagnosis of Malignant melanoma.” J.Invest.Dermatol. 129 (2009): 1752-1758. [37] K.Konig. “Clinical Multiphoton Tomography.” J.Biophoton. 1 (2008): 13-23. [38] M.B.Ericson, C.Simonsson, S.Guldbrand. “[38]Two-photon Laser-scanning Fluorescence Microscopy Applied for Studies of Human Skin.” J.Biophoton. 1 (2008): 320-330. [39] P.J.Campagnola. “Three-Dimensional High-Resolution Second-Harmonic Generation Imaging of Endogenous Structural Proteins in Biological Tissues.” Biophys.J. 82 (2002): 493-508. [40] R.Cicchi, D.Massi, S.Sestini. “Multidimensional Non-linear Laser Scanning of Basal Cell Carcinoma.” Opt.Exp. 15 (2007): 10135-10148. [41] R.Cicchi, S.Sestini, V.De Giorgi. “Non-linear Laser Imaging of Skin Lesions.” J.Biophotonics 1 (2008): 62-73. [42] P.J.Campagnola, C.Y.Dong. “Second Harmonic Generation Microscopy: Principles and Applications to Disease Diagnosis.” Laser & Photon. Rev., 2009: 1-15. [43] N.Kollias, R.Gilles, M.Moran. “Endogenous Skin Fluorescence Includes Bands That May Serve as Quantitative Markers of Aging and Photoaging.” J.Invest.Dermatol. 111 (1998): 776780. [44] A.G.Doukas, N.S.Soukos, S.Babusis. “Fluorescence Excitation Spectroscopy for the Measurement of Epidermal Proliferation.” Photochem.Phobiol. 74 (2001): 96-102. 81  [45] L.Brancaleon, A.J.Durkin, J.H.Tu. “In vivo Fluorescence Spectroscopy of Nonmelanoma Skin Cancer.” Photochem. Phobiol. 73 (2000): 178-183. [46] H.Zeng, C.MacAulay, D.I.McLean, B.Palcic. “Spectroscopic and Microscopic Characteristics of Human Skin Autofluorescence Emission.” Photochem. Photobiol. 61 (1995): 639-645. [47] H.Zeng, D.I.McLean, C.MacAulay, H.Lui. “Autofluorescence Properties of Skin and Applications in Dermatology.” Proc. SPIE 4224 (2000): 366-373. [48] N.Kollias, G.N.Stamatas. “Optical Non-invasive Approaches to Diagnosis of Skin Diseases.” J.Invest.Dermatol. (Symposium Proceedings) 7 (2002): 64-75. [49] N.Kollias, G.Zonios, G.N.Stamatas. “Fluorescence Spectroscopy of Skin.” Vibrational Spectroscopy 28 (2001): 17-23. [50] J.Chen, S.Zhuo, T.Luo, X.Jiang, J.Zhao. “Spectral Characteristics of Autofluorescence and Second Harmonic Generation from ex vivo Human Skin Induced by Femtosecond Laser and Visible Lasers.” Scanning 28 (2006): 319-326. [51] B.R.Masters, P.T.C.So, E.Gratton. “Multiphoton Excitation Fluorescence Microscopy and Spectroscopy of in vivo Human Skin.” Biophys.J. 72 (1997): 2405-2412. [52] J.Chen, S.Zhuo, T.Luo, J.Zhao. “Two-photon Excited Spectroscopies of ex vivo Human Skin Endogenous Species Irradiated by Femtosecond Laser Pulses.” Chinese Opt. Lett. 4 (2006): 598600. [53] G.Chen, J.Chen, S.Zhuo. “Nonlinear Spectral Imaging of Human Hypertrophic Scar Based on Two-photon Excited Fluorescence and Second-Harmonic Generation.” British J. of Dermatol. 161 (2009): 48-55. [54] L.H.Laiho, S.Pelet, T.M.Hancewicz, P.D.Kaplan, P.T.C.So. “Two-photon 3-D Mapping of ex vivo Human Skin Endogenous Fluorescence Species Based on Fluorescence Emission Spectra.” J. of Biomed. Opt. 10(2) (2005): 024016. [55] R.Gillies, G.Zonios, R.R.Anderson. “Fluorescence Excitation Spectroscopy Provides Information about Human Skin in vivo.” J.Invest.Dermatol. 115 (2000): 704-707. [56] A.G.Doukas, N.S.Soukos, S.Babusis, Y.Appa, N.Kollias. “Fluorescene Excitation Spectroscopy for the Measurement of Epidermal Proliferation.” 74 (2001): 96-102. [57] P.Juzenas, A.Juzeniiene, O.Kaalhus, V.Iani, J.Moan. “Noninvasive Fluorescence Excitation Spectroscopy during Application of 5-Aminolevulinic Acid in vivo.” Photochem. Photobiol. Sci. 1 (2002): 745-748. [58] A.F.Zuluaga, U.Utzinger, A.J.Durkin. “Fluorescence Excitation Emission Matrix of Human Tissue: A System for in vivo Measurement and Method of Data Analysis.” Appl. Spectrosc. 53(3) (1999): 302-311. 82  [59] R.A.Zangaro, L.Silveira, R.Manoharan, G.Zonios. “Rapid Multiexcitation Fluorescence Spectroscopy System for in vivo Tissue Diagnosis.” Appl.Optics. 35 (1996): 5211-5219. [60] G.M.Palmer, C.Zhu, T.M.Breslin. “Comparison of Multiexcitation Fluorescence and Diffuse Reflectance Spectroscopy for the Diagnosis of Breast Cancer (March 2003).” IEEE Trans. Biomed. Eng. 50(11) (2003): 1233-1242. [61] R.Drezek, K.Sokolov, U.Utzinger, I.Boiko. “Understanding the Contributions of NADH and Collagen to Cervical Tissue Fluorescence Spectra: Modeling, Measurements, and Implications.” J. Biomed. Opt. 6(4) (2001): 385-396. [62] B.H.Li, S.S.Xie. “Autofluorescence Excitation-Emission Matrices for Diagnosis of Colonic Cancer.” World J. Gastroenterol. 11 (2005): 3931-3934. [63] R.S.DaCosta, H.Andersson, B.C.Wilson. “Molecular Fluorescence Excitation-Emission Matrices Relevant to Tissue Spectroscopy.” Photochem.Photobiol. 78 (2003): 384-392. [64] I.Georgakoudi, B.C.Jacobson. “NAD(P)H and Collagen as in vivo Quantitative Fluorescent Biomarkers of Epithelial Precancerous Changes.” Cancer Res. 62 (2002): 682-687. [65] P.Diagaradjane, M.A.Yaseen, J.Yu. “Autofluorescence Characterization for the Early Diagnosis of Neoplastic Changes in DMBA/TPA-Induced Mouse Skin Carcinogenesis.” Lasers Surg. Med. 37 (2005): 382-395. [66] J.Chen, A.Lee, J.Zhao. “Spectroscopic Characterization and Microscopic Imaging of Extracted and in situ Cutaneous Collagen and Elastic Tissue Components under Two-photon Excitation.” Skin Res. and Tech., 2009: 1-9. [67] J.B.Pawley. Handbook of Biological Confocal Microscopy. 2. New York: Plenum Press, 1995. [68] G.Y.Fan. “Video-rate Scanning Two-photon Excitation Fluorescence Microscopy and Ratio Imaging with Cameleons.” Biophys. J. 76 (1999): 2412-2420. [69] Q.T.Nguyen, N.Callamaras, C.Hsieh, I.Parker. “Construction of a Two-photon Microscope for Video-rate Ca(2+) Imaging.” Cell Calcium 30 (2001): 383-393. [70] M.J.Sanderson. “Acquisition of Multiple Real-Time Images for Laser Scanning Microscopy.” Microscopy and Analysis 18(4) (2004): 17-23. [71] W.R.Zipfel, R.M.Williams, W.W.Webb. “Nonlinear Magic: Multiphoton Microscopy In the Biosciences.” Nat. Biotechnol. 21 (2003): 1369-1377. [72] C.Xu, W.W.Webb. “Measurement of Two-photon Excitation Cross Sections of Molecular Fluorophores with Data from 690 to 1050 nm.” J.Opt.Soc.Am.B 13(3) (1996): 481-491. [73] S.Tang, T.B.Krasiieva, Z.Chen, G.Tempea, B.J.Tromberg. “Effect of Pulse Duration on Two-photon Excited Fluorescence and Second Harmonic Generation in Nonlinear Optical Microscopy.” J. Biomed. Opt. 11(2) (2006): 020501. 83  [74] C.Y.Dong, K.Koenig, P.So. “Characterizing Point Spread Functions of Two-photon Fluorescence Microscopy in Turbid Medium.” J. Biomed. Opt. 8(3) (2003): 450-459. [75] Z.Hu, Y.Pan, A.M.Rollins. "Analytical Model of Spectrometer-based Two-beam Spectral Interferometry." Applied Opt. 46(35) (2007): 8499-8505. [76] A.M.Pena, M.Strupler, T.Boulesteix, M.C.Schanne-Klein. “Spectroscopic Analysis of Keratin Endogenous Signal for Skin Multiphoton Microscopy.” Optics Express 13(16) (2005): 6268-6274. [77] S.Huang, A.A.Heikal, W.W.Webb. “Two-Photon Fluorescence Spectroscopy and Microscopy of NAD(P)H and Flavoprotein.” Biophys. J. 82 (2002): 2811-2825. [78] K.Teuchner, J.Ehlert, W.Freyer. “Fluorescence Studies of Melanin by Stepwise Two-Photon Femtosecond Laser Excitation.” J. of Fluoresc. 10(3) (2000): 275-281.  84  Appendices Appendix A: Details of driving boards for the X-Y scanners  Table A.1 – CRS Driver Board Connection Pins  Pin  Signal  Pin  J1 8KHz Scanner Connector  Signal J3 Main Connector  1  Ground  1  Zoom High (0 - 5 V)  2  Velocity Coil Start  2  Zoom Return  3  Velocity Coil Return  3  Signal Ground, Digital  4  Drive Coil Return  4  Relay Shutoff  5  Drive Coil Start  5  SEL 0  6  SEL 1  7  Analog Ground  J2 Aux. Connector 1,2,3  +5Vdc Output  8  CRS Sync  4  TTL Sync. Pulse  9,10  +5Vdc  5,6,8  Digital Ground  11,12  Analog Ground  7,9,10  Analog Ground  13,14  +15Vdc  11,12  -12Vdc Output  15,16  -15Vdc  13,14  +12Vdc Output  85  Power J1 Scanner Motor J4 Control Signal J2  Figure A.1 MiniSAX II driver board electronic connections  86  Table A.2 – MiniSAX II Driver Board Connection Pins  Power Connector Pin  Function  Mating Connector  Panduit: CE100F22-4-D  1  Reserved  Strain Relief  Panduit: SCC100F-4-D  2  Supply + (+15 to +24 V)  Assembly Tool  Panduit: MRT-100F  3  Supply Ground  Specified Wire Size  AWG#22  4  Supply - (-15 to -24 V)  Signal Interface Connector Pin  Function  Mating Connector  Panduit: CE100F22-8-D  1  Command +  Strain Relief  Panduit: SCC100F-8-D  2  Command -  Assembly Tool  Panduit: MRT-100F  3  Ground  Specified Wire Size  AWG#22  4  Temperature Status  5  Servo Enable  6  Servo Ready  7  Scanner Position +  8  Scanner Position -  87  Appendix B: Labview Program for Signal Synchronization  Output to Part 3  Input from Part 2  Output to Part 4  Figure A.2 Analog output channel setup  88  Output to Part 1  Output to Part 3  Output to Part 4 Figure A.3 Analog output waveform initialization  89  Input from Part 1  Figure A.4 Writing loop for analog output and Stop of writing  90  Input from Part 1  Input from Part 2  Figure A.5 Digital Output  91  Appendix C: Automated EEM Acquisition Program I use time frame structure to control the operation flow. There are three main time frames in addition to the Initialization and Data Writing Part. Laser Initialization  Output to Frame 1  To Set Laser Wavelength Tuning Pattern Data Array Initialization  Output to Frame 1 Output to Unit 2 Loop  Rotation Motor Initialization Loading Angle Array for Excitation Power Calibration  Output to Frame 1  Figure A.6 Imaging System Initialization  92  Output to Frame 3  Figure A.7 Spectrometer Initialization  93  Input from Unit 1  Loop Structure for Unit 2  Frame 1 – Set laser wavelength  Input from Unit 1  Output to Frame 2 and 3 Figure A.8 Loop of Spectrum Acquisition (Frame 1)  94  Loop Structure for Unit 2  Frame 2 – Set the Angle of Rotation Motor for Power Calibration  Input from Frame 1  Input from Loop parameter  Output to Unit 3  Output to Frame 3  Figure A.9 Loop of Spectrum Acquisition (Frame 2)  95  Loop Structure for Unit 2  Frame 2 – Measure a group of Emission Spectra for one excitation wavelength  Input from Loop Parameter  Output to Unit 3  Input from Unit 1  Input from Frame 2  Figure A.10 Loop of Spectrum Acquisition (Frame 3)  96  Stop Motor Control Stop Communication with Laser  Input from Unit 2  Stop Spectrometer Acquisition Write Averaged Spectra to .TXT file  Input from Unit 2  Figure A.11 Stop Each Module and Write the Average Emission Spectra to a .TXT File  97  Appendix D: Program code for real-time imaging processing and display The code is for the first channel. In practice, four channels can run simultaneously with the same functions by using similar code as the first one. The differences between the codes for different channels are the variable names representing each channel. #include #include #include #include #include #include #include #include #include #include  "stdafx.h" "CiView.h" "CiViewDoc.h" "CiViewView.h" <math.h> "BiApi.h" "DSapi.h" "BFErApi.h" // DoAboutDialog <cmath> "BFTabRegister.h" // for chaning the register value  #ifdef _DEBUG #define new DEBUG_NEW #undef THIS_FILE static char THIS_FILE[] = __FILE__; #endif extern extern extern extern  Bd hBoard_1; BFU32 g_RTP_1; BFU32 openbrd1; BFU32 g_AqState_1;  //BFU32 g_LastLine_RTP; //double g_fps_RTP; //double g_dfps_RTP; int hDspSrf_RTP_1 = -1; int hDspSrf_RTP_temp_1 = -1; BFU32 blank1 = 2; BFU32 g_blackline_1 = 0; BFU32 NumBuffers_1 = 30; // Number of buffers that were allocated. BFBOOL BrdIsSetup_1; // TRUE <=> Setup has been called BIBA pBufStruc_1; // Holds the array of buffers and other bufin info BIBA pBufStruc_temp_1; PBFVOID pBitmap_RTP_1;// pointer to buffer of image data for display PBFVOID pBitmap_RTP_temp_1; CWinThread* pProcessingThread_1 = NULL; //Thread Handler UINT ProcessingThread_1(LPVOID lpdwParam); //LOCAL Prototype BFBOOL StartProcessing_1 () { BFU32 xsize,ysize,pixdepth; BFRC Error = BI_OK; BFRC Error_temp = BI_OK; //allocate memory for the array of buffer pointers Error = BiBufferAllocCam (hBoard_1, &pBufStruc_1, NumBuffers_1); if (Error != BI_OK) { BiErrorShow(hBoard_1,Error);  98  return FALSE; } //~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ temp buffer Error = BiBufferAllocCam (hBoard_1, &pBufStruc_temp_1, NumBuffers_1); if (Error != BI_OK) { BiErrorShow(hBoard_1,Error); return FALSE; } // find out camera info */ BiBrdInquire(hBoard_1, BiCamInqXSize, &xsize); BiBrdInquire(hBoard_1, BiCamInqYSize0, &ysize); BiBrdInquire(hBoard_1, BiCamInqBitsPerPixDisplay, &pixdepth); //DISPLAY SURFACE // create display surface to view what is in bitmap in buffers if(!DispSurfCreate((PBFS32)&hDspSrf_RTP_1,xsize,ysize,pixdepth,Af xGetApp()->m_pMainWnd->GetSafeHwnd())) { AfxMessageBox("Could not open display surface.",MB_OK|MB_ICONINFORMATION); return FALSE; } // get pointer to bitmap data memory */ if(!DispSurfGetBitmap(hDspSrf_RTP_1,&pBitmap_RTP_1)) { AfxMessageBox("No display surface available ",MB_OK|MB_ICONEXCLAMATION); return FALSE; } // offset Display surface */ DispSurfOffset(hDspSrf_RTP_1,50,100); // Give window a title char st[100]; BFsprintf(st,sizeof(st),"RTP Window - Board %d",openbrd1); DispSurfTitle(hDspSrf_RTP_1,st); // create display surface to view bitmap in buffers */ if(!DispSurfCreate((PBFS32)&hDspSrf_RTP_temp_1,xsize,ysize,pixdep th,AfxGetApp()->m_pMainWnd->GetSafeHwnd())) { AfxMessageBox("Could not open display surface.",MB_OK|MB_ICONINFORMATION); return FALSE; } // get pointer to bitmap data memory */ if(!DispSurfGetBitmap(hDspSrf_RTP_temp_1,&pBitmap_RTP_temp_1)) { AfxMessageBox("No display surface available ",MB_OK|MB_ICONEXCLAMATION); return FALSE; } // offset Display surface */ DispSurfOffset(hDspSrf_RTP_temp_1,600,100); // Give window a title char st_temp[100]; BFsprintf(st_temp,sizeof(st_temp),"Original Preview Window Board %d",openbrd1);  99  DispSurfTitle(hDspSrf_RTP_temp_1,st_temp); // SETUP THE BOARD FOR CIRCULAR ACQUISITION Error = BiCircAqSetup(hBoard_1, &pBufStruc_1, CirErIgnore, BiAqEngJ); if(Error) { BiErrorShow(hBoard_1,Error); return FALSE; } else { BrdIsSetup_1 = TRUE; } Error = BiCircAqSetup(hBoard_1, &pBufStruc_temp_1, CirErIgnore, BiAqEngJ); if(Error) { BiErrorShow(hBoard_1,Error); return FALSE; } else { BrdIsSetup_1 = TRUE; } //Create processing thread////////////////////////////////////// pProcessingThread_1 = AfxBeginThread(ProcessingThread_1, (LPVOID)NULL, THREAD_PRIORITY_HIGHEST); // start continuous acquisition Error = BiCirControl(hBoard_1, &pBufStruc_1, BISTART, BiWait); if(Error) BiErrorShow(hBoard_1, Error); Error = BiCirControl(hBoard_1, &pBufStruc_temp_1, BISTART, BiWait); if(Error) BiErrorShow(hBoard_1, Error); } // PROCESSING THREAD UINT ProcessingThread_1(LPVOID lpdwParam) { BFRC BFRC BOOL BiCirHandle BiCirHandle PBIBA PBIBA BFU32 in lines BFU32 BFU32 BFU32 bytes BFU32 PBFU8 PBFU8  Error = BI_OK; Error_temp = BI_OK; KeepLooping; CirHandle; CirHandle_temp; BufStruc = &pBufStruc_1; BufStruc_temp = &pBufStruc_temp_1; xsize,ysize; //width in pixels and height BytesPerPix; bitdepth; imagesize;  //bytes per pixel //bits per pixel //total size of image in  timeout; lp8buf,lp8bufend; lp8buf_temp,lp8bufend_temp;  100  int double long bool  i,Diff; pi=3.14159, ph, c; uxsize,shift_uxsize; g_display = true;  BiBrdInquire(hBoard_1, BiBrdInquire(hBoard_1, BiBrdInquire(hBoard_1, BiBrdInquire(hBoard_1, BiBrdInquire(hBoard_1, BiBrdInquire(hBoard_1,  BiCamInqBitsPerPix, &bitdepth); BiCamInqFrameSize0, &imagesize); BiCamInqXSize, &xsize); BiCamInqYSize0, &ysize); BiCamInqBytesPerPix, &BytesPerPix); BiCamInqAqTimeout, &timeout);  uxsize = (long)(xsize); shift_uxsize = (long)(uxsize/2 - blank1); KeepLooping = TRUE; while(KeepLooping) { // Wait for frame to be cached into the buffer Error = BiCirWaitDoneFrame(hBoard_1, BufStruc_temp, timeout, &CirHandle); Error_temp = BiCirWaitDoneFrame(hBoard_1, BufStruc_temp, timeout, &CirHandle_temp); if((Error != BI_OK) || (Error_temp != BI_OK) ) { if(Error < BI_WARNINGS) { StopProcessing_1(); BiErrorShow(hBoard_1, Error); } KeepLooping = FALSE; } if ((Error == BI_OK )&&( Error_temp == BI_OK)) { if(bitdepth == 8) { lp8buf = (BFU8*)CirHandle.pBufData; lp8bufend = (BFU8*)CirHandle.pBufData + imagesize; lp8buf_temp = (BFU8*)CirHandle_temp.pBufData; lp8bufend_temp = (BFU8*)CirHandle_temp.pBufData+imagesize; i=0; while ((lp8buf+i) <= lp8bufend ) { //calculate the new position for each pixel if ((i%uxsize)<shift_uxsize) { ph = pi/shift_uxsize*(i%uxsize-shift_uxsize/2); c = ph/sin(ph); Diff = (int)(((float)(i%uxsize-shift_uxsize/2))*(c-1)); } if ((i%uxsize+Diff)<shift_uxsize && (i%uxsize+Diff)> 0) { *(lp8buf+i)=*(lp8buf_temp+i+Diff); }  101  else { *(lp8buf+i) = *(lp8buf+i)*0; } if ((g_blackline_1 == 0)&&(i%uxsize == (shift_uxsize/2))) { *(lp8buf+i) = *(lp8buf_temp+i); } i++; }  } // now update display & clean up after closing display surface if ((!DispSurfBlit(hDspSrf_RTP_1))||(!DispSurfBlit(hDspSrf_RTP_temp_1))) { StopProcessing_1(); KeepLooping = FALSE; return 0; } //copy data from the temp buffer to the buffer for display surface if (g_display) { memcpy(pBitmap_RTP_1,lp8buf, imagesize); memcpy(pBitmap_RTP_temp_1,lp8buf_temp, imagesize); }  // Set the buffer to AVAILABLE after processing Error = BiCirStatusSet(hBoard_1, BufStruc, CirHandle, BIAVAILABLE); if(Error) { BiErrorShow(hBoard_1, Error); //////if error, then stop the processing //pView->OnProcessingStop(); return 0; } Error = BiCirStatusSet(hBoard_1, BufStruc_temp, CirHandle_temp, BIAVAILABLE); if(Error) { BiErrorShow(hBoard_1, Error); return 0; } } } return 0; } void StopProcessing_1()  102  { BFRC Error = BI_OK; BFRC Error_temp = BI_OK; if (CiAqCommand(hBoard_1,CiConGrab,CiConAsync,CiQTabBank0,AqEngJ)) { } g_AqState_1 = CiConGrab; // Clean things up Error = BiCircCleanUp(hBoard_1, &pBufStruc_1); if(Error != BI_OK) BiErrorShow(hBoard_1,Error); Error = BiCircCleanUp(hBoard_1, &pBufStruc_temp_1); if(Error != BI_OK) BiErrorShow(hBoard_1,Error); //Close Display window DispSurfClose(hDspSrf_RTP_1); DispSurfClose(hDspSrf_RTP_temp_1); // free memory Error = BiBufferFree(hBoard_1, &pBufStruc_1); if(Error != BI_OK) BiErrorShow(hBoard_1,Error); Error = BiBufferFree(hBoard_1, &pBufStruc_temp_1); if(Error != BI_OK) BiErrorShow(hBoard_1,Error); }  103  Appendix E: Attenuator Box Design  Figure A.12 Attenuator electronic design  104  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0071505/manifest

Comment

Related Items