Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Computer-controlled microscope for the automatic classification of white blood cells. Gabert, Howard Frederick 1973

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


UBC_1973_A7 G32_4.pdf [ 4.62MB ]
JSON: 1.0101427.json
JSON-LD: 1.0101427+ld.json
RDF/XML (Pretty): 1.0101427.xml
RDF/JSON: 1.0101427+rdf.json
Turtle: 1.0101427+rdf-turtle.txt
N-Triples: 1.0101427+rdf-ntriples.txt
Original Record: 1.0101427 +original-record.json
Full Text

Full Text

A COMPUTER-CONTROLLED MICROSCOPE FOR THE AUTOMATIC CLASSIFICATION OF WHITE BLOOD CELLS by Howard Frederick Gabert B.A.Sc, The University of British Columbia, 1970 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in the Department of Electrical Engineering We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA July 1973 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the Head of my Department or by his representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver 8, Canada ABSTRACT A microscope was interfaced to a PDP-9 computer in order to develop techniques suitable for on-line classification of white blood cells. The computer visual-input system is composed of an image dis sector optically coupled to a microscope used in a transmitted light mode. The position of the slide under the microscope and the fine focus control are controllable from the computer. A technique of auto-focusing was developed to efficiently focus the microscope under computer control.- This algorithm is described in detail, followed by a discussion of the physical factors that affect the performance of this technique. A method of locating or isolating the leukocytes (white blood cells) is described next. A constant, referred to as the "contrast ratio' is used to extract the threshold for the nuclei of the leukocytes based on the average background intensity. Finally, contour tracing and curvature function extraction are used as a means of testing the system. A specific test is conducted to obtain a comparison of the system's efficiency as compared to that of "manual" techniques used by a technician. The system described here is not only suitable for automatic leukocyte classification, but could also be used for many other routine tests requiring the examination of microscopic cells. TABLE OF CONTENTS Page ABSTRACT i TABLE OF CONTENTS ii LIST OF ILLUSTRATIONS . iv LIST OF TABLES . • vACKNOWLEDGEMENT vii 1. INTRODUCTION 1 1.1 Clinical Procedure for the Differential Leukocyte Count 3 1.2 Characteristics of Leukocytes 4 1.3 System Design Objectives .... 6 1.4 Other Microscopic Image Processing Systems 7 2. THE HARDWARE 8 2.1 The Computer Visual-Input System 8 2.2 The Image Dissector 8 2.3 The Microscope 13 2.4 The Computer Interface 12.5 The System Resolution 6 3. DEVELOPMENT OF THE AUTO-FOCUS ALGORITHM 22 3.1 Background 23.2 Developing the Algorithm 23 3.3 Factors Affecting the Variance Function 28 4. THE LEUKOCYTE LOCATION PROCEDURE . . . 35 4.1 The Conditions of the Search4.2 Threshold Determination 36 4.3 The Scan Pattern 8 5. THE SOFTWARE IMPLEMENTATION 41 5.1 "FOCUS" . 45.2 "THRESH" 3 5.3 "JOYSTK" 5 5.4 "STRPNT" 4ii Page 6. SYSTEM EVALUATION • 49 6.1 The Test System .• • • 49 6.2 Contour Tracing 50 6.3 The Discrete Area Operator 51 6.4 Reliability and Execution Time Measurements 53 7. CONCLUSIONS 58 APPENDIX I 60 APPENDIX II 62 APPENDIX III . 6 REFERENCES • 6? iii LIST OF ILLUSTRATIONS Figure Page 1.1.1 Sketch of a normal blood film 3 1.2.1 Sketches of typical leukocytes 4 2.1.1 Photograph of the microscope and hardware 9 2.2.1 Interpretation of percentage modulation. 100% modulation at GS = 5 is shown in (A) and 86.5% modu lation at GS = 4 is shown in (B) 10 2.2.2 Signal to noise ratio of the image dissector as a function of At, the dwell time 12 2.3.1 Block diagram of the optical system 14 2.4.1 Block diagram of the computer interface 15 2.5.1 Resolution as a function of magnification for the image dissector (A) and the microscope (B) 19 2.5.2 System resolution as a function of magnification ... 20 3.2.1 A video scan line when in focus (A) and when out of focus (B) 24 3.2.2 Characteristics searched for in a video scan line ... 25 3.2.3 The variance plotted against focus control position unfiltered (A) and filtered (B) with a digital filter window size of ten 26 3.3.1 The variance function for two different fields of view showing peak variance scaling 28 3.3.2 Scaling of the variance function due to magnification shown at 800X (A), 1024X (B), 1280X (C), and 2000X (D). 29 3.3.3 The effect of no filter (A), a red filter (B), a blue filter (C), and a green filter (D) on the variance function extracted from an image of cells . . 31 3.3.4 The variance function extracted at full intensity and at half intensity. The bottom curve is the one at half intensity 32 3.3.5 The effect of non-critical illumination on the variance function at a magnification of 2560X. The top curve represents critical illumination 33 iv Figure Page 4.1.1 Spectral density curve of the Zeiss 46 78 06 wide-band pass green interference filter 35 4.1.2 Video scan lines across a leukocyte with (B) and without (A) contrast enhancing filters 36 4.3.1 Standard scan patterns used in the differential leukocyte count. The straight-edge pattern (A), the battlement pattern (B), the cross-sectional pattern (C) , and the longitudinal pattern (D) 39 4.3.2 Placement of adjacent scan areas 34.3.3 Relationship between current and adjacent scan areas, showing the placement of the "dead" zone 40 5.1.1 Flow diagram of the focusing program, "FOCUS" 42 5.1.2 The scan lines used to sample the variance 43 5.2.1 Flow diagram of the threshold determining program, "THRESH" 44 5.4.1 Flow diagram of "STRPNT" 46 5.4.2 The sequential search technique used to locate leukocytes, demonstrating the minimum square fit (A) of size AGS, and the location of the starting point (B) by using GS 47 6.2.1 The contour tracing algorithm. The black dots are spaced at GS and represent the four points that are interrogated to determine which direction (the circles) the operator should be moved .50 6.3.1 The discrete area operator used to extract boundary curvature 51 6.3.2 Photographs of several cells, their contour traces, and unfiltered curvature functions as extracted by the area operator 2 6.4.1 The scan pattern used to measure the time required to locate and process 100 cells 54 A.2.1 A block diagram demonstrating a possible arrangement of the system programs 63 v LIST OF TABLES Table Page 1.2.1 Morphological and spectral characteristics of leukocytes 5 2.5.1 Results of the calibration of the image dissector resolution 17 6.4.1 Results of a test designed to count leukocytes across an area with a true count of 100 cells 54 6.4.2 Times required to locate and process 100 leukocytes on ten randomly selected blood films 55 A. 1.1 The optimum values for the six magnification dependent parameters 60 A.2.1 Internal and external references to global symbols and subroutines. "X" indicates internally defined, "A" indicates external access necessary, and "(A)" indicates external access probable 63 vi ACKNOWLEDGEMENT I wish to extend my sincere thanks to my supervisor, Dr. J.S. MacDonald, for his guidance and encouragement during my period of study, and to Dr. R. Pearce, of the Department of Pathology, U.B.C, for his advice concerning the medical background of this project. I would also like to gratefully acknowledge the assistance of the National Research Council of Canada for its financial support through scholarships and the major equipment grant used to purchase the micro scope . Thanks are also owing Mr. A. Leugner for his expert technical guidance during construction of the hardware, and to Miss N. Duggan for typing the final manuscript. And finally, I must give credit to my wife, Nadine, who, through her continued encouragement and understanding, has made it possible for me to complete this work. vii 1. INTRODUCTION The biomedical sciences characteristically deal with large volumes of data, which must be collected, organized, reduced, analyzed, and generally processed in many ways. In particular, the analysis of biological specimens containing cells is known to be important for the medical diagnosis of many diseases and chemical disorders in the human body. Standard medical tests for the differential diagnosis of disease and therapy are required to be accurate, economical and efficient. In most cases, these tests require a qualified and experienced technician to examine the biological specimens under a high power microscope. This type of visual examination is subjective and no longer essential consider ing the techniques developed in the field of pattern recognition. A situation such as this suggests that automation of some of these routine tests should be attempted. The differential white blood cell count is one procedure that invites automation. It has been estimated that this routine test involves manual counts on over 240,000 slides per day in the United States alone (1). In July 1972, Technicon International of Canada Limited announced TM TM the Hemalog D (2). The Hemalog D is designed to perform the differ ential white blood cell count by incorporating cytochemistry into con tinuous-flow equipment. Cytochemistry may be defined as the identification of specific components within the confines of individual cells by stain-TM ing. In the Hemalog D , the liquid blood sample is divided into several portions and each portion is treated with a different stain to identify different constituents of the sample. Each portion then passes through a view chamber where the stained cells are sensed by photo-sensors 1 2 which measure two variables, light loss and light scattering. By means TM of thresholding the light loss and light scattering, the Hemalog D classifies and counts the white blood cells in the sample. The Hemalog TM D is currently undergoing clinical environment testing and evaluation. Some disadvantages of this system seem apparent. The Hemalog TM D is not modelled after the normal procedure used by the technicians in the clincial laboratory. The initial differential white blood cell count is usually a screening procedure, and if abnormalities are detected, a qualified hematologist is invited to visually examine the specimen. TM With the Hemalog D there is no means for visual examination and there fore extra expense and time are required to prepare part of the fluid specimen for such examination. Cross-contamination between consecutive specimens in the fluid channels will need evaluation in the clinical environment, and will probably be related to the quality and frequency of maintenance. At least two additional systems are currently under development. These are the Perkin-Elmer Instrument (16) and the Corning Larc System TM (17). In contrast to the Hemalog D , both of these systems reportedly use visual classification techniques to perform the differential white blood count, although the methods used by these systems have not yet been published. One or both of these systems are expected to be commercially available soon. The primary context in the design of the system to be reported here has been the differential leukocyte count. However, due to the analogous properties of the system to that of the current clinical ap proach, the hardware and software control protion of this system is appli cable to many other routine tasks other than the differential leukocyte 3 count. 1.1 Clinical Procedure for the Differential Leukocyte Count An understanding of the normal clinical procedure used to per form a differential count is helpful when automating this test. Approxi-3 mately 10 mm of blood taken from a patient is smeared on a glass slide and stained with Wright's blood stain (Fig. 1.1.1). This prepared slide is then mounted under a microscope to be examined by a trained labora tory technician. Commonly 1000X magnification and white light, oil-immersion microscopy is used. At this magnification, the three principal cellular components of blood are visible: the leukocytes (white blood cells), the erythrocytes (red blood cells) , and the thrombocytes (plate lets). With the aid of a mechanical stage, the technician controls the position of the slide, identifying and classifying each leukocyte en countered. A total count for each category of leukocytes is simultan eously recorded. Normal procedure calls for the identification and clas sification of only the first one hundred leukocytes encountered. Studies have shown that on a test slide with a true proportion of 50% neutrophils, a one hundred cell count produces an expected mean measure of neutrophils HEAD TAIL KLlm Too Thick Ideal Thickness Him Too Thin Fig. 1.1.1 Sketch of a normal blood film. 4 from 40% to 60%, a two hundred cell count from 43% to 57%, and a five hundred cell count from 45.6% to 54.4% (3). This suggests that a large number of cells, perhaps five hundred or one thousand, should be scruti nized to obtain an accurate differential count. A normal one hundred cell count takes a technicain from five to ten minutes. 1.2 Characteristics of Leukocytes The primary classifications of leukocytes may be divided into two categories, granular and nongranular. The granular category includes eosinophils, neutrophils, and basophils. The nongranular category Fig. 1.2.1 Sketches of typical leukocytes, includes lymphocytes and monocytes. Size, colour, nuclear shape, and percentage population for each of these classifications are summarized in Table 1.2.1. Sketches of typical leukocyte specimens are shown in Figure 1.2.1. Many schemes have been derived to classify leukocytes Cell Type Lymphocyte Monocyte Neutrophil Eosinophil Basophil Population (?) Dianister (microns) Cytoplasm Nucleus Granules Color Amount Color Shape Amount Aeount 25-33 7-12 Clear Light Blue 10-30? Purple Round 70-90? Few 2-6 13-20 Grayish Blue 50? Light Purple Indented 50? Few 60-70 10-12 Pink to Lilac 50? Reddish Purple Segmented 50? Many 1-4 10-14 Rad Stippling 60-70? Dark Blue Tuo-Lobsd 30-40? Many • i i .25-.S 10-12 Blue 5C? Purple Elongated 50? Many Table 1.2.1 Morphological and spectral characteristics of leukocytes. 6 by using some combination of the following properties: cytoplasmic colour, shape and size, nuclear colour, shape and size, and concentration of granules. Colour is the only property that cannot be measured by the' system developed here. Colour identification is generally expensive to implement and slow to evaluate, especially when the colours lie spec trally adjacent and are not too easily defined. In the case of a blood smear, colour is affected by the composition and pH of the stain used, the staining time, and the thickness of the smear. 1.3 System Design Objectives The objective has been to design and develop a computer control led system capable of locating microscopic objects and measuring such properties as size and shape of these objects. All techniques and al gorithms have been designed to operate "on-line", that is, by using the image as "read-only" memory. Such a method seems desirable for two reasons: computer memory size is reduced since the image information need not be stored in memory, and access time of random picture-point information is generally faster than with either a disc or drum memory. The performance objective has been maximum speed, without sacrificing resolution. Dynamic control of the microscope has been an essential part of this performance objective. Application of the system to the differential leukocyte count has been partially implemented in order to evaluate system efficiency, as compared to normal "manual" techniques used by a technician. However, the differential leukocyte count is not the only suitable application for the system. Many routine microscopy techniques could be automated, and the system developed provides a good basis for the dynamic control of a microscope. Histological or density per area measurements could 7 easily be implemented with this system. 1.4 Other Microscopic-Image Processing Systems Publications from all sciences have included articles descri bing hardware and software techniques for image processing. All the systems cannot be enumerated here, but none of them are on-line integrated systems offering speed comparable to a technician performing the same task. Many systems substitute a physical memory device to store the in^ formation of the image: SCAD (4) scans coloured photographs of cell images and stores the resulting digital data on magnetic tape for sub sequent processing, CYTOSCAN (5) scans individual cells through a micro scope and stores the data on punched paper or magnetic tape, FIDAC (6) scans pictures of cell images and transfers the data directly to a com puter in block form, CYDAC (7) scans a microscopic image and transfers data to magnetic tape, and-SPECTRE II (7) provides point addressable in formation from a microscopic image. SPECTRE II is the closest in con cept to the system developed here, although SPECTRE II does not provide dynamic or "hands-off" control of the microscope. SPECTRE II is also exceptionally slow, requiring 120 seconds for a full raster scan of 256 by 256 points. Any further comparison to these systems can be made by checking the references. 8 2. THE HARDWARE 2.1 The Computer Visual-Input System The computer visual-input system is composed of an I.T.T. Vidissector camera and a Zeiss Universal microscope (Figure 2.1.1). Resolution of the system becomes a function of the resolution of both the image dissector and the microscope. A primary consideration in the design was to ensure that the resolution of the system was variable and would always be capable of resolving pertinent image detail. It should be noted that any cell recognition scheme permitting a decrease in resol ution will probably offer an increase in speed during data acquisition and microscope control. 2.2 The Image Dissector The image dissector is nothing more than a photomultiplier with a sensitive and electronically movable photocathode area. The optics are adjusted such that an image is formed on the photocathode, which emits electrons proportional to the intensity at any given point. These electrons are accelerated and focused onto the dissecting aperture plane forming an electron image which is current density modulated ac cording to the optical input intensity pattern. This electron image may be deflected electronically across the aperture, such that at any instant the photoelectrons from a small, well-defined area of the image are incident on the aperture. These electrons enter a photomultiplier and emerge as a current in the output anode circuit. The image dissec tor can be visualized as a means of randomly accessing point intensity information from the image. The digital control used on the dissector is well documented Figure 2.1.1 Photograph of the microscope and hardware. 10 SP 50 . -10 -5 0 5 10 Dlotanco From Boundary (Units of GS) -12-8-4 0 K 8 12 Distance From Boundary (Units of CS) Fig. 2.2.1 Interpretation of percentage modulation. 100% modulation at GS = 5 is shown in (A) and 86.5% modulation at GS = 4 is shown in (B). (8). With the deflection control provided, it is possible to randomly access or address any one of 1024 by 1024 points on the image. The dis sector is currently equipped with a tube having a .005 inch round de-fining-aperture. To achieve 100% modulation between adjacent points across the one inch square surface of the tube, approximately 200 by 200 points are unique, represented by an incremental step size (GS) of 11 5 addressing units (Fig. 2.2.1). However, to increase resolution, a step size of 4 is assumed to be the minimum, thereby effectively reducing the depth of modulation to 86.5%, and creating a grid of 256 by 256 addressable points. Therefore a maximum of 65,536 points on the image will be con sidered unique. The number of grey levels of intensity that can be resolved is dependent on the signal to noise ratio of the video signal. As the sampling frequency increases, the signal to noise ratio decreases (9,10) The signal to noise ratio is given by the following equation: g = 1.2 X 109 /J a 2At (2.2.1) Nrms where J = the average photocathode current density a = the aperture area At = the element dwell time before interrogation. 2 -4 2 For the dissector tube in use, J is 1 ua/cm and a is 1.26 X 10 cm . The results of this equation are plotted in Figure 2.2.2. To obtain a desired signal to noise ratio, the total delay time preceding interroga tion is a function of three quantities: the settling time of the deflec tion circuitry, At, and the signal propagation delay of the video filter used to limit the high frequency noise. This total delay time is best determined experimentally. It was found that a programmed delay of 50 microseconds between D/A loading and the beginning of A/D conversion was sufficient to achieve 5-bit grey-level resolution. This implies that within the 50 microseconds, the filtered video signal settled to within the analog voltage equivalent to one-half of the least signifi cant bit of quantization. This 50 microsecond delay is the nominal value used in all discussions and system measurements to follow. 12 0 10 20 30 40 50 60 Dwell Tlma At (microseconds) Fig. 2.2.2 Signal to Noise ratio of the image dissector as a function of At, the dwell time. 13 2.3 The Microscope The image dissector is optically coupled to a Zeiss Universal microscope such that an image is formed on the photocathode of the dis sector. Figure 2.3.1 is a block diagram of the optical arrangement. For transmitted light microscopy, a 60 watt 12 volt illuminator is used. The power supply for the illuminator is required to be well regulated and filtered to remove any 60 cycle ripple, since this would appear as noise on the video signal from the dissector. The microscope is equipped with a mechanical stage driven by two stepping motors. The stepping frequency is 200 hz and the step size is 10 microns in both the X and Y directions. The fine focus control has also been equipped with a step ping motor, providing 2000 steps per revolution at a stepping frequency of 600 hz. To move the stage manually under program control with dynamic focusing, the system includes a joystick. The joystick can also be used to drive the focus motor manually. Vibration or mechanical shock in the neighbourhood of the micro scope can cause jitter in the image. The system could be improved by strengthening the optical coupling between the microscope and dissector, or by mounting these components on a shock table. Most confusion from jitter can be compensated for by careful programming, but it is best to attempt to prevent the causes of the jitter. 2.4 The Computer Interface The computer used in this system is a PDP-9 equipped with 16K of memory and three Dectape units. The interface for the dissector has been well documented (8), although a few changes have been made. Figure 2.4.1 is a block diagram of the hardware configuration. Although much 14 Imago Dissector Eyepiece »» A Oil Blood Filni Scanning Stage - Eyepiece . Optical Coupling . Microscopo Head Oil Immersion Eyepiece Conden3or Light Path Light Source Figure 2.3.1 Block diagram of the optical system. (TO <-DECTAPS UNITS CRT DISPLAY TELETYPE PDP-9 COMPUTER LOAD D/A 10 DATA LINES 10 aiTA LIKES LOAD D/A X D/A CCNV. Y D/A CONV. START A/D 6 DATA LINES READ A/D A/D CONV. STEP X STEP X STEP I STEP I STEP CLOCKWISE STEP COUNTERCLOCKWISE 3 DATA LINES START. RES!) A/D CONV. 8 CH. Max. MODE LOAD MUX. CH. IMAGE DISSECTOR AKALOG VIDEO SIGNAL SCANNING STAGE JOYSTICK Figure 2.4.1 Block diagram of the computer interface. 16 time has been spent in designing and constructing the hardware, the details of the construction will not be explained here. It is felt that the algorithms and techniques developed for the application of this system are more important, although much practical experience has been ' obtained in assembling the hardware. Appendix II summarizes the software instructions used to control the hardware. These descriptions should adequately explain the possible hardware functions from the user's point of view. 2.5 The System Resolution Resolution of the system is a bounded function of the resolution of both the image dissector and the microscope. Before the combined re solution can be determined, the resolution of each system component must be evaluated separately. Optical resolution of a microscope is commonly expressed as: microns (2.5.1) N.A. , + N.A. , . cond obj where X = the wavelength of illumination N.A. , = the numerical aperture of the condenser cond N.A.Q^J = the numerical aperture of the objective. This equation represents the highest resolving power possible with the chosen lenses and type of illumination. The equation only applies under conditions of critical illumination, that is, when the source of light is focused on the object in such a manner that the beam fills approxi mately two-thirds of the aperture of the objective (11). Critical illu mination usually implies that N.A. , is equal to N.A. , .. The impor-J r cond obj tant feature about optical resolution is that a limit exists beyond which Optical-Magnifi Magnification to CRT Display Magnification to Image Dissector Size of Field 1 of View (microns) Inage Dissector Eosoluticn Optical Resolution cation 1 micron = ... cm. Pover 1 micron Pover GS = 1 GS = 4 200 .0145 ' H5 .0046 46 551.76 .55 2.20 1.2 I 256 .0180 ISO .0057 57 444=43 .44 1.76 1.2 320 * .0221 221 .0070 70 362.00 o36 1.44 1.2 320 * .0230 230 .0073 73 347.34 .35 1.40 .63 410 .0237 287 .0091 91 273.72 .23 1.13 .68 512 .0355 355 .0113 113 225.36 .25 1.00 .68 500 .036S 368 .0117 117 217.63 .22 .83 .42 640 .0453 453 .0144 144 176.64 .18 .72 .42 SCO * ,0561 561 .0173 .178 142.64 .14 .56 .42 800 * .0572 572 .0182 182 139.84 .14 .56 .27 1024 .0715 715 .0227 227- 111.92 .11 .44 .27 12S0 „C3S9 889 .0282 282 90.00 .090 .36 .27 2000 .1430 1430 .0454 454 55.94 .056 .224 .21 . 2560 .1783 1783 .0566 566 44.87 .045 .180 .21 3200 .2200 2200 .0699 699 36,36 .036 .144 .21 * Same magnification, but vith different objective. Note Optical Resolution. Table 2.5.1 Results of the calibration of the image dissector resolution. 18 the magnifying power cannot be profitably increased. The best numerical aperture possible is about 1.4, and at this value, oil-immersion techni ques are necessary to increase the index of refraction of the inter-lens medium. Applying the equation for optical resolution, it is found that with green light (A = .54u) and a N.A. of 1.3, optical resolution is about . 21u. This corresponds well with the size of granules in the leukocyte which are about .25u in diameter. This also represents the highest resolution possible with this microscope and occurs when the 100X objective is used. The image dissector has a .005 inch round defining-aperture and a one inch square photocathode surface. Therefore at 86.5% depth of modulation with GS = 4 (refer to section 2.2), 256 by 256 distinct points can be resolved on the. image area. In order to translate the dissector step size into a meaningful measure of length, a slide scribed at 10 micron intervals was used to cali brate the system. Image measurements were made on an eight centimeter square CRT display unit, and these measurements were converted into their equivalents across the one inch square photocathode. Once the system has been calibrated, the calibration is applicable only as long as the image dissector remains in a constant position relative to the eyepiece. Table 2.5.1 summarizes the results of the calibration based on magnifi cation and different step sizes for the dissector. These results are also plotted in Figure 2.5.1A for GS = 4. This graph shows that dissect tor resolution may be expressed as: 19 1.0 B g •A 0.5 o.o 1000 I I I 1 I I • 2000 30O0 Pover of Kaenification Fig. 2.5.1 Resolution as a function of magnification for the image dissector (A) and the microscope (B). 20 Fig. 2.5.2 System resolution as a function of magnification. 21 Optical resolution is plotted in Figure 2.5.IB. Optical resolution and dissector resolution are then combined in Figure 2.5.2 to produce the bounded system resolution. 22 3. DEVELOPMENT OF THE AUTO-FOCUS ALGORITHM 3.1 Background Optical systems must be properly focused to yield optimum per formance. This is especially true for a high-power microscope where the depth of field may be less than .25 micron (.11). The most common method of evaluating image quality is subjective human evaluation. Sharp or well defined edges, clearly defined shapes, fine detail, and overall crispness or contrast of the image are some of the features an observer looks for. Auto-focusing algorithms that optimize image quality must define the means used to extract measures of these properties. Auto-focusing algorithms have been previously proposed, although most of them are too complex or too inefficient to effectively implement on-line. Mendelsohn and Mayall (12) have suggested that focusing can be accomplished by maximizing the function i=l <(> = I (OD. - ty) for all OD. > ty ' (3.1.1) n where OD^ is the grayness at a point, n is the number of points in the image with OD^ > ty, and ty Is a reference arbitrarily fixed at an optical density in the mid-range of image greyness. Mendelsohn and Mayall admit that the choice of tj; may not be straightforward, but explain that the choice of ty can control which component of the image will be in focus when all components do not lie in the same horizontal plane. This algo rithm may also be difficult to evaluate at high speed. Conceptually, each evaluation of the function ty during maximization involves a scan of the complete field of view. The auto-focusing algorithm to be presen ted here eliminates one dimension of the scan, and therefore should offer 23 sufficient speed for effective implementation in an on-line image pro cessing system. 3.2 Developing the Algorithm one-dimensional function. Figure 3.2.1A shows a scan line across a field of view when the image is in focus. Characteristically, as an image is put out of focus, the intensity information on the scan line deterio rates as shown in Figure 3.2.IB. By examining the image while varying the focus, correlation between image quality and certain characteristics in the scan line may be established. Figure 3.2.2 shows the primary features searched for in a scan line, namely, sharp edge definition, good contrast ratio, and rendition of fine detail. Based on this cor relation, a focusing algorithm is proposed that utilizes a function that has a maximum when the scan line contains these characteristics. The func tion to be used is statistical variance of the intensity levels along the scan line. It is proposed that focusing may be accomplished by adjusting the focus control until the variance reaches a maximum. iation of the single measurements within that set. In Figure 3.2.1, the intensity levels of the in-focus scan line contain more variation than the out-of-focus scan line. It follows that variance, a measure of variation, could be used to judge the quality of the image. The unbiased 2 estimator of the variance, s , is traditionally defined as follows (18). A single video scan line across an image may be viewed as a The variance of a set of measurements is a measure of the var-2 I (x. - x)2 i s N-1 (3.2.1) I x. where i = 1 to N; x. = an individual sample; x = x .N 24 nu«*ctor I Coordinate 200 400 Mjiector X Coordinate 600 Fig. 3.2.1 A video scan line when in focus (A) and when out of focus (B). 2 To simplify the calculations required to extract s , it is more conven-2 ient to use the biased estimator of s , which is defined (18) as follows. 2 i (x± - x)2 N (3.2.2) where i • 1 to N • an individual sample 25 > o I -P m 60 AO 20 Sharp Edges . Fine / Detail _1_ JL 200 ^00 Dissector 1 Coordinate 600 Fig. 3.2.2 Characteristics searched for in a video scan line. Although the magnitudes of the estimators differ, it is permissable to use either estimator as both equations have a maximum with the same subset of data. Equation 3.2.2 may be reduced algebraically. y (x.2 - 2x x. + x2) . i i x s = N I x. + 9-2 -2 - 2x + x Nx2 7 1 N N V N < (3'2-3) Equation 3.2.3 was used in all the measurements that follow. Numerically this function is evaluated by simple recursive evaluation as each intensity 26 200 -0 1 ' ' i i i i i i i i— -SO -60 -40 -20 0 20 40 ' 60 80 Nuriber of Focua Stops Number of Focus Steps Fig. 3.2.3 The variance plotted against focus control position unfiltered (A) and filtered (B) with a digital filter window size of ten. 27 level, x., becomes available: accumulate the two totals Tx.2 and Jx., 2 and when the scan is. complete, square the second total and divide by N , subtracting the result from the first total divided by N. Figure 3.2.3A shows the variance of intensity plotted against the fine focus adjustment for a typical cluster of cells. The "zero" point on the horizontal axis represents the focus control position where the image was judged as being of the best quality. Figure 3.2.3B shows the' same curve after digital low-pass filtering. Filtering simplifies peak detection by smoothing the curve. The digital filter used simply averages the variance from several adjacent points and assigns the average to the central point. To state this in the form of an equation, the variance att.the k^ point is defined as follows: l v, = I v. k N (3.2.4) where N = the digital filter window size (^1) th. v. = the variance at the i point x and i varies from k - — to N k + — -1 for N even and from i N-1 ^ , , N-1 . k — to k H — for N odd. Several subjective evaluations of image quality verified that the variance peaks when the image is in focus, thereby justifying the use of this algorithm. At first, it may be disconcerting to find that an automatic system can focus more critically than the human operator. One of the tests conducted required a person to manually focus the microscope to produce the best image. The automatic focus would then 28 be activated and many times an improvement of the manually adjusted image was noticeable. Focus by variance maximization is numerically simple to imple ment, provides good averaging over the field of view, and in such avera ging, compensates for optical curvature. By maximizing the variance along several non-adjacent scan lines, the best overall image is obtained. 3.3 Factors Affecting the Variance Function Several factors alter the characteristic shape of the variance function. In order to apply variance maximization techniques effectively, these factors must be understood and controlled. During the following discussions concerning these factors, it should be assumed that all vari ables and conditions affecting the variance function are held constant, unless it is stated otherwise. The "zero" point on the horizontal axis represents the point where image quality was judged to be "best". The contents of the field of view affect the variance function. 01 I • I | I 1 L o I I J 1 1 1--20 -10 0 10 20 -20 -10 0 10 20 Hunbor of Focua Steps Number of Focus Stopo Fig. 3.3.1 The variance function of two different fields of view showing peak variance scaling. 29 Number of Focus Stops • Uvaber of Focus Stops Fig. 3.3.2 Scaling of the variance function due to magnification .shown at 800X (A), 1024X (B), 1280X (C), and 2000X (D). 30 Different objects contain varying amounts of information, and so, as the field of view is changed, one can expect that the peak variance value will also change (Figure 3.3.1). It is possible that a new field of view may have a peak variance that is below the sensitivity of the variance maximization algorithm. For such cases where a maximum variance value may not be detectable, or may not even exist, the maximization algorithm should maintain control by recognizing this situation. Therefore, the information content of the objects in the field of view determines the peak value of the variance function. System magnification controls the scaling on the horizontal axis as shown in Figure 3.3.2. As magnification is increased, the depth of field of the optics decreases, and therefore the range in which image information is received also decreases, giving the variance func tion, a narrower, shape. Since peak detection is the objective, the focus control step size should be made smaller as magnification is increased, effectively spreading the variance function over the horizontal axis. It is difficult to generalize this scaling property, as the depth of field is dependent on the numerical aperture (N.A.) and exit pupil size of the specific objectives used (11). However, a convenient estimate is to say that as magnification is doubled, the focus step size should be reduced by 1/3 to 1/2 (See Appendix I concerning FOCSTP for nominal values) as determined experimentally. Optical filters are usually chosen to enhance the contrast of the image in a meaningful way. However, the filter selection will also affect the spread and peak value of the variance function. Figure 3.3.3 demonstrates how different filters affect the variance function extracted from an image of cells. As a filter narrows the variance 31 o l i -i I i 1 L. o I  1 1 1—-—i »• -80 -40 0 40 80 _g0 0 40 80 Nunbov of Fotroo Steps Kunber of Focuc Steps Fig. 3.3.3 The effect of no filter (A), a red filter (B), a blue filter (C), and a green filter (D) on the variance function extracted from an image of cells. 32 function, the filter is being more selective. That is, some parts of the image are losing contrast. As a filter increases the peak value of the variance function, the contrast of specific parts of the image is being enhanced. An image closely represents the object at only one plane of thickness. If the filter is chosen such that image information from the planes surrounding the plane of interest is symmetrical, then the variance function will also be symmetrical. Generally, the best filter produces a well-defined symmetrical variance function and pro vides sufficient contrast in the interesting areas of the image. The intensity of illumination produces a scaling affect on the variance function. As background illumination is halved, the difference of intensity between a light and dark object is also halved. Considering that the variance is related to the squares of these differences, 0 I i —i -—i ' 1 1 1  -60 -40 -20 0 20 40 60 Number of Focu3 Steps Fig. 3.3.4 The variance function extracted at full intensity and at half intensity. The bottom curve is the one at half intensity. 33 halving the intensity reduces the variance to approximately one quarter of the full value (Figure 3.3.4). Critical illumination (refer to Section 2.5 for the definition of critical illumination) affects the variance function in two ways. First, maximum optical resolution occurs under conditions of critical illumination, and therefore the variance function should have a higher peak value under critical illumination conditions. Second, the two terms in equation 3.2.3 for the variance should peak at the same time or else a shift in the peak variance will occur. This peaking is coin cident only under conditions of critical illumination. Figure 3.3.5 shows the effect of critical illumination at a high magnification. For tunately, at lower powers of magnification this phenonmena is not as 0 I _J »_i i i i 1 u -60 -40 -20 0 20 40 60 Hunber of Focus Steps Fig. 3.3.5 The effect of non-critical illumination on the variance function at a magnification of 2560X.' The top curve represents critical illumination. 34 pronounced. All these factors can easily be considered and compensated for during implementation of auto-focusing by variance maximization. Through an understanding of these properties, a generalized maximization algorithm can be developed that is both magnification and object inde pendent. 35 4. THE LEUKOCYTE LOCATION PROCEDURE 4.1 The Conditions of the Search Erythrocytes outnumber leukocytes by approximately 600:1 on a normal blood smear, and therefore locating or isolating leukocytes can be a time-consuming task. As magnification increases, the area of the field of view decreases and therefore the probability of a leukocyte existing in any particular field of view also decreases. This implies that a longer time is required to locate a leukocyte. Most leukocyte location algorithms take advantage of the colour properties of a blood film stained with Wright's stain (4,13). Since the image dissector cannot distinguish between colours and is only sensitive to intensity, it is helpful to select optical filters to enhance the contrast ratio of the leukocvtes so thev mav be separated 300 400 ' 500 600 700 800 900 Wavelength A in nm. Fig. 4.1.1 Spectral density curve of the Zeiss 46 78 06 wide-band pass green interference filter. from the erythrocytes. By making use of the heavy absorption of green light in the stained leukocyte nucleus, the leukocytes may be located. The spectral density curve of the filter that gave the best results at all magnifications is shown in Figure 4.1.1. When choosing a filter for this system, it must be realized that the erythrocytes are required to 36 show in the image, or else the auto-focusing algorithm (Chapter 3) will not function properly. Figure 4.1.2 shows scan lines across a leukocyte with and without contrast enhancing filters. Under conditions of con trolled illumination and uniform staining, searching for the optically dense nucleus is a satisfactory criterion for locating the nucleus. 4.2 Threshold Determination Some quantitative measure of "optically dense" is required to Dissector X Coordinate Dissector I Coordinate Fig. 4.1.2 Video scan lines across a leukocyte with (B) and without (A) contrast enhancing filters. locate the leukocytes. Detection seems best done in two stages since the nuclei of all classes of leukocytes are not of the same density. The first stage involves the establishing of a crude threshold that is expected to include all leukocytes. The second stage refines the first threshold for the particular cell found so that measurements can be made on this cell. The first threshold is the most critical in the sense that cells designated by this threshold should, with a high probability, be leukocytes. An absolute threshold value does not work as the nuclear 37 intensity varies significantly from slide to slide and from field of view to field of view. This suggests that the first threshold should be a function of the mean intensity of the current field of view. As the search enters into a new field of view, the average background intensity sampled over various areas of the image is multiplied by a constant, defined as the contrast ratio, to produce the threshold value. To put this in the form of an equation, the „ _ _. ^. Threshold for Leukocytes , . •_ 1X .Contrast Ratio = —: —; 1—: (4.2.1) Average Background Intensity The threshold defined in this way works well as long as the contrast ratio has been carefully chosen. However, due to the varying transmit-tance properties of each blood film, it is necessary to redefine the contrast ratio for each slide. This is done by locating a leukocyte on the new slide, and by adjusting the contrast ratio until the resulting threshold includes this leukocyte. The whole slide can then be pro cessed using this updated contrast ratio. A fixed contrast ratio cannot be used for all slides since a wide variation in the staining density and general quality of the blood films exists. Approximately 25% of the films cannot be properly pro cessed with a fixed contrast ratio. The blood on the test slides had been smeared by hand and then stained automatically. There are now available automatic blood spinners and stainers, for example, the Perkin-Elmer Coleman Model 90 Blood Spinner (16). These units are used to prepare slides for instruments which are not able to make the subtle adjustments in perception that the experienced human can make. It is claimed that such automatically prepared slides are unfailingly excel lent, containing randomly distributed, uniformly spread cells that have an absolute minimum of mechanical distortion (16,17). Slides produced 38 by such an automatic process should be studied, and it is expected that a fixed contrast ratio could be determined and used successfully to process all such prepared slides. The second threshold value is required once a suspected leukocyte has been found. A "suspected" leukocyte is defined as an object that falls below the first threshold and is of a certain size. Various measurements of this suspected leukocyte may be required, and contour tracing of the nucleus is assumed to be one of these tests. In order to accurately contour trace an object, a refined threshold value is required. This threshold is usually defined to be the intensity level occurring at the maximum gradient of intensity across the cell boundary. Thresholds for both the nuclear and cytoplasmic material of a leukocyte may be determined in this manner. 4.3 The Scan Pattern Figure 4.3.1 shows four of the standard scan patterns used in the differential count (14). The longitudinal pattern is the one most commonly used. Several patterns have been standardized since the distribution of leukocyte types across the smear is not uniform: neutrophils and monocytes predominate at the margins and the tail, and lymphocytes predominate in the middle of the film (3). No commitment of any preset pattern has been made, and the pattern selection is com pletely flexible (Refer to Appendix II concerning "PATTRN"). However, independent of the pattern used, some care must be taken to ensure that cells are not counted twice since parts of them lie in adjacent fields of view. A "dead" zone needs to be left between processed areas of the field of view as shown in Figure 4.3.2. This 39 » > A B C D Fig. 4.3.1 Standard scan patterns used in the differential leukocyte count. The straight edge pattern (A), the battlement pattern (B), the cross-sectional pattern (C), and the longitudinal pattern (D). "> o •K "-V-i 3f Current Search Area Dead Zone Adjacent Search Aroa 1 Fig. 4.3.2 Placement of adjacent scan areas. 40 "dead" zone should ideally be equal to the size of the largest cell expected. In the case of white blood cells, the monocyte is the lar gest type, with a size ranging from 13 to 19 microns. This dead zone should also be considered to frame each field of view. As the center area is processed and a cell is located which overlaps onto the frame, the whole cell will then be visible and processing can cross onto the frame (Figure 4.3.3). If these considerations are made, the probability of missing any leukocytes is very small indeed. Dead Zone y Current Field of View Fig. 4.3.3 Relationship between current and adjacent scan areas, showing the placement of the dead zone. 41 5. THE SOFTWARE IMPLEMENTATION Four global routines, each having a distinct function, have been written to control the microscope and locate the leukocytes. These routines have been written in such a way that they are logically inde pendent of magnification. Compensation for any change in magnification is achieved by altering the values assigned to the six variables des cribed in Appendix I. The next chapter will describe how the system was tested using such techniques as contour tracing and curvature measure ment by an area operator. 5.1 "FOCUS" As already discussed, auto-focusing is accomplished by variance maximization. A flow diagram of the variance maximization algorithm, called "FOCUS", is shown in Figure 5.1.1. Execution time of this routine is' a function of digital filter window size, magnification, and the initial position of the focus control. Digital filtering is accomplished by storing the variance values in matrix form, and as the digital filter size increases, the length of this matrix also increases. The average value of this matrix is considered to represent the filtered variance at the point represented by the central entry in the matrix. As the magnification is increased the depth of field decreases, and therefore the number of discrete steps defining a logical clockwise or counter clockwise incremental focus adjustment decreases. This decrease in the incremental distance that the focus motor must be driven helps to speed up execution. This is the only routine that executes more quickly at higher magnifications. If future application permits the use of a single magnification, the focus motor should be equipped with different gearing to provide the proper incremental step size. 42 Initialize VARIANCE matrix for digital filtering. ( Step fccus motor CW "| by FCCSTP. [ Got nev variance. X Update and evaluate VARIANCE matrix. Step CCW by TCCSTP and get new variance, _ y Update and evaluate VARIAUCE matrix. Stop 2 X POCGT? CH _to_Varianco peak. Fig. 5.1.1 Flow Diagram of the focusing program, "FOCUS" 43 Field of View Scan y Lines Fig. 5.1.2 The scan lines used to sample the variance. The variance is normally sampled across the search area as shown in Figure 5.1.2. The critical global factors are VARNUM, the digital filter window size, and FOCSTP, the number of physical focus steps equivalent to one logical focus increment in this algorithm c(Refer to Appendix I for nominal values) . 5.2 "THRESH" "THRESH" is the routine that calculates the threshold level of intensity used to segregate leukocytes from the background. A sim plified flow diagram of THRESH is shown in Figure 5.2.1. A single scan line across the search area is sampled to obtain the intensity level of the two brightest points. The average of these two points is then tested to see if it falls between predefined limits. If the intensity is outside these limits, a message is displayed asking the operator to either increase or decrease the illumination. During system design, the deviation of intensity due to changes in specimen thickness and concentration of stain was underestimated, and therefore the system Zero Bl and D2. Sot C=1000. PCot tho intensity of . a point on tho lino Y - C. Tont tho intensity of this point. Test if the line T = C h&s been completely scanned. Average Bl and B2 and tost size of average. Store negative of average in TLSTEL. Display message intensity too dull for timed interval. Display noasage intensity too bright for a timed interval. Sot C = 700. Zero Bl and B2. Fig. 5.2.1 Flow diagram of the threshold determining program "THRESH" 45 could be improved if THRESH had some means of remotely controlling the illumination when the intensity exceeds certain limits. However, if the average intensity is acceptable, the threshold is calculated by multi plying the average background intensity by the contrast ratio,. CR. The negative of this value is stored in TLEVEL for global access. 5.3 "JOYSTK" "JOYSTK" is the handler for the joystick attached to the. sys tem. Two modes of joystick control are available, focus control only, or slide position control with dynamic auto-focusing. The mode is selec ted by means of a labelled two-position switch on the joystick unit. The proportional position and direction that the joystick is moved controls the velocity and direction that either the stage or focus motor is driven. The joystick handler must be initialized before each use by calling JOYINT. After initialization, JOYSTK should be called repeatedly from a program loop as long as control is desired. The frequency with which JOYSTK is called will affect the response speed of the joystick action. 5.4 "STRPNT" "STRPNT" is the routine that actually locates the leukocytes. As each leukocyte is found, coordinates defining the location of the leukocyte are passed to an external subroutine called CNTOUR. CNTOUR interprets these coordinates as defining the location of a cell, and uses the coordinates to contour trace and classify the cell. From the point of view of CNTOUR, STRPNT merely produces a data stream of cell coordinates. STRPNT begins searching a new field of view in the bottom 46 rinHl«.iir.o pcati and jstn^a control p&rn-Imotors. , :T::Z... _J Inilinllio couutor to j j call auto-focus, :L. :.:z. Call TOCU3 and THRESH 1 Get point intensity ar.d incrcrant scan coordinates by AGS. j Perform AGS -"] minimum square | teat, Tost if current fiold' of view baa been scan-ned ^completely, J. ~ Access* P/:TTRH andmovo Btage to noxt field of viev. Tost-if tho end of PATTRIJ has been roached. To a Increment focus count and tost If timo to auto-focus• 183 Move to cell boundary by GS increments.  Call CHIOUB to process the coll. Two returns are possible 1 Noroal\ Roturu \ Error^ Return Store cell coordinates Bo this cell can ba identified as found. • Incrcmont coll count and test if enough colls have been found. Done? [Normal\ Exit ig. 5.4.1 Flow diagram of "STRPNT" 47 left hand corner, testing one row at a time (Figure 5.4.2A). Sequential points are spaced apart by an incremental step size called AGS. When a point is found that falls below the threshold, an area test is made where a minimum square of size AGS is used. If all four corners of this square fall below the threshold, the edge of the object is searched for. The edge is detected by stepping left with a step size GS until the first point above the threshold is found (Figure 5.4.2B). GS is assumed to be the incremental step size to be used by CNTOUR in the classification work. The coordinates of the edge point are stored in A B Fig. 5.4.2 The sequential search technique used to locate leukocytes, demonstrating the minimum square fit (A) of size AGS, and the location of the starting point (B) by using GS. SXD2 and SYD2 and the subroutine CNTOUR is called. CNTOUR may reject these coordinates and STRPNT will verify the validity of SXD2 and SYD2. If the coordinates are accepted, STRPNT will continue searching the field of view for other cells, noting the position of the cell already found. After the field of view has been searched, STRPNT moves the slide ac cording to the directions stored in PATTRN (Refer to Appendix II) which describe how the stage should be moved. 48 STRPNT also calls FOCUS to maintain the image in focus. Ideally, FOCUS should be called each time the field of view is changed, since there is no assurance that the image will still be in focus after the stage is moved. However, it was found that it was acceptable to focus when either one of two events occurred: when a suspected leuko cyte was found and this field of view had not been focused, or when focusing had not occurred for a fixed number of stage moves. The number of times that the stage can be moved before focusing is needed is depen dent on magnification, and as magnification increases, focusing must occur more often. STRPNT has two exits to the calling program. One exit, the normal one, occurs when the specified cell count is reached. The other exit occurs when the scan pattern stored in PATTRN is exhausted before the cell count is reached. 49 6. SYSTEM EVALUATION 6.1 The Test System Auto-focusing was subjectively evaluated apart from the rest of the system. Several individual opinions of image quality were ob tained, and correlation between peak variance and image quality was established under a wide range of circumstances. Attempts to manually improve the focus only verified that variance maximization produced the best average image quality. If the considerations discussed in Chapter 3 are implemented, variance maximization provides an efficeint and consistent means of focusing. A complete software package modelled after the one in Appendix II was written to test the efficiency of the system. This package offered several distinct modes as listed below: 1. Initialization - Initializes the parameters Appendix I on the basis of cation used. 2. Scan Line Display - Displays the intensity levels encoun tered on a single scan line in the X direction. The Y coordinate is controlled through the teletype keyboard. 3. Threshold Intensity Map - Displays all the points in the field of view that lie either above or below the threshold. This mode is used to pick the contrast ratio, CR, which is used to define the•threshold. 4. Joystick Control - Activates continual control of focus and stage position through the joystick. 5. Contour Tracing and Curvature Measurement - Locates a specified number of leukocytes, and contour traces their nuclei. The contour traces and curvature functions may be displayed for viewing. listed in the magnifi-The last mode is the most important one for determining the efficienty 50 of the system. The remainder of this chapter will describe the techni ques used for contour tracing and curvature measurement, and will dis cuss the relative speed one can expect using these techniques. 6.2 Contour Tracing Contour tracing was conducted on-line using the tracing al gorithm shown in Figure 6.2.1. A boundary point is defined as lying at the centroid of any four of the array points forming a minimum square, the side of the square being defined as GS in this context. The four black dots designate the array points in the minimum square configuration. The centroid of the minimum square is defined as a boundary point whenever one, two, or three of the four corners of the square fall below the threshold intensity. If none or all of the points fall below the threshold, the centroid is not a boundary point. The arrows in Figure 6.2.1 indicate the four possible directions in which the next boundary 3 O . A D3 © e DA 2 O -< « > o /, D2 O OD1 O 1 Fig. 6.2.1 The contour tracing algorithm. The black dots are spaced at GS and represent the four points that are interrogated to determine which direc tion (the circles) the operator should be moved. point is sought. For a counterclockwise trace, the four directions are interrogated in a counterclockwise order. The first direction inter rogated depends upon which of the minimum square points are below the 51 threshold. Other tracing algorithms may be found described in (15). This tracing algorithm was implemented so that the discrete area opera tor described in the next section could be used to measure curvature. 6.3 The Discrete Area Operator The discrete area operator shown in Figure 6.3.1 was used to measure boundary curvature. The area operator is centered over a boun dary point as determined by the contour tracing algorithm. The centroid of the area operator assumes a value of +386, and each array point having an intensity below the threshold assumes the value as shown in Figure 6.3.1. All other array points assume a zero value. The total of the values of the array points and centroid give a measure of the curvature o o o o„ o o -1 -3 -7 -7 -3 -1 o o o o o c -3 -17- -37 • -37 -17 -3 o o a o o o -7 -37 -81 • x -81 -37 -7 •»386 o o o o o. o -7 -37 -81 -81 -37 -7 o o o o o o -3 -17 -37 • -37 -17 -3 o 6 o o o o -1 -3 -7 -7 -3 -1 Fig. 6.3.1 The discrete area operator used to extract boundary curvature. at the boundary point. The values associated with the array points follow a Gaussian weighted profile modelled after the physiology of the concentric receptive fields of a cat's eye (15). A curvature function can be generated efficiently once the contour has been traced. At the first boundary point, all 36 points of the area operator are interrogated and stored in matrix form. Further Fig. 6.3.2 Photographs of several cells, their contour traces, and unfiltered curvature functions as extracted by the area operator. boundary points involve the interrogation of only six points, which represent a new row or column of the matrix. The row or column that these new points represent depends on which direction the centroid of the matrix must be shifted in order to lie on the next boundary point. Figure 6.3.2 shows contour traces and curvature functions of several cells. Nuclear curvature has been shown to be an effective parameter in the classification of leukocytes (1,5,8,12). Classifica tion of leukocytes still remains to be completed with this system. The purpose of implementing on-line contour and curvature techniques has been to demonstrate that such on-line processing techniques are suit able and efficient means of automating the differential leukocyte count. 6.4 Reliability and Execution Time Measurements Several performance tests over a certain number of cells were conducted to extract a mean measure of the system performance. These tests involved locating or isolating a specified number of cells, contour tracing these cells, and generating their curvature functions. The first test was designed to obtain a measure of the relia bility of correctly isolating or locating leukocytes. A scan pattern was chosen that had a true count of 100 leukocytes on it. A program was then arranged that would scan this pattern ten times, printing out the number of cells actually isolated and traced. This test was conduc ted several times and typical results are shown in Table 6.4.1. Pass 3 obviously missed a cell, but it was impossible to determine which cell was missed and why. Pass 8 indicated an extra cell had been picked up. This may have been a large platelet and a more stringent 54 Pass § True Cell Count As Counted Error Count 1 100 100 2 100 100 3 100 99 -1 A 100 100 5 100 100 6 100 100 7 100 100 8 100 101 +1 9 100 100 10 100 100 Table 6.4.1 Results of a test designed to count leukocytes acrsos an area with a true count of 100 cells. area criterion could probably eliminate this error. The mean cell count from this test is 100 cells, and the standard deviation is + .11 cells. The second test was designed to give an absolute measure of the time required to isolate and process 100 cells. A scan pattern as shown in Figure 6.4.1 was used in this test. The scan pattern was generally long enough that 100 cells would be found before the end of Fig. 6.4.1 The scan pattern used to measure the time required to locate and process 100 cells. the pattern was reached. If this was not the case, the pattern would just be repeated. At the end of a 100 cell pass, the slide would always be returned to the initial position. Ten slides were selected at Slide Tino in Seconds for 100 Coll Scan © Magnification Below Nuiabor 40 X 16 X 1.6 40 X 16 X 2 100 X 16 X 1.25 A B C A+3+C D S F D+E+F G E I G-i-H-s-I 1 160 167 153 162 160 153 163 160 205 193 194 199 2 105' 103 107 105 130 123 127 123 128 135 132 132 3 183 180 185 133 194 195 193 196 250 244 247 247 4 101 100 101 101 120 117 121 119 167 162 163 164 5 120 116 116 117 124 125 130 126 194 196 195 195 6 265 269 272 269 283 290 286 239 306 311 309 309 7 99 100 100 1C0 113 115 114 115 125 132 123 127 8 120 123 126 123 130 129 129 129 171 169 167 169 9 230 226 223 228 259 261 260 260 296 299 299 293 10 140 139 143 141 148 150 147 I43 175 179 181 179 Average Tims 153 Average Tiaa 167 Average Tine 202 Table 6.4.2 Times required to locate and process 100 leukocytes on.ten randomly selected blood films. 56 random and the test performed across the same area of each slide three times at three different magnifications. The results of this test are summarized in Table 6.4.2. The contrast ratio for each slide was chosen as described at the end of Section 4.2. The results of this second test are most interesting. Notice that the mean time required to complete the test increased i^ith magni fication as expected. However, the increase in time is not very large. As magnification is doubled, the probability of locating a leukocyte in any one particular field of view is halved. Supposedly this should cause a doubling of the time required to run the test. This does not occur since focusing takes less time at the higher magnifications. Focusing takes longer at the lower magnifications because the variance function is stretched over the horizontal axis and peak detection re quires that the focusing motor be driven greater distances. If the gearing of the focus motor were changed for each magnification such that the step size of the focus motor would be equivalent to the number of incremental steps required, the mean time for this test at the lower magnifications would decrease. Therefore once the required magnifica tion is determined, it will be advantageous to tailor the focus control hardware to this magnification. The distribution of the times recorded for the second test is not Gaussian, and therefore it is difficult to interpret the meaning of the deviation of the results. However, at 1024X the deviation is + 5 7.95 seconds, at 1280X it is + 61.85 seconds, and at 2000X it is + 64.20 seconds. Therefore the deviation also increases for higher magnifications indicating that there is a two-dimensional spread as magnification is increased. 57 Some measure of success of this system design can be obtained by comparing the results of this test to the time a technicain requires to perform a differential leukocyte count. A technician requires from five to ten minutes to perform a 100 cell differential leukocyte count. The times for the second test do not include complete classification of the cells, although curvature functions for the nuclei of the cells have been extracted. Keeping this in mind, it is conservatively estimated that one tenth of a second per cell should be sufficient to complete the classification. Based on this assumption, the system developed is at least twice as fast as the technician performing the same task. Further advantages are also obvious when it is considered that the technician normally cannot work at the microscope for more than an hour at a time due to eye strain. Based on this second test alone, the techniques used in this system are comparatively efficient. 4 58 7. CONCLUSIONS The system provides general purpose control of a microscope and is adaptable to various types of on-line image processing. The system was designed to develop efficient techniques to eventually auto mate the differential count. The system can be equally well configured to perform other types of biological image processing. The logical algorithms for controlling the functions of the microscope have in themselves worked successfully. The efficiency of these algorithms in their interaction with the hardware has been accep table. By tightening the specifications of the hardware and by tailoring the hardware to a specific magnification, further gains can be made in system efficiency. It is worthwhile to review some of the ways in which this could be done. The settling time of the D/A converters and ampli fiers in the deflection circuitry of the image dissector can be impro ved. Currently 50 microseconds plus the A/D conversion time is required to interrogate a point. It should be possible to reduce this time to 25 microseconds as the dissector requires a dwell time of only 11 micro seconds to maintain a signal to noise ratio of 64 representing five-bit grey level resolution. With this improvement, a 20% to 35% reduction in the times of the second test in Chapter 6 could be expected (Table 6.4.2). Another substantial time saving can be made by gearing the focusing motor to a specific magnification. Focusing time could be reduced by as much as 15% to 25% at the lower magnifications. This would imply about a 10% reduction in the results of the second test in Chapter 6. Currently the wide tolerances found in the quality of the blood films has forced updating of the contrast ratio for each new slide 59 processed. This method would be impractical in a system where the slides are automatically inserted under the microscope. The quality of blood films produced from an automatic blood spreader and stainer should be investigated. In such an automatically controlled process, the trans-mittance properties of the films should be quite uniform, and the use of a fixed contrast ratio can be expected to work well. Alternatively if this is not true, the contrast ratio will have to be updated for each new slide since there is no means of guaranteeing the quality of a slide. It would be advantageous in a practical system to provide some means of digitally controlling the intensity of illumination within a small range. Presently a variance in image intensity of 12.5% is tolerable though software compensation and the flexibility of the con trast ratio method of threshold determination. One of the major objectives of the future application of this system to leukocyte classification should be to determine the minimum spatial resolution or amount of information required to classify leukocytes. A decrease in required resolution implies that a decrease in magnification is possible. A decrease in magnification offers several advantages: lower power lenses cost less, microscope adjustments such as condenser placement are not as critical, the depth of field increases, execution speed increases, and vibration and shock do not disturb the image as much. From the experience acquired from contour tracing and curvature measurement, the lens arrangement of 40X2X16 producing a magnification of 1280X is expected to be a good compromise between the tradeoffs mentioned above. 60 APPENDIX I The programs to control the microscope are written in such a way that they are logically independent of magnification. It was possible to compensate for the affects of magnification by defining six variables which assume different values for different magnifications. These variables are described below. GS - The incremental step size for the X and Y coordinates of the image dissector to be used when performing recognition work on the cells. GS = 4 represents the step size for maximum resolution. AGS - The incremental step size for the X and Y coordinates of the image dissector to be used when searching for cells. Generally AGS is chosen by determining the size of the minimum square with side AGS that will fit on all leukocytes at the magnification in use. AGS commonly becomes two to six times GS. Variable Fover of Magnification 102/+X 1280X 2000X (40X16X1.6) (40X16X2) (100X16X1.25) GS 4 4 4 AGS 20 20 30 CR 16 20 22 STPEQ 6 5 3 VARNUM .5- 5 5 FOCSTP -4 -3 -1 Table A.1.1 The optimum values for the six magnification dependent parameters. CR - A six-bit binary, number considered to be a fraction. The average background intensity is multiplied by this fraction to produce the threshold used to locate or isolate the leukocytes. STPEQ - The number of 10 micron steps necessary to position the stage on the next field of view. For example, STPEQ = 6 will cause the stage to move 60 microns 61 each time the field of view is changed. VARNUM - The number of discrete variance values that fall under the window of the digital filter when auto-focusing. FOCSTP - The negative of the number of steps the fine focus motor will be advanced each time the auto-focusing algorithm requires the focus control to be logically moved. The nominal values for each of these variables are listed in Table A.1.1 according to magnification. 62 APPENDIX II Four subroutines have been written to control the microscope and locate the leukocytes. These programs may be arranged in a complete system as shown in Figure A.2.1 although other arrangements are possible. Besides the magnification parameters defined in Appendix I, other vari ables are involved in subroutine communication. These variables are described below. TLEVEL - A negative number representing the threshold used to locate leukocytes. SXD2, SYD2 - Absolute coordinates for the image dissector defining the loation of the nucleus of a leuco cyte. The next point to the right at an incre ment of GS liec within the cell nucleus. WCNT - The total number of cells to be found. VARI - The value of the filtered variance function at the current focus.setting. Three subroutines must be provided by the user with the following functions. DELAY - Provides the delay time required for the dissector to settle before the intensity of a point is inter rogated. CNTOUR - It is assumed that this user written subroutine will classify the cell after the location coordinates SXD2 and SYD2 are passed to it. CRT - A collection of subroutines responsible for displaying messages in various forms and structure on the Type 30D display unit attached to the PDP-9. Any type of message handling can be substituted for this device. Table A.2.1 summarizes the internal and external references to the global symbols and subroutines. "X" indicates internally de fined. "A" indicates external access necessary. "(A)" indicates external access probable. This table should assist in using the system programs 63 USER'S PROGRAM - Initialization and Control Tho six magnification dopondont varinblo3 for tho SYSTEM Programs Euat bo initialized before Rny SYSTEM Program is colled. SYSTEM PROORAMS •4* P. "1 Lcer Jo Roto JOYIHT Initializes JOYSTK. Call onco before JOYSTK io usod. USER'S PROGRAMS 3 ••8-' q Hi O . JCYSTK Provided nanual control of microscope through the joystick. Call recursivoly froQ Dome program loop to maintain control. G "0 P u o o 3 -g .q r-l p. H M -l> o5 <S 3 STRPNT Searches for specified number of colls. Moves stage, calls FOCUS and THRESH, Passe3 call coordinates to soisa user subroutine called CNTOUR vhere recognition work takes place, T FOCUS Automatically focuse3 tho microscope on the colls in the current field of view. THRESH Datonnino3 the threshold for the nucleus of the leukocyte. Monitors intensity level. Displays messogo if intensity exceeds certain limits. CRT Device handler for the Type 30D CRT display unit. (This handler has boon written) CNTOUR Called by STRPUT. Here is vhero user write3 recognition routines for the leukocytes. There are tvo returns to STRPNT: 1. Coll coordinates okay. 2. Coll coordinates bad. » THRESH and FOCUS may also bo called by the User. Fig. A.2.1 A block diagram demonstrating a possible arrangement of the system programs. 64 Variable Name Hame of Pro gran Where Variables are Defined e Monitor JOISTK FCCUS THRESH STRFNT CNTOUR* GS X A A A (A) AGS X A (A) ca X A TLEVEL X A (A) STPEQ X A FOCSTP X A VAHNUM X A SXD2 X A STD2 X A WCNT X A VARI X Subroutine Namo DELAY. X A A (A). JOYSTK A X •JCYINT A X FOCUS (A) A' A VAR X THRESH (A) X A STRPNT A X CNTOUR A X CRT (A) A . (A) * User written prograns, «» CRT message display routines. This handler is available. Table A.2.1 Internal and external references to global symbols and subroutines. "X" indicates internally defined, "A" indicates external access necessary, and "(A)" indicates external access probable. 65 and in understanding the program listings. The pattern that is used to move the stage about is stored in "PATTRN". STRPNT accesses this table to determine how the stage should be moved. The structure of the pattern may be changed by changing the entries in PATTRN. The following structure is used for PATTRN: PATTRN IOT COUNT IOT COUNT ooo 0 "IOT" is one of the four instructions to move the stage in the four possible directions, and "COUNT" is the number of 10 micron steps to be taken in the direction of the IOT. For example, to completely scan a rectangular pattern of size 100 microns by 2000 microns starting at'the top left hand corner, PATTRN would appear as follows: PATTRN SSXP / +X direction 200 SSYM / -Y direction 10 SSXM / -X direction 200 SSYP / +Y direction 10 0 / Buffer terminator There is no logical restriction on the size of PATTRN. _ A zero must be used to indicate the end of the instruction buffer. 66 APPENDIX III The IOT instructions for the special hardware are listed below. Mnemonic Code Description STCL 707042 Start remote scan of dissector. SPCL 707021 Stop remote scan of dissector. LDX 707002 Load X coordinate of dissector from the accumulator (10 bits). LDY 707022 Load Y coordinate of dissector from the accumulator (10 bits). ADCV 707041 Start A/D conversion on dissector signal. ADSF 707061 Skip if dissector A/D conversion is done. ADRB 707076 Read result of the dissector A/D conversion into the accumulator (6 bits). SSXP 707202 Step the stage in the X positive direction (10 micron step). SSXM 707241 Step the stage in the X negative direction. SSYP 707204 Step the stage in the Y positive direction. SSYM 707242 Step the stage in the Y negative direction. SSKP 707201 Skip if the scanning stage step is done. FCW 707264 Step the focus motor clockwise. FCCW 707262 Step the focus motor counterclockwise. FSKP 707261 Skip if the focus motor step is done. LMUX 707244 Load joystick multiplexer channel from the accumulator (3 bits - 8 channels) . JSTR 707224 Start A/D conversion of joystick signal. JDNE 707221 Skip if joystick A/D conversion is done. JGET 707222 Read result of joystick A/D conversion into the accumulator (6 bits). 67 REFERENCES 1. Young, Ian T., "The Classification of White Blood Cells", IEEE  Transactions on Biomedical Engineering, Vol. BME-19, No. 4, pp. 291-298, July, 1972. 2. Saunders, Alex M., "Development of Automation of Differential Counts by use of Cytochemistry", Clinical Chemistry, Vol. 18, No. 8, pp. 783-788, August, 1972. 3. Dacie, J.C, Lewis, S.M., Practical Haematology, J. & A. Churchill Ltd., London, 1968. 4. Young, Ian T., "Automatic Leukocyte Recognition", Automated Cell  Identification and Cell Sorting, Academic Press, New York, 1970. 5. Weid, George L., Bahr, Gunther F., Bartels, Peter H., "Automatic Analysis of Cell Images by Ticas", Automated Cell Identification  and Cell Sorting, Academic Press, New York, 19 70. 6. Ledley, R.S., "Automatic Pattern Recognition for Clinical Medicine", Proceedings of IEEE, Vol. 57, No. 11, pp. 2007-2020, November, 1969. 7. Stein, P.G., Lipkin, L.E., Shapiro, H.M., "Spectre II: A General Purpose Microscope Input for a Computer", Science, Vol. 166, No. 3903,. pp. 328-335, October, 1969. 8. Cossalter, John G., "A Computer Visual-Input System for the Auto matic Recognition of Blood Cells", M.A.Sc. Thesis, The University of  British Columbia, Department of Electrical Engineering, 1970. 9. Eberhardt, E.H., "Noise in Image Dissector Tubes", Research Memo No. 337, I.T.T. Industrial Laboratories, Indiana, 1960. 10. Eberhardt, E.H., "Singal-to-Noise Ratio in Image Dissectors", Research Memo No. 386, I.T.T. Industrial Laboratories, Indiana, 1960. 11. Hardy, Arthur C, Perrin, Fred H., The Principles of Optics, McGraw-Hill Book Company, Inc., New York, 1932. 12. Mendelsohn, Mortimer L., Mayall, Brian H., "Computer Oriented Analysis of Human Chromosones - III. Focus", Computers in Biology and Medicine, Vol. 2, pp. 137-150, October, 1972. 13. Young, Ian T., "Biological Image Processing - Automated Leukocyte Recognition", M.I.T. Quarterly Progress Report, No. 89, April, 1968. 14. MacGregor, R.C., Scott, R.W., Loh, G.L., "The Differential Leukocyte Count", Journal of Pathology and Bacteriology, Vol. 51, pp. 337-368, 1940. 68 15. Bennett, John R., "On the Measurement of the Curvature of the Boundaries of Two-Dimensional Quantized Shapes", Ph.D. Thesis, The University of British Columbia, Department of Electrical Engi  neering, January, 1972. 16c Ingram, M., "The Perkin-Elmer Instrument for Automatic Analysis of Blood Cell Images", Clinical Chemistry Newsletter, Vol. 4, Mo. 2, pp. 33-37, Winter, 19 72. 17. The Larc System by Corning, pamphlet distributed by Corning Scienti fic Instruments, May, 1973. 18. Li, Jerome C.R., Statistical Inference, Edwards Brother, Inc., Michigan, 1969. 


Citation Scheme:


Usage Statistics

Country Views Downloads
China 11 11
United States 8 2
Indonesia 3 1
India 1 0
Poland 1 0
France 1 0
City Views Downloads
Beijing 9 10
Unknown 5 1
Ashburn 3 0
Shenzhen 2 1
Wilmington 2 0
Feasterville-Trevose 1 0
Yogyakarta 1 0
Redmond 1 0
Chicago 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items