UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Segmentation of overlapping nuclei in lung epithelial biopsy sections Kemp, Roger


This thesis describes the development of an automated image segmentation system for resolving overlapping cell nuclei in crowded scenes such as tissue biopsy section images. The system uses a succession of imaging algorithms that combine to double the number of extracted nuclei from lung epithelial section images compared to selecting free lying nuclei. 1. The system uses the distance transform and watershed transformation to analyse the shape of object clusters and split them into smaller clusters or into individual nuclei. The watershed algorithm reliably separates two overlapping ellipses provided less than 30% of either ellipse's perimeter is occluded by the other. 2. A Hough transform algorithm was created by combining the ellipse center finding routine of Yuen with the least squares ellipse fitting formula of Fitzgibbon. Fitzgibbon's formula was adapted to include a weighting for data points so that strong ellipse edges contribute more in the determination of ellipse fit parameters. The transform was tested on a set of 431 overlapping nuclei in cytological images of lung tumour cell lines grown in culture. The Hough transform was able to produce good ellipse fits for 85% of the nuclei in the set. 3. Active contour refinement is used to refine the borders of objects segmented using the Hough transform. It was applied to the cytological image set and reduced the area misfit measure between the true nuclear mask and the Hough ellipse approximations from 8 ± 4% to 4 ± 2%. The final segmentation of the nuclei created borders that delineated the overlap regions between nuclear pairs. These overlap regions were then measured for cytological and histological images to determine if the mean optical density (OD) in non-overlap regions could be used to predict the mean OD in overlap regions. It was found that the overlap regions contain 60-70% of the predicted OD. This result was used as an empirical factor in the development of an OD apportioning scheme for the reconstruction of individual nuclei from overlapping pairs based on a maximum likelihood probability model. The complete segmentation system was used to automatically segment biopsy section images for the purpose of recovering intact nuclei for morphometric analysis. Experiments on a set of ten tissue section images revealed that an average of 55 free lying nuclear shaped objects can be extracted from typical section frames, 83 can be extracted by applying the watershed algorithm and 102 can be extracted using the complete segmentation system. An experiment was performed on a set of nine biopsy section images that had been manually segmented and given a score using a morphometric index (MI) scoring system, which categorizes image frames based on nuclear irregularity. A decision tree was created to select "valid" nuclei based on their feature values and these nuclei were then classified with the existing MI system. The automated MI scores were compared to the manual ones and agreed for three of the nine cases. The automated system was less likely to identify abnormal appearing nuclei than normal ones. This caused it to disagree with the manual MI system for images containing severe abnormalities. The lack of success in the MI experiment is due the difficulty of combining a decision tree which attempts to throw out irregular objects with a second decision tree that seeks to categorize them. Further work in the classifier design may yield an automated biopsy section analysis system which performs as well as humans.

Item Media

Item Citations and Data


For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.