UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Registration of 3D ultrasound to computed tomography images of the kidney Xiang, Jing 2010

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2010_fall_xiang_jing.pdf [ 1.31MB ]
Metadata
JSON: 24-1.0071184.json
JSON-LD: 24-1.0071184-ld.json
RDF/XML (Pretty): 24-1.0071184-rdf.xml
RDF/JSON: 24-1.0071184-rdf.json
Turtle: 24-1.0071184-turtle.txt
N-Triples: 24-1.0071184-rdf-ntriples.txt
Original Record: 24-1.0071184-source.json
Full Text
24-1.0071184-fulltext.txt
Citation
24-1.0071184.ris

Full Text

Registration of 3D Ultrasound to Computed Tomography Images of the Kidney by Jing Xiang B.Comp., Queen’s University, 2008 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in The Faculty of Graduate Studies (Biomedical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) July 2010 c© Jing Xiang 2010 Abstract The integration of 3D computed tomography (CT) and ultrasound (US) is of considerable interest because it can potentially improve many minimally invasive procedures such as robot-assisted laparoscopic partial nephrectomy. Partial nephrectomy patients often receive preoperative CT angiography for diagnosis. The 3D CT image is of high quality and has a large field of view. Intraoperatively, dynamic real-time images are acquired using ultrasound. While US is real-time and safe for frequent imaging, the images captured are noisy and only provide a limited perspective. Providing accurate registration between the two modalities would enhance navigation and image guidance for the surgeon because it can bring the pre-operative CT into a current view of the patient provided by US. The challenging aspect of this registration problem is that US and CT produce very different images. Thus, a recurring strategy is to use pre- processing techniques to highlight the similar elements between the images. The registration technique presented here goes further by dynamically sim- ulating an US image from the CT, and registering the simulated image to the actual US. This is validated on US and CT volumes of porcine phan- tom data. Validation on realistic phantoms remains an ongoing problem in the development of registration methods. A detailed protocol is presented ii Abstract here for constructing tissue phantoms that incorporate contrast agent into the tissue such that the kidneys appear representative of in vivo human CT angiography. Registration with 3D CT is performed successfully on the reconstructed 3D US volumes, and the mean TREs ranged from 1.8 to 3.5 mm. In addition, the simulation-based algorithm was revised to consider the shape of the US beam by using pre-scan converted US data. The cor- responding CT image is iteratively interpolated along the direction of the US beam during simulation. The mean TREs resulting from registering the pre-scan US data and CT data were between 1.4 to 2.6 mm. The results show that both methods yield similar results and are promising for clinical application. Finally, the method is tested on a set of in vivo CT and US images of a partial nephrectomy patient, and the registration results are discussed. iii Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . x Statement of Co-Authorship . . . . . . . . . . . . . . . . . . . . . xii 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Thesis Objectives . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1 Anatomy of the Kidney . . . . . . . . . . . . . . . . . . . . . 9 2.2 Overview of Medical Image Registration . . . . . . . . . . . . 10 2.3 Multimodality Registration . . . . . . . . . . . . . . . . . . . 13 iv Table of Contents 2.3.1 US To CT Registration of Bone . . . . . . . . . . . . 14 2.3.2 US To CT Registration of Soft Tissue . . . . . . . . . 20 2.3.3 US Simulation-Based Registration of US and CT . . 25 2.4 Validation of Registration Methods on Phantom Data . . . . 27 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . 31 3.1 Phantom Data Acquisition . . . . . . . . . . . . . . . . . . . 32 3.1.1 Procedure for Kidney Phantom Construction . . . . . 32 3.1.2 CT Acquisition . . . . . . . . . . . . . . . . . . . . . 38 3.1.3 Ultrasound Acquisition . . . . . . . . . . . . . . . . . 39 3.1.4 Defining the Gold Standard Alignment . . . . . . . . 40 3.1.5 Method of Error Analysis . . . . . . . . . . . . . . . . 40 3.1.6 Scan Conversion Correction . . . . . . . . . . . . . . 40 3.2 Clinical Data Acquisition . . . . . . . . . . . . . . . . . . . . 45 3.2.1 Positioning the Patient for Imaging . . . . . . . . . . 46 3.2.2 Preoperative US and CT Acquisition . . . . . . . . . 47 3.2.3 Finding the Bronze Standard Alignment of the US and CT images . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2.4 Method of Error Analysis . . . . . . . . . . . . . . . . 49 3.2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 51 4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.1 Registration via the Simulated Ultrasound . . . . . . . . . . 54 4.1.1 Creating the Simulated Ultrasound . . . . . . . . . . 56 4.2 Registration using Scan Converted US Data . . . . . . . . . 60 v Table of Contents 4.3 Registration using Pre-scan Converted US Data . . . . . . . 62 5 Registration Results and Discussion . . . . . . . . . . . . . . 70 5.1 Porcine Kidney Phantom Data . . . . . . . . . . . . . . . . . 70 5.1.1 Registration using Scan Converted US Phantom Data 70 5.1.2 Registration Results using Pre-scan Converted US Phan- tom Data . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . 72 5.2 Patient Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.1 Summary of Contributions . . . . . . . . . . . . . . . . . . . 81 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Appendices A UBC Research Ethics Board Certificate . . . . . . . . . . . 98 vi List of Tables 5.1 Registration Results for Scan Converted US and CT . . . . . 71 5.2 Registration Results for Pre-scan Converted US and Inverse Scan Converted CT . . . . . . . . . . . . . . . . . . . . . . . . 73 vii List of Figures 1.1 Overview of the Integration of Medical Imaging in Robot- Assisted Partial Nephrectomy. . . . . . . . . . . . . . . . . . . 4 2.1 CT Angiogram of Human Patient. . . . . . . . . . . . . . . . 10 2.2 Anatomy of the Kidney. . . . . . . . . . . . . . . . . . . . . . 11 3.1 Comparison of Ultrasound images of Kidney Phantoms. . . . 33 3.2 Comparison of CT images of Kidney Phantoms. . . . . . . . . 34 3.3 Preparation of the Porcine Kidney. . . . . . . . . . . . . . . . 36 3.4 Positioning the Porcine Kidney in Agar. . . . . . . . . . . . . 37 3.5 Steel Ball Fiducial Markers. . . . . . . . . . . . . . . . . . . . 38 3.6 Setup for Ultrasound Acquisition. . . . . . . . . . . . . . . . . 39 3.7 Photograph of the N-wire phantom. . . . . . . . . . . . . . . 42 3.8 US Slice Showing Segmented Wires. . . . . . . . . . . . . . . 43 3.9 Geometry of the N-wire Phantom. . . . . . . . . . . . . . . . 44 3.10 Plot of the Average Angles for the Slower Half of 3D Probe Sweep. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.11 Plot of the Average Angles for Half of 3D Probe Sweep with the Expected Speed. . . . . . . . . . . . . . . . . . . . . . . . 46 viii List of Figures 3.12 Positioning of a Patient for Imaging. . . . . . . . . . . . . . . 47 3.13 Overlay of the CT and US image of a Renal Cancer Patient. . 49 3.14 CT and US images of Patient Data Aligned using PCA. . . . 50 4.1 Overview of the Methods of Simulation. . . . . . . . . . . . . 54 4.2 US to CT Registration Workflow. . . . . . . . . . . . . . . . . 55 4.3 Components of the Simulated Ultrasound. . . . . . . . . . . . 64 4.4 Diagrammatic Representation of Pre-scan Converted US data. 65 4.5 Geometry of Scan Conversion: The XY Sector. . . . . . . . . 66 4.6 Geometry of Scan Conversion: The YZ Sector. . . . . . . . . 67 4.7 3D Scan Converted Volume . . . . . . . . . . . . . . . . . . . 68 4.8 Pre-scan Converted US and Corresponding Inverse Scan Con- verted CT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.9 Geometric Explanation of how the Inverse Scan Converted CT is Generated. . . . . . . . . . . . . . . . . . . . . . . . . . 69 5.1 The Patient US and Simulated US . . . . . . . . . . . . . . . 76 5.2 Results for Post-scan Converted Patient Data. . . . . . . . . . 77 5.3 Results for Pre-scan Converted Patient Data. . . . . . . . . . 78 5.4 Scatter plot showing the Final TRE vs. the Initial Misalign- ment for both Registration Techniques. . . . . . . . . . . . . . 79 ix Acknowledgements First and foremost, I must thank my supervisors Dr. Robert Rohling and Dr. Purang Abolmaesumi. They are both consummate professionals and I have truly benefitted from their support and expertise. Thanks to Dr. Chris Nguan for his clinical knowledge and enthusiasm that has supported the clinical aspects of the project. Thanks to all the techs, Dean Malpas, Chris Eddy and Vickie Lessoway that made the data collection possible. Special thanks to Eric Pospisil and Sean Gill for answering ALL my questions, to Troy Adebar for his help with some of the illustrations, and to Caitlin Schneider for being my partner in coordinating the clinical studies. I would also like to thank Dr. Sathish Gopalakrishnan and Dr. Parvin Mousavi for their counsel and encouragement, and for challenging me to go further with my academic career. To my colleagues in the Robotics and Control Lab, Jeff Abeysekera, Caitlin Schneider, Raoul Kingma, Troy Adebar, Michael Yip, John Bartlett, Ali Baghani, Denis Tran and Diego Prananta, it’s been a slice! Their friend- ship and support will be missed. To my Vancouver friends: Ana Crisan, Joe Hall, AndreaWilliams, Mehdi Moradi and Paria Abolmaesumi, thanks for everything that happened of- x Acknowledgements fline. Finally, thank you to my parents Drs. Yang Xiang and Jing Yu Zhu. I wouldn’t be here without their constant love, support, and inspiration. With that said, I am looking forward to life without boxes for awhile. xi Statement of Co-Authorship This thesis was prepared under the supervision and guidance of Dr. Robert Rohling and Dr. Purang Abolmaesumi. They provided the research topic of multimodality registration of US to CT for the application of robot-assisted partial nephrectomy and suggested using a simulated US approach. They provided ongoing feedback as the research progressed and assisted with revi- sions of manuscripts such as the conference paper and this thesis. Otherwise, the contents in this thesis are original material by the author. The data acquisition was facilitated by medical personnel including Dr. Chris Nguan (MD, Dept. of Urologic Sciences, Vancouver General Hospital), Victoria Lessoway (RDMS, Dept. of Ultrasound, British Columbia Women’s Hospital and Health Center), Stephanie Smith (AHT and RVT, Jack Bell Research Centre) and Dean Malpas (CT Technologist, Canada Diagnostic Centers). Part of Chapter 3 appeared in SPIE Medical Imaging [70]. This ar- ticle was co-authored with Sean Gill, Chris Nguan, Purang Abolmaesumi and Robert Rohling. Sean Gill provided most of the C++ code used for registration. Chris Nguan contributed suggestions on constructing the soft tissue phantoms. The author built the phantoms and acquired ultrasound data. The author modified the C++ program for the soft tissue application xii Statement of Co-Authorship and tested the registration procedure on the acquired data sets. The au- thor also wrote the conference paper. Dr. Robert Rohling and Dr. Purang Abolmaesumi assisted with revisions for the manuscript. xiii Chapter 1 Introduction 1.1 Motivation As medical technology advances, a greater number of surgical procedures are becoming minimally invasive. In laparoscopy, small incisions are made on the skin of the patient’s abdomen. The surgeon then inserts ports such that a video camera and surgical tools can be introduced and conduct the surgical operation. By using minimally invasive techniques, patients receive many benefits including less scarring and injury to the tissue, and reduced post-operative pain. This all contributes to a shorter hospital stays and faster recovery time for patients. Recently, there has been considerable interest in laparoscopic partial nephrectomy (LPN) and robot-assisted LPNs for the resection of tumours of the kidney. Kidney cancer refers to tumours that occur in the renal parenchyma of the kidney, and approximately 80% of these cancers are re- nal cell carcinomas. According to Canadian Cancer Statistics 2010, kidney cancer is the 10th most common newly diagnosed cancer and the 13th lead- ing cause of cancer death [18]. In Canada in 2010, it is estimated that it will be responsible for 2800 new cancer cases and 1650 deaths. Amongst Canadians, kidney cancer is the sixth most common cancer in men and the 1 1.1. Motivation tenth most common in women [4]. In addition, the incidence rate of kidney cancer has increased by approximately 1.3% per year for both males and females since the late 1990s [18]. The standard of care for a patient who is suspected of having kidney cancer is to obtain a diagnostic computed tomography (CT) scan. The CT confirms the presence of the tumour and is used for surgical planning in the case that the tumour requires surgical resection. The patient often has CT angiography performed which involves getting a small dose of contrast agent injected intravenously at the time of the CT. This type of imaging highlights the vascular anatomy and the relative position of the tumour to normal parenchyma. Partial nephrectomy, also referred to as nephron spar- ing surgery, is where only the section of the kidney containing the tumour is removed. It is preferred over radical nephrectomy, where the whole kidney is excised, because it controls the cancer while preserving renal function. Currently, it is widely accepted as the standard treatment for small (≤4 cm) renal tumours [41] [44]. In addition, some studies have demonstrated that partial nephrectomy is equivalent to radical nephrectomy for tumours of 4-7 cm [1] [50] [52]. Furthermore, laparoscopic partial nephrectomy has increased in popu- larity since it was originally proposed in 1991 [17]. In comparative studies between laparoscopic and open procedures, researchers have suggested that while laparoscopic partial nephrectomy requires more technical experience of the surgeon, it maintains comparative functional and oncological out- comes [27] [58]. Our goal is to overcome some of the technical challenges of LPN by integrating various medical imaging modalities. Because laparo- 2 1.1. Motivation scopic cameras only allow surgeons to view the surface of tissues inside the peritoneum, visualization of the anatomy beneath including blood vessels and the target tumour would improve navigation during the surgery. CT provides a complete anatomical map of the patient but may be out of date, whereas US shows real-time updated images of the tissue, but has a smaller field of view. The fusion of these modalities would be useful. This can be performed by using image registration techniques, methods that integrate multiple images by transforming them into the same coordinate system. Re- cently, there has been some work published on registering video to CT [61] and to US [16] particularly for robot-assisted partial nephrectomy. Thus, the automatic registration of US to CT would make it possible to display both US and CT with video, which has the potential to improve the efficiency and success of these procedures. US to CT registration of the kidney is a difficult problem because imaging the kidney with US produces very different images than those created with CT. Thus, the challenge is to extract similar components in both modali- ties and use them for registration. In addition to the disparity between US and CT images, sources of registration error exist when working with pa- tient data. The preoperative CT is taken with the patient in flank position and on breath hold. However, these conditions change for the acquisition of intraoperative US and thus the image obtained may be slightly different. During the operation, the patient may not be in exactly the same position, the abdomen is insufflated, and the patient is breathing. Surgical manip- ulation of the kidney and surrounding area as well as pressure from the ultrasound probe may also cause a difference in the US from the CT. It is 3 1.2. Thesis Objectives useful to be aware of these issues as we tackle this registration problem. 1.2 Thesis Objectives Figure 1.1: Overview of how the US and CT image guidance can be in- tegrated into robot-assisted partial nephrectomy. The long term goal is to have tracked US captured in the robot coordinate space. The CT is then registered to the US. Thus, both images are available to the surgeon during the operation. The focus of this thesis is the dashed area of the figure, which addresses the problem of CT to US registration. The research presented in this thesis is part of a larger project to inte- grate 3D image guidance with robot-assisted laparoscopic partial nephrec- tomy. As shown in Figure 1.1, the objective is to register the CT image 4 1.2. Thesis Objectives to the US image. During the operation, tracked US is acquired so that the US can be captured in robot coordinate space. The registered images are then provided to the surgeon during the procedure. The focus of this thesis is described in the dashed area of the figure. It is to investigate and de- velop different approaches to registering US and CT, such that an efficient and accurate method can be used during the robot-assisted operation. The development of these techniques has the potential to provide a comprehen- sive anatomical map, combined with real-time US image guidance, to the surgeon during the procedure. Since the camera can only depict the tissue surface, providing surgeons with registered images of blood vessels, tumour boundaries, and organ structures beneath the surface can improve the speed and accuracy of robot-assisted nephrectomies. Specifically, the objective of this thesis is to develop a robust method for the multimodality registration of US and CT images of the kidney by extending previous work that used a simple physics-based model to dynamically simulate US. This goal is divided into several smaller objectives outlined below: 1. Provide a suitable validation platform for the registration method. This entails developing a suitable soft tissue phantom that can create realistic images in both US and CT and provide a gold standard for alignment. 2. Develop a registration method to align US and CT images of kidney using rigid-body registration. The method must be able to register US 5 1.3. Contributions and CT images of the kidney, must be fully automatic, and must not require prior knowledge of the direction of US scanning. 3. Examine two approaches to US simulation-based registration that dif- fer in how the ultrasound simulation is created. 4. Acquire clinical data and demonstrate how the registration performs with the additional challenges that arise with real subjects. While the data collected for this thesis was inspired by the robot-assisted laparoscopic partial nephrectomy application, the registration techniques developed can be applied to a broader range of applications where US to CT registration of the kidney is required. 1.3 Contributions The contributions of this thesis include: 1. Developed realistic phantoms for validating US to CT registration of the kidney using fiducials as a gold standard. 2. Implemented and evaluated two volume-to-volume registration meth- ods for aligning US and CT images of the kidney. The methods differed 6 1.4. Thesis Outline in the way US images were simulated from CT during the registration process. 3. Validated the performance of the registration techniques on a limited set of clinical data. 1.4 Thesis Outline This thesis covers the relevant literature and previous research that has brought about the proposed registration method, the experimental design and registration framework and the results for phantom and clinical data. An outline of the thesis is as follows: Chapter 2 Background provides a brief overview of medical image regis- tration, and discusses previous work on US to CT registration for a variety of image-guided surgical applications. Chapter 3 Experimental Design describes the protocols followed for data acquisition including the construction of soft tissue phantoms and col- lection of clinical data. It also outlines how corresponding US and CT datasets were acquired. Chapter 4 Methods discusses how the simulated US is generated from the CT, and how it is used in the rigid registration framework. This section 7 1.4. Thesis Outline also presents two methods of dynamically simulating US within the regis- tration process. Chapter 5 Results details the quantitative results from all registration tests and discusses the results obtained from phantom data and patient data. Chapter 6 Conclusion summarizes the thesis and highlights the main conclusions that can be drawn from the study. It also suggests possible fu- ture work in improving the algorithms and modifying the implementation to increase speed and accuracy. 8 Chapter 2 Background 2.1 Anatomy of the Kidney The target of laparoscopic partial nephrectomy is the kidney. The kidneys are located in the abdomen below the lungs. The left kidney sits below the spleen while the right kidney is situated underneath the liver. This is shown in a CT angiogram of a cancer patient (Fig. 2.1). This particular patient has a tumour on the inferior side of the right kidney. There are two main tissue regions in the kidney. The renal cortex is the outer section of the kidney and appears bright in CT angiography (Fig. 2.1 and 2.2). It is surrounded by the renal capsule which is a membrane that encapsulates the kidney. The renal medulla is the inner part of the kidney and is divided into sections called the renal pyramids. These pyramids contain the functional structures of the kidney called nephrons that are responsible for filtration of the blood. The renal artery and vein transport blood into and out of the kidney (Fig. 2.2). It is important to note that during surgery, these vessels are clamped so that blood flow is stopped before resecting the tumour. Finally, the ureter drains to the bladder. 9 2.2. Overview of Medical Image Registration Figure 2.1: CT angiogram from a human patient showing the healthy left kidney on the right side of the image and the tumourous right kidney on the left. 2.2 Overview of Medical Image Registration The acquisition and analysis of medical images is crucial to a large num- ber of clinical applications. In many cases, medical imaging is an integral part of the entire clinical work flow including diagnosis, planning and the final execution of the interventional procedure. It is especially important in minimally invasive surgery because medical imaging gives surgeons vision of structures that they would otherwise not see. Presently, there are a variety of different imaging modalities employed including X-ray, computed tomog- raphy (CT), magnetic resonance (MR) and ultrasound (US). The various techniques differ in cost and each has its advantages and shortcomings. For 10 2.2. Overview of Medical Image Registration Figure 2.2: A depiction of the anatomy of the kidney including the different regions of tissue and vascular structures. instance, MR and CT machines produce high quality images with a large field of view. While they are heavily utilized for diagnostic purposes, it may not be practical to use them within the operating room (OR). Also, in the case of CT, it may be hazardous to the patient to expose them frequently to ionizing radiation during surgery. However, while US does not offer the same type of tissue contrast and field of view, it is real-time, portable and safe to use on a patient repeatedly. A specific clinical case often has multiple images acquired at different times or from different viewpoints. It can also have images taken with dif- 11 2.2. Overview of Medical Image Registration fering imaging modalities. Thus, the integration of the various sources of information is often useful. The integration process involves bringing the different images into a spatial alignment; this is referred to as registration. More specifically, registration is the estimation of the correct mapping be- tween the coordinate systems of two images such that the images will share the same coordinate frame [5]. Image registration continues to be an area of ongoing research. In 1992, A comprehensive survey of registration methods was published by Brown [12]. Many thorough reviews followed describing the state-of-the-art registration algorithms and applications [5] [31] [34] [43] [74]. Defining an image registration method can be thought of as defining three basic components, the transform, the optimizer and the similarity metric. First, the type of transformation must be selected. The simplest case is rigid registration which consists of six parameters (for 3D data) that represent three rotations and three translations. Affine transformation al- lows for scaling and shearing of the image, and deformable registration finds non-rigid curved transformations between images. Next, the optimization strategy must be chosen to optimize the parameters of the transform. If the search space is smooth, using an algorithm such as gradient descent may be appropriate. If the search space is noisier, then a stochastic approach may be suitable such as using genetic algorithms. Finally, the most defining feature of a registration method is what it uses as similarity metric. This is the criteria for finding or evaluating the alignment between two images. The choice of similarity metric is important and will largely influence whether the registration is robust. Registration methods are often classified by the type of similarity metric 12 2.3. Multimodality Registration that is employed. There are two main categories of metrics: feature-based registration and intensity-based registration. The two categories of regis- tration algorithms are often implemented differently. Feature-based regis- tration evaluates how alike two images are by finding matching geometrical features such as points, lines and surfaces. The transform T is found by determining the mapping between corresponding features. Iterative clos- est point [8] (ICP) is a method that is commonly applied to datasets with point-based features and is used widely in multimodality registration. These methods differ from intensity-based registration where T is optimized based on a similarity metric that is calculated by utilizing the information within each voxel of the images. Many common intensity-based multimodality reg- istration approaches uses the mutual information similarity metric [53] [63]. In general, intensity-based registration is more computationally expensive than feature-based. However, it is successful in particular applications where feature-based algorithms fail. Constructing a suitable registration algorithm largely depends on the characteristics of the medical images acquired for the clinical application. 2.3 Multimodality Registration The choice of registration algorithms is largely influenced by the type of med- ical images that are being aligned and whether monomodality registration or multimodality registration method is required. Monomodality registration is used when the images to be aligned are both taken with the same medical imaging modality, such as two CT images. However, in this study, CT and 13 2.3. Multimodality Registration US images are taken for each patient requiring a multimodality registration. Thus, the rest of this section will discuss multimodality methods and appli- cations specifically focusing on the registration of US to other modalities. Both soft tissue and bone applications are discussed. 2.3.1 US To CT Registration of Bone The multimodality registration of bone is an area of keen investigation. The registration of US to CT imaging of bone is pertinent because of the wide application to image-guided orthopedic procedures including interventions that involve the knee, spine, femur and pelvis. Anatomical structures such as bone are ideal for registration because the structures are hard and rigid. This allows the registration problem to be formulated as a rigid registration. The common challenges of registering US images of bone is handling the occlusion of soft tissue and shadows that are caused by total reflection of the US signal. Feature-based Methods The trend amongst early work in the registration of bone was to use feature- based registration. Specifically, point-based and surface-based methods were popular. Because bone is rigid and does not deform like soft tissue, it is common to collect surface points and use ICP [8]. However, there are certain challenges with this approach. First, while CT produces clear high quality images of bone, US is noisy and often only captures sections of the bone. Thus, identifying the surfaces automatically or manually is difficult and unreliable. In addition, the success of ICP is dependent on a good initial 14 2.3. Multimodality Registration alignment. One of the early studies was done by Herring et. al [30] where the goal was to register preoperative CT of the spine to intraoperatively acquired tracked US in order to improve the accuracy of pedicle screw placement. Data was acquired from a plastic spine phantom and US surfaces was ex- tracted using a morphological open operator followed by a linear threshold and a ray-tracing algorithm. Surface-based registration using a variation of ICP was performed. Fidicial markers were used as a the gold standard and the authors obtained submillimeter accuracy. Muratore et al. [47] carried out a similar study with a spine phantom and fiducial markers for valida- tion. The US surfaces were extracted using the same approach as Herring et al. and the CT surfaces were extracted using a modified Marching Cubes algorithm. A TRE of 1-3 mm was achieved depending on the slice thickness of the CT. Carrat et al. [13] and Tonetti et. al [62] applied CT to US reg- istration to a similar application, the percutaneous placement of iliosacral screws. Bone surfaces in US were manually extracted from cadaveric and clinical data. Surface-based registration was employed using a method sim- ilar to ICP, but used more sophisticated distance maps to determine the closest point to the surface. Segmentation of the US images was performed to build 2D curves which were then converted to 3D points clouds that were registered to the CT. While the authors did not provide quantitative errors, they suggested that their system was an improvement on the traditional procedure under fluoroscopy guidance, which is real-time image guidance using an x-ray source. Amin et al. collected US and CT data from a pelvic phantom and from patients intraoperatively. The datasets were also regis- 15 2.3. Multimodality Registration tered with ICP but their approach differed from previous methods. The 2D regions in US most likely to be bone surface were extracted. During ICP, prior information about the bone surface in US including location, intensity and edges, was used within the algorithm. For the phantom experiments, the translation error was reported to be less than 1 mm and the rotational error less than 2◦ in each axis as compared with a fiducial-based ground truth. For the intraoperative patient data, no absolute errors were reported but a point-based registration was performed for comparison of accuracy. The difference in error between the ICP registration and the point-based registration in translation was approximately 2 mm and in rotation was 1.6◦ in each axis. It should be noted that the spatial prior is dependent on the initial registration estimate making the initial alignment important for the algorithm’s success. Recent studies have moved away from ICP as the primary component of registration. Barratt et al. [6] developed a method for simultaneous reg- istration and calibration of the US probe for in vivo data. A non-linear optimization scheme was used to perform a least-squares minimization of the distances between the CT and US surface points. The authors claim that updating the calibration throughout the registration process produces less error than doing calibration on a phantom ahead of time especially since we cannot always assume the ultrasound to be a constant speed of 1540 m/s in human tissue. The method was tested on manually segmented US and CT surfaces from images of the pelvis and femur, and the self-calibrated procedure was reported to reduce the TRE for the whole bone from 2.2 mm to 1.59 mm, a reduction of 28%. As the authors mention, the current 16 2.3. Multimodality Registration implementation is a point-based system that requires manual segmentation. An extension to image-based registration would enable further automation of the procedure. Moghari and Abolmaesumi [46] also advanced the reg- istration process by introducing the Unscented Kalman Filter (UKF) as a substitute for ICP. US and CT data of a phantom scaphoid bone surface were used to test the registration algorithms. US surface points from the bone were manually selected. Compared to a root mean square distance error of 3.7 mm, the UKF results in an error reduction to 1.4 mm. To summarize, there have been algorithm improvements in recent feature-based registration studies. However, most still contain manual steps that are not repeatable and difficult to implement automatically. Intensity-based Methods Intensity-based registration focuses on matching individual pixels or voxels rather than specific features in the images. While it is often more com- putationally expensive, it can exploit additional information that would be thrown away in feature-based approaches. Intensity-based methods are of- ten more automatic because no segmentation of contours or selection of points are required prior to registration. For example, Brendel et al. [9] and Winter et al. [68] use a surface-based registration of spine data, but the surface was extracted by thresholding the CT, as opposed to point selection or segmentation. After thresholding, the surface was filtered such that only those elements that were visible from the skin and nearly perpendicular to the skin were kept. The surface points extracted from the CT were then pro- jected onto the ultrasound and the intensity of the overlapping voxels are 17 2.3. Multimodality Registration optimized. These authors exploit the fact that the tissue-bone interface ap- pears bright in ultrasound because of total reflection. Winter et al. showed qualitative results and Brendel et al. reported less than 1 mm translational and 1◦ rotational deviation from the reference registration. A variation on this approach is later presented by Brendel et al. [10] and also used by Win- ter et al. [69]. Preprocessing of the US involved using an adaptive depth gain compensation technique to highlight the bone surface. The bone sur- face in CT is estimated by a combination of thresholding and filtering with the consideration of the US scan path as per their the authors’ previous work and a similar optimization strategy is employed. It is important to note that for any registration methods with this surface to volume tecnique, the orientation and scanning path of the transducer must be known prior to registration to allow the surface to be properly filtered. Recently Dekomien et al [19] applied the same method to US and MR data of the knee. T2- weighted MR images were used because the sequence proved to have the highest bone-tissue contrast. The results showed that 99.2% of the registra- tions had less than 1 mm error from the optimum. Shao et al. [60] also used a surface to image registration on TRUS (transrectal ultrasound) and T2- weighted MR images of the pubic arch. Shao et al. compared the results of optimizing three different similarity measures: the overlapping surface and image intensities, the surface normals and image gradients, and the coincid- ing surface and shadow intensities. It was determined that incorporating the shadow information improved the registration accuracy from simply using intensities or gradients. The previous intensity-based methods discussed have all used the bone 18 2.3. Multimodality Registration surface to facilitate the registration. The challenge of intensity-based meth- ods for CT to US registration without utilizing surfaces, is that the two modalities produce images that appear very different. In particular the noise and speckle artifact in US deters the simple mapping to the corresponding pixels in CT. Therefore, it seems reasonable to modify either the US or CT such that the two images look more similar. Huang et al. [33] preprocessed both the US and the CT before registering images of a rib cage phantom. Aligning the rib cage served as an initialization for the registration of the beating heart. Preprocessing of the US involved thresholding such that only the top surface of the ribs remained in the image. A more complicated pro- cedure is performed on the CT involving morphological erosion and gradient filtering for which the direction of US scanning is required. Mutual informa- tion (MI) and cross correlation (CC) similarity metrics are then used with gradient descent optimization to register the images and a TRE of 2.5 mm was reported. Motivated by the placement of ablation needles, von Berg et al. [64] designed a similar registration framework for images of an abdominal phantom. The US is filtered to reduce noise and the CT is modified by a ray casting approach such that only bones are highlighted and the interior regions are masked. Like previous methods discussed, the ray source was estimated from ultrasound calibration and thus was only as accurate as the initialization. CC and MI was used in conjunction with the simplex method for optimization and achieved an accuracy of 3.4 mm. Penney et al. [51] presented a novel method for registering pelvis and femur where both the US and CT images were converted to a common type of image and then registered. The US and CT images were converted from intensity images to 19 2.3. Multimodality Registration probability images where each pixel or voxel represented the probability of it containing a bone to soft tissue interface. The presence of certain features were used to determine the probability. For CT, the features were the gra- dient and the maximum value in the surrounding region. The probability of the tissue-bone interface in US was deduced from the intensity of the US and the number of pixels beneath that were not identified as artifact. For each type of image, the probability of a voxel being a tissue-bone interface was equal to the number of voxels that were on the interface with both features divided by the number of voxels in the image with both features. This algo- rithm produced successful registrations as the mean TRE was reported to be less than 2.3 mm. However, a large amount of training data was prepared in order to calculate the probabilities. Not only does this require manual intervention, but the probabilities generated by the training data may not always extend well to test data. 2.3.2 US To CT Registration of Soft Tissue Multimodality registration of soft tissue is a topic of interest because it has the potential to improve the integration of imaging in many soft-tissue applications. Much like surgical interventions with bone, a patient will of- ten receive a preoperative MR or CT. US registration to other modalities is particularly useful because it would consolidate real-time intraoperative information with the preoperative data. Compared to the registration of bone, soft tissue poses extra challenges in that the tissue can deform be- tween corresponding images. However, US images of soft tissue often show more detail throughout the organ or structure since there aren’t large areas 20 2.3. Multimodality Registration that are occluded. Feature-based Methods As previously mentioned, soft tissue does not create bright outlines in ultra- sound that allow the highlighted surface to be exploited as a landmark for registration. Instead, other structures are used such as the vessels. Porter et al. [54] investigate the use of vessels to align US and MR images of the prostate or liver. The vessels in the MR were segmented using threshold- ing or region growing algorithms. In the US, the vessels were segmented using a combination of thresholding and morphological operators. A rough manual alignment followed by an automated correlation-based registration was performed on a prostate phantom data set and in vivo liver data, and achieved under 4 mm of displacement error that was measured from the sur- faces and edges. One of the challenges of this approach it that segmenting structures from US is difficult. Lange et al. [39] also used landmarks, specif- ically the branch points of vessels, to facilitate registration. However, the vessels in the US were identified manually since they were unable to achieve robust automatic segmentation. The non-rigid registration of liver US and CT volumes was formulated as a constraint optimization problem that com- bined landmark and intensity information. The authors demonstrated that their method obtained an improvement on simply rigid registration using the landmarks. Olesch et al. [49] used a similar method except that an ad- ditional constraint was included allowing the user to regulate the accuracy of the alignment of landmarks. In addition to work on the liver, several groups have investigated US to CT registration of the prostate. Firle et al. [20] 21 2.3. Multimodality Registration registered images from prostate phantoms by first segmenting the urethral contour from corresponding datasets and then registering the two sets of points using ICP. They then compared this method to using distance maps calculated from the contoured volumes and optimized by simulated anneal- ing. They demonstrated that using the second method with distance maps achieved a higher accuracy. Another interesting multimodality registration problem is described in Narayanan et al. [48], and that is the registration of US to histology. This group suggested that guidance of prostate biopsy procedures would be improved if the information contained within the 3D statistical prostate atlas can be extracted by registration to the intraoper- ative ultrasound. To achieve this, the prostate was first segmented from the US using a semiautomatic method and then registered to the statistical atlas via a shaped based registration followed by elastic warping of the atlas volume. Feature-based methods for soft tissue often face similar drawbacks as those for bone, particularly the necessity for manual segmentation or selection of landmarks in the registration process. In addition, for certain applications such as registration of the kidney, it may be difficult to segment structures such as vessels because they are not as defined in CT or US. Intensity-based Methods Intensity-based methods allow registration methods to become more auto- matic. Segmentation and manual selection of features can be avoided. This is favourable when working with ultrasound images where identifying con- tours or features is often difficult because of noise and artifacts. When reviewing the literature on CT to US intensity-based registration, it is also 22 2.3. Multimodality Registration useful to examine the similar problem of MR to US image registration. Sev- eral intensity-based methods use Correlation Ratio (CR) or a variation of it as the similarity metric. For example, Roche et al. [57] registered MR and US data of the brain by using an extension of the CR. The authors proposed the Bivariate CR which incorporates both intensity and gradient informa- tion. The justification for this was that US images are in essence, gradient images, because the US signal is high at tissue interfaces. The registration method was optimized using the gradient descent search strategy and its success was evaluated visually. Leroy et al. [40] also used CR but employed a different optimizer, the Powell-Brent method on kidney data. Both the US and CT kidney images were heavily preprocessed. The US was filtered to remove speckle and emphasize tissue boundaries. Acoustic shadows were also removed. Median blurring and gradient filtering were applied separately to the CT and the resulting images were superimposed. The purpose was to emphasize the tissue boundaries while smoothing the rest of the image. The registration resulted in an average distance of 5.36 mm from their bronze standard. Because of the success of the mutual information similarity metric with multimodality applications, many groups have implemented it for registra- tion of MR or CT to US images. Firle et al. [21] obtained an accurate alignment of phantom US and CT data by using MI as the metric with a simulated annealing search strategy. A manually defined bounding box was used to specify a region of interest in the CT. While high accuracy was obtained with error consistently below 2 mm, there was no clinical data pre- sented and the phantom images are not representative of human data. Chan 23 2.3. Multimodality Registration et al. [14] applied MI-based registration to MR and US images of isolated carotid arteries. A rigid registration was followed by a non-rigid registration, both of which used normalized mutual information (NMI) as the similarity metric. The group achieved mean registration errors on the order of 1 mm on the ex vivo data. Similar to the previous study, it is difficult to assess how the algorithm would perform in vivo. One of the drawbacks of the MI metric is its greater computation time as compared with most of the feature-based methods. For real-time registration in an intraoperative situ- ation, strategies for speeding up the algorithm must be considered. This is addressed by Huang et al. [33]. US and MR volumes were preprocessed to create masked images containing the most representative features such as the cardiac chamber walls. The volumes were then subsampled randomly from the highlighted features and used in the registration procedure. MI was then optimized with gradient descent on US and MR images of the beating heart. There was no quantitative analysis of the errors but the computation time was roughly decreased by a factor of 40. Zhang et al. [71] [72] developed a variation of MI-based registration by replacing image intensity with image phase for both US and MR cardiovascular data. This was originally imple- mented as a global affine registration method. The authors then improved the registration accuracy by creating an adaptive multiscale framework that allowed for local affine transformations. The adaptive region approach is also adopted by Zhang [73] for a MI-based US to CT registration of the abdominal region using image intensities. Other intensity-based approaches include Milko et al. [45]’s work, where they presented a registration method to align liver. Their algorithm is sim- 24 2.3. Multimodality Registration ilar to Penney et al. [51], which was discussed previously for aligning femur and pelvis images. Using dynamic texture analysis of the US, probability maps are created that classify pixels as liver parenchyma or vessel. These probability maps are then reconstructed into volumes which are then regis- tered to segmented vessels in the MR. Because segmentation is required, this method is not fully automatic. A mean error of 1.98◦ and 4.10 mm from expert ground truth was achieved. A non-linear registration method was proposed by Arbel et al. [2] for MR to US image registration of the brain. A pseudo US image is generated by segmenting major brain structures from the MR image. The segmented image is then modified to include only struc- tures visible in US and preprocessed to reflect the appearance of US. It is important to recognize that Arbel et al. present an example of deformable registration of soft tissue as well as that they are one of the first to modify the CT to resemble the US, an important strategy that is discussed further in the next section. 2.3.3 US Simulation-Based Registration of US and CT The most challenging aspect of US to CT registration is that the images are dissimilar. Many of the feature-based and intensity-based approaches have the common strategy of using preprocessing techniques to emphasize the similar elements of images in both modalities. Wein et al. take this one step further by presenting an intensity-based registration that iteratively generates an US simulation from CT throughout the registration process. This method was first applied to rigid registration of the head and neck [67]. In this early work, instead of creating a realistic simulation of US, an effi- 25 2.3. Multimodality Registration cient intermediate representation of the CT was constructed that combined intensity, gradient and edge information. The US was searched from top to bottom and the occluded regions are discarded from the registration. Fi- nally, an NMI metric that incorporated skin clamping and edge alignment was used to register the image volumes. Three optimization strategies were tested and exhaustive hill climbing was determined to perform the best. While all registration trials converged to ground truth, it should be noted that the authors tested a small capture range of 5 mm and 5◦. Thus, a good initial alignment would be necessary to use this method clinically. This study was succeeded by proposing a method that iteratively simulates US from CT. Wein et al. [66] [65] realized that the acoustic properties of tissue could be inferred from CT and thus, could be exploited to simulate the US signal propagating through tissue surfaces. The simulated US was generated iteratively from the CT throughout the registration. The CT intensities were mapped to values that were more similar to the corresponding US values. The simulated US and modified CT were then compared using a correlation ratio framework and optimized by the simplex algorithm. This method was tested on a combination of liver and kidney data sets and was determined to be an improvement on CC or MI alone. Shams et al. [59] extended this work by focusing on creating a more realistic ultrasound simulation for training purposes. The quality of the simulated US was improved by incorporating the effects of scattering and beam width during the simulation. However, the scatter volume of the CT had to be created by using the Field II US sim- ulator [36] [35] which is time consuming. The sample simulation of a cyst phantom presented by the authors took 11 hours to generate. Therefore, 26 2.4. Validation of Registration Methods on Phantom Data while the enhanced simulated US images has the potential to improve the alignment of images, it may not be practical to employ them for registration procedures being prepared for intraoperative use. Because speed is often a deterrent from employing intensity-based algorithms, Reichl et al. [56] and Kutter et al. [38] have implemented the simulation-based registration on the Graphics Processing Unit (GPU). The GPU-accelerated version re- sulted in significant speed-ups in computation time because multiple rays in US are processed in parallel. Several other groups have adopted Wein et al ’s approach for a variety of applications. Gill et al. [25] [24] have fo- cused on bone-registration, specifically for the lumbar spine. They perform a group-wise registration and also used a biomechanical model to constrain the motion of the vertebra. In addition, we have et al. [70] applied the algorithm to registration of the kidney. Both approaches used an evolution- ary optimization strategy which has been proven to perform better than gradient descent or simplex methods [25] [69]. 2.4 Validation of Registration Methods on Phantom Data A popular method for validation of registration methods is to construct phantoms that model the anatomical structure of interest. For registration of bone, researchers can simply embed a rigid model of the structure such as spine in a gel medium and acquire images. This was performed in the study by Gill et al. [25]. However, creating a suitable phantom of soft tissue for a surgical application such as partial nephrectomy is difficult. These 27 2.4. Validation of Registration Methods on Phantom Data phantoms must appear realistic and be representative of the anatomy of in- terest in both US and CT imaging modalities. In this case, the phantoms must strive to accurately depict the kidney including the internal structures such as the renal pyramids and renal vasculature. At the present time, little work has been published on phantoms designed for US to CT reg- istration. Recently, Cheung et al. [16], also investigating registration for partial nephrectomy, used a phantom-less approach for validating the fusion of stereoscopic video and laparoscopic ultrasound. However, without the presence of kidney tissue in the image, it is not possible to discern features that are important to registration such as characteristic structures, average intensities of common regions, or realistic boundaries. Benincasa et al. [7] also used kidney phantoms for US to CT registration. However, these were made with silicon rubber and were designed only to model the surfaces of the kidney, not the internal anatomy. Recently, Keil et al. [37] constructed phantoms that allowed US and CT imaging to illustrate the internal struc- tures. This group treated the ex vivo porcine kidneys with contrast agent and embedded them in a gelatin medium before they were imaged. This is similar to the protocol that we present for creating kidney phantoms. Our phantoms were designed not only to depict the surface boundaries, but also to clearly define the vascular and pyramid anatomy of the kidney in both modalities. 28 2.5. Summary 2.5 Summary US to CT registration has a wide variety of applications in image-guided surgery. These range from interventional procedures on bone such as surgery of the knee to percutaneous injection into the spine. In addition, soft- tissue applications include surgical procedures of the kidney, liver, brain, and heart. This has resulted in the ongoing developing of both feature- based and intensity-based approaches to registration. Because we intend to register US and CT images of the kidney intraoperatively, it is important that the method selected be fully automatic and require no manual interven- tion. Thus, feature-based methods become less desirable since many of them require the segmentation of surfaces or the selection of landmark points. In addition, there is currently no technology in the operating room to provide an initial estimate of the registration. Hence, the method should not require the initial scanning direction of the US, so the work of Brendel et al. [9] and Winter et al. [68] is not suitable for this application. An interesting strategy has been taken by Roche et al. [57] and Leroy et al. [40], who modified the US and CT images to resemble each other more closely before registration. However, the preprocessing occurs once and the CT is not updated to con- sider the direction of the US beam throughout the registration. Thus, we have chosen to use the simulated US-based registration method presented by Wein et al. [65]. It is fully automatic and the US simulation is dynami- cally updated such the US reflections in the simulation change based on the transform obtained by the optimizer. Because the parameters of the trans- formation are being optimized, and the simulation is updated iteratively, 29 2.5. Summary the direction of the US beam is updated and there is no need to determine it ahead of time. This serves as a suitable approach for registration of US and CT images of kidney for our phantom and clinical data. 30 Chapter 3 Experimental Design The US to CT registration technique was first validated on porcine kidney phantom data. There were several objectives in designing the protocol for preparing the kidney phantoms. First, the phantoms had to produce high quality US images with minimal shadowing throughout the tissue. The phantoms also had to produce realistic CT angiography images that were representative of in vivo human angiograms. In addition, they were designed not only to depict the surface boundaries of the kidney, but also to clearly define the vascular and pyramid anatomy of the kidney in both modalities. The materials used were selected so that fiducials could be easily placed to provide a gold standard alignment. The advantage of testing on phantoms is that it eliminates certain challenges such patient breathing and motion, allowing us to focus on developing the registration technique. US and CT angiography images were acquired from the phantoms to provide a suitable platform for validation. The registration algorithm was then tested on a partial nephrectomy patient to investigate the performance and additional challenges of images from patients. The data set was acquired preoperatively from a renal cancer patient. 31 3.1. Phantom Data Acquisition 3.1 Phantom Data Acquisition When we began investigating the construction of soft tissue phantoms, we used kidneys obtained from butchers. These butchered kidneys had the renal hilum removed and were exposed to air. Thus, getting high quality realistic images was difficult. Air inside the kidney created a problem for the US since it cast a large shadow in the US volumes (Fig. 3.1A). Because the renal hilum was removed, there was no method of introducing water or gelatin into the vessels to remove the air. In addition, there was no means of injecting contrast. Therefore, it was difficult to distinguish between tissue and background in the CT and impossible to identify any vessel or pyramid structure (Fig. 3.2A). Thus, we eventually acquired kidneys from live pigs that had their renal hilum intact. As observed in the images taken from these kidneys, the shadow in the US is no longer a problem and the anatomical structure of the kidney is distinct in both modalities. Seven phantoms were prepared with the kidneys suspended in agar and imaged with US and CT angiography. 3.1.1 Procedure for Kidney Phantom Construction A plastic box was used to hold the kidneys which were enclosed in agar. The agar solution consisted of the following ingredients in percentage by mass: 1.17% high gel strength agar, 4% glycerol, 0.25% bleach, and 94.58% water. The amount of glycerol added was selected to regulate the speed of sound in agar such that it matches the typical speed of sound in soft tissue, which is 1540 m/s [42]. To prepare the agar, the reagents were mechanically 32 3.1. Phantom Data Acquisition Figure 3.1: A) US of a butchered kidney that lacks its renal hilum and thus has no preparation. B) US of a porcine kidney from a live pig. 33 3.1. Phantom Data Acquisition Figure 3.2: A) CT of a butchered kidney that lacks its renal hilum and thus has no preparation. B) CT angiography of a porcine kidney from a live pig with contrast agent injected according to our protocol. 34 3.1. Phantom Data Acquisition stirred with a stir bar in a large beaker on a magnetic stir hot plate, and the temperature was brought to 90◦C. Stirring continued to ensure a uniform solution while the agar was allowed to cool at room temperature to 50◦C, prior to being placed in the refrigerator to solidify. A base layer was poured in the box first and allowed to harden. Porcine kidneys were excised from live pigs from the Animal Laboratory at the Jack Bell Research Center (Vancouver General Hospital, Vancouver, BC, ethics certificate number A050316). The pigs were injected with heparin prior to excision to prevent clotting within the kidneys. The renal hilum and ureter were left attached and the renal artery and vein were dissected. Promptly after the kidneys were taken out, saline was flushed via a syringe through the renal artery. Flushing was repeated until the saline solution appeared clear when exiting the renal vein, indicating that most of the blood had been removed. The renal capsule was then removed so that it did not trap air bubbles that disrupt the US and CT images (Fig. 3.3A). Removal of the thin membrane had no noticeable effect on image features in either the CT or US. Two solutions containing the contrast agent Omnipaque iohexol (GE Healthcare, Mississauga, Ontario) injection (300 mg I/ml) were then prepared. First, 10 ml of a 1 in 40 dilution in water was injected through the renal artery (Fig. 3.3B). Because the contrast was diluted with water, it diffuses into the parenchyma of the kidney and highlights the tissue in the CT image. Next, a 3.6% by mass gelatin solution was prepared by adding gelatin to water and heating until it dissolved. The solution was then cooled to 50◦C. For the second contrast solution, Omnipaque was diluted 1 to 5 by the gelatin solution. It is important to recognize that the second solution 35 3.1. Phantom Data Acquisition was more concentrated than the first and 10 ml of it was injected into the kidney. Gelatin was used instead of water so that as the phantoms cool, the gelatin solidifies in the arteries, allowing us to highlight the arteries by not permitting the contrast to diffuse into the surrounding parenchyma. Finally, the artery and vein (Fig. 3.3C) were tied off (Fig. 3.3D) so that contrast agent does not diffuse into the surrounding agar. Figure 3.3: A) Porcine kidney during the removal of the renal capsule. B) Injection of diluted contrast agent into the renal artery. C) The dissected renal artery on the right and renal vein on the left. D) The renal artery and renal vein are tied off to prevent leakage of contrast agent. Once the kidneys were properly prepared with contrast agent, two of them were placed on the agar base layer (Fig. 3.4A). The agar cooled to 36 3.1. Phantom Data Acquisition Figure 3.4: A) Placement of porcine kidneys on the surface of base layer of agar. B) Phantom after top layer of agar has been poured to cover kidneys. 50◦C, and was poured overtop until it covered the top of the tissue by about a centimeter (Fig. 3.4B). The box was then refrigerated until the agar had completely solidified. Next, four 1 mm stainless steel balls were pressed into the agar above each kidney (Fig. 3.5). These steel balls serve as fiducials, which provided the gold standard for registration. More agar was poured over the steel balls to fill the rest of the box and the newly poured solution was cooled. There should be at least 25 mm of agar between the steel balls and the surface. This aids the 3D US acquisition as it ensures that the entire kidney is captured in the field of view of the sector-shaped volume. 37 3.1. Phantom Data Acquisition Figure 3.5: Photograph of the surface of the agar in a prepared porcine kidney phantom. The locations of 4 steel ball fiducials are shown by the arrows. These fiducials are pressed into the agar. The dotted line represents the axis of the native scan. CT and US slices were taken parallel to that line. 3.1.2 CT Acquisition The CT scans were acquired using the Aquilion 64-slice CT scanner (Toshiba Medical Systems, Tustin, CA, USA) at 120kVp. The pixel spacing was 0.468 mm and the slice thickness was 1 mm. The CT images of the phantoms were acquired first so that the agar was undisturbed. After the scanning, the ultrasound images were carefully taken with a 3D probe. 38 3.1. Phantom Data Acquisition 3.1.3 Ultrasound Acquisition Pre-scan converted B-mode ultrasound volumes were obtained using the Sonix RP machine (Ultrasonix Medical Corporation, Richmond, BC, Canada) with the 3-7 MHz convex curvilinear abdominal probe (4DC7-3). The 3D probe we used is a mechanical probe where the curvilinear transducer sweeps back and forth inside the case. The 3D probe was securely fastened with a clamp above the kidney phantom (Fig. 3.6). The 3D volume was then acquired at a transmit frequency of 5.0 MHz. The sampling frequency was 2.5 MHz. The depth setting varied between 8 cm - 10 cm, depending on the depth of the kidney. Figure 3.6: For ultrasound acquisition, the 3D curvilinear probe is clamped in place above the phantom. 39 3.1. Phantom Data Acquisition 3.1.4 Defining the Gold Standard Alignment Manual registration was performed using the locations of the four fiducial markers. To do this, the coordinates of the steel balls were found manually with a graphical user interface. Because the location of the bright intensity response in the US occurs at the top surface, the coordinates of the steel balls obtained represent the point on the top surface in both the CT and US. It has been shown that this approach will achieve sub-millimeter fiducial localization error [28]. These points were used to calculate the six rigid registration parameters by using Horn’s method [32]. The rotation and translation in the x, y, and z direction were used to align the US and CT volumes, and was considered the gold standard. 3.1.5 Method of Error Analysis The volumes were aligned at the gold standard before registration was per- formed. The CT volume was then perturbed by a transform selected ran- domly from a uniform distribution of 10◦ rotation about each axis and 10 mm translation along each axis. The targets used to determine the target registration error [22] (TRE) was a set of surface points from the kidney surface of each phantom. Each phantom was segmented using Stradwin [23] to produce the surface points used. 3.1.6 Scan Conversion Correction Recall that the motor of the 3D US probe sweeps back and forth, taking images that then fill the 3D volume. Perfect motor control would take 40 3.1. Phantom Data Acquisition each frame at an equal distance from the previous frame. However, if the motor control is not ideal, the scan converted volume that is produced will be slightly different from the actual volume and the CT volume. The 3D probe that was used to capture data swept slightly slower on one side than the angle step specified by Ultrasonix. This resulted in scan converted US volumes that were slightly distorted when aligned to the CT. To correct these images, an N-wire phantom was constructed to find the true angle step during the probe sweeps. The N-wire phantom was constructed similar to the one used by Chen et al. [15] using plexiglass and is shown in Figure 3.7. The nylon wire used to create the N-wires was 0.5 mm in diameter and was sanded prior to use so that strong echoes are created in US. A CT of the phantom was then taken to confirm the positions of each of the wires. Next, the phantom was filled with water at 37◦C and the 3D probe was clamped above the phantom. Nine US volumes of the N-wires were taken from exactly the same position with the same field of view of 71◦. After acquiring the data, each frame was scan converted using Pando, a software development kit provided by Ultrasonix. This resulted in a stack of scan converted slices for each volume. Chen et al ’s automated segmentation algorithm was then used to segment the circle-shaped dots that represented the location of the wires (Fig. 3.8). The distances between the top and bottom wires are calculated and averaged to create one distance per slice, represented by h. The objective of these experiments was to find the true angle that each slice was taken at. Figure 3.9 illustrates the geometry of the problem. The parallel lines AB and CD represent the top and bottom wires whereas the 41 3.1. Phantom Data Acquisition Figure 3.7: The N-wire phantom. Wires are strung yielding two parallel N’s. blue line is the US slice. The distance between the wires as mentioned earlier h is determined from the US slices. A CT image was taken of the phantom and d, the distance between the wires in the phantom, was determined. This allowed us to use equation 3.1 to solve for the angle. This was done over nine US volumes and an average angle for each slice was determined. 42 3.1. Phantom Data Acquisition Figure 3.8: US slice depicting the segmented wires. The red arrows indicate the distance between parallel wires in the US image. cosθ = d h (3.1) The curvilinear array in the 3D probe sweeps from one end, to the center and then to the other side. The angles were plotted and it was determined that on one side, the probe sweeps slower than the 0.731◦ that was specified. Instead, the slope of the line indicated that the probe was only stepping 0.611◦ per frame (Fig. 3.10). This can be compared with the angles plotted on the other side of the sweep (Fig. 3.11) where the probe was stepping 0.747◦ per frame, which is much closer to the expected 0.731◦. The value for 43 3.1. Phantom Data Acquisition Figure 3.9: Shows the US slice in blue, and the parallel wires AB and CD. The angle θ is the angle that the US slice is taken at. the slower sweep of 0.611◦ per frame was integrated into the scan conversion of the US volumes and also, the inverse scan conversion of CT volumes 44 3.2. Clinical Data Acquisition presented later. It served to reduce some of the distortion of the US and improve the gold standard alignment to the CT. 0 10 20 30 40 50 5 10 15 20 25 30 35 40 Slice Index An gl e (de gre es ) Average Angles    y = − 0.611*x + 36.1 Figure 3.10: Graph plots the average angle calculated for each slice on half of the probe sweep, where the probe sweeps slower than expected. The slope of the line indicates that the angle changes by 0.611◦ per frame. 3.2 Clinical Data Acquisition In addition to phantom data, collection of clinical data from renal cancer patients has recently begun. Preoperative CT and US data were taken from patients prior to them receiving the robot-assisted partial nephrectomy procedure. The first complete dataset is presented in this thesis. 45 3.2. Clinical Data Acquisition 50 55 60 65 70 75 80 85 90 95 0 5 10 15 20 25 30 35 40 Slice Index An gl e (de gre es ) Average Angles    y = 0.747*x − 35.7 Figure 3.11: Graph plots the average angle calculated for each slice on half of the probe sweep. The slope of the line indicates that the angle changes by 0.747◦ per frame which is closer to the expected angle step 0.731◦. 3.2.1 Positioning the Patient for Imaging All imaging took place at Vancouver General Hospital (Vancouver, BC) in the CT room (Vancouver, BC, ethics certificate number H08-02798). Extra care was taken to ensure that minimal movement of the patient occurred between the US and CT scans. The patient was positioned on the CT table in flank position (Fig. 3.12). A cushion was placed underneath the lateral side of the patient’s abdomen in order to expose the affected kidney for scanning. This arrangement is similar to the positioning of the patient on the OR table during the surgery. The patient was kept in flank position 46 3.2. Clinical Data Acquisition during both the US and CT imaging. Figure 3.12: This illustrates how the patient is positioned for image capture. The patient is kept in flank position during both the US and CT imaging. A cushion is placed underneath the patient’s abdomen to expose the kidney for scanning. A stand-off pad with embedded fiducial markers is placed on the area of scanning between the US probe and the skin. 3.2.2 Preoperative US and CT Acquisition The preoperative US was acquired first using the Sonix RP machine with the 3-7 MHz convex curvilinear abdominal probe (4DC7-3). Similar to the phantom data, pre-scan converted data was saved. A stand-off pad contain- ing steel ball fiducials was then fastened to the patient with surgical tape as shown in Figure 3.12. These fiducial markers were intended for use as the gold standard for registration. Several US scans were then taken on breath hold. After the US scanning was complete, the patient was asked not move 47 3.2. Clinical Data Acquisition as the CT was taken immediately after. The preoperative CT was taken of the patient with the Somatom Sensa- tion 64-slice CT scanner (Siemens Medical Solutions, Malvern, PA, USA) at Vancouver General Hospital. The patient was injected with contrast agent so that CT angiography with several phases could be recorded. The stand- off pad that was fastened to the patient was also scanned during the CT examination. 3.2.3 Finding the Bronze Standard Alignment of the US and CT images It was originally thought that the steel ball fiducial markers could be used to achieve a gold standard alignment of the US and CT images. However, using the markers with Horn’s method to find the transformation resulted in residual alignment errors of up to 15 mm. Figure 3.13 shows an overlay of the CT and US of a patient where the kidneys are aligned. It can be seen that the fiducials circled in yellow in the images are not aligned. It is possible that the pressure of the probe on the stand-off pad and the skin during scanning contributed to the misalignment. In addition, a patient’s breath hold is not always repeatable. Thus, in the absence of a gold standard in clinical data sets, anatom- ical landmarks can be used to form a bronze standard. A more detailed description is provided by Glatard et al. [26]. In this case, the surface of the kidney was used. A method that was developed in the Robotics and Con- trol Laboratory (UBC, Vancouver, BC) was employed to register the two volumes using Principal Components Analysis (PCA). Both the CT and US 48 3.2. Clinical Data Acquisition Figure 3.13: Shows that the fiducial marker in CT circled in yellow on top and the fiducial marker in US circled in yellow on bottom. datasets were segmented to generate two sets of surface points. Next, for each volume, the principal axes were calculated using the eigenvectors and eigenvalues. The axes of the two volumes were then aligned. The trans- formation obtained was then used to resample the original images. Once resampled, a region of interest was cropped in the US to eliminate the areas outside the imaged sector. Several slices of the aligned patient images are shown in Figure 3.14. 3.2.4 Method of Error Analysis For registration, the CT was perturbed by a transform selected randomly from a uniform distribution of 20◦ rotation about each axis and 20 mm 49 3.2. Clinical Data Acquisition Figure 3.14: Various slices of a volumetric datatset from a single patient. Volumes are aligned using PCA as the bronze standard. Pairs of correspond- ing US and CT slices are shown in each row. translation along each axis. The TRE was calculated from the surface points of the kidney using the same protocol as the phantom data sets. The initial misalignment resulting from each set of transform was determined. Twenty tests were performed for each range of misalignment errors including: 0 - 50 3.2. Clinical Data Acquisition 5 mm, 5 - 10 mm, 10 - 15 mm, and 15 - 20 mm to determine the capture range for the registration. 3.2.5 Summary Validation of the registration algorithm is done on porcine kidney phantoms. A detailed recipe of how to construct the kidney phantoms to achieve real- istic images in both US and CT angiography is presented. It was important that major structures in the kidney such as the vascular system, the renal pyramids and the renal cortex be visible in both the US and CT. A gold standard alignment was determined from steel ball fiducials that were placed in the agar-based phantoms. The limitations of this design is that the kid- neys remain as fairly rigid structures throughout the imaging process. This does not account for deformation as a result of breathing, probe pressure on the skin and patient motion. To address some of the additional challenges of registration on patient data, US and CT angiography images of a partial nephrectomy patient was taken. Careful positioning of the patient and acquisition of the data served to reduce the motion of the patient between the US and CT scans. A bronze standard was formulated to validate the registration of this data set. 51 Chapter 4 Methods The following section describes the method of registration proposed. This registration technique is an extension of Wein’s work [65]. Wein et al. evalu- ated the registration method on both liver and kidney data using a simplex optimization scheme. However, this work focuses on registration of kidney and uses the Convariance Matrix Adaptation Evolution Strategy (CMA-ES) optimizer [29] which has been found to be more robust than gradient descent or simplex optimization strategies [25]. A rigid registration is performed that optimizes six parameters representing the rotation and translation of the images. The main steps of the registration include: 1. The generation of an US image from the CT by simulating the US sig- nal being transmitted through the tissue layers using the basic physics of sound propagation and attenuation. 2. Mapping of CT intensities to corresponding intensity values in US. 3. Using the Linear Correlation of Linear Combination (LC2) similarity metric to assign a similarity value. The parameters are optimized such that there is the greatest similarity between the US and CT volumes. All registrations tests were performed on 52 Chapter 4. Methods an Intel Core 2 Duo 2.66 GHz CPU with 3.24 GB of RAM. Unlike many of the registration techniques discussed previously, this ap- proach does not require prior knowledge of the direction of US scanning with respect to the patient anatomy. The simulation of US from CT is updated iteratively throughout the registration process. Because the direction of the US beam affects which tissue interfaces are highlighted in the US image, it is crucial that the simulated US is updated with the current transform provided by the optimizer. The original implementation by Wein et al. used tracked freehand US. Since this study used a 3D curvilinear probe, the registration technique was implemented in two ways affecting the simulation of the US at each step. The first method allows for a simple implementation where the scan- converted US volumes are compared to CT volumes. The direction of US simulation is vertical, from top to bottom in each column of the image. The second method however, considers the geometry of the probe and US propagation in generating the simulation. Pre-scan converted US data is compared to CT volumes that have been interpolated along the direction of the US signal. Thus, a conversion of the CT data from physical coordinates to the pre-scan converted coordinate system occurs iteratively throughout the registration process, just prior to the US simulation. Consequently, the US is simulated from the CT in the direction of the actual US beam. A diagrammatic overview of the two methods is depicted in Figure 4.1. 53 4.1. Registration via the Simulated Ultrasound Figure 4.1: A) Shows a simple implementation where the simulation occurs along the vertical direction of the image. B) Depicts the approach where the simulation occurs along the direction of the US beam. 4.1 Registration via the Simulated Ultrasound The registration process is visually depicted in the flowchart shown below (Fig. 4.2). A rigid registration is performed using a six-parameter Euler transform. The optimal transform is found by the CMA-ES optimization strategy [29]. CMA-ES is a stochastic method, which means that a certain level of randomness is introduced during optimization. This helps prevent the optimizer from settling on a local minimum. Once the transform is 54 4.1. Registration via the Simulated Ultrasound obtained, the US simulation process begins. The CT volume is resampled according to the transform. Next, the US reflections are simulated from the CT. Simultaneously, the CT values are mapped to those found in US. These intermediate images are used to find two weights and a bias term, which are iteratively updated to generate the simulation that achieves the best estimation of US image. Figure 4.2: Workflow showing the generation of the simulated US from CT and registration of the US to simulated US. 55 4.1. Registration via the Simulated Ultrasound 4.1.1 Creating the Simulated Ultrasound The use of a simulated ultrasound for registration came out of Wein et. al ’s research [65]. The main goal is to generate an intermediate modality from the CT that resembled the US more closely and resulted in a better registration. To achieve this, an understanding of the basic physics be- hind the two modalities is required. The acoustic impedance of the tissue controls the amount of reflection and transmission of an ultrasound wave. Acoustic impedance Z = ρc is proportional to tissue density and the speed of sound. The ratio of ultrasound intensity reflected at a tissue boundary where the angle of incidence is equivalent to the angle of reflection is given by (Z 2 − Z 1) 2 /(Z 2 + Z 1) 2 at a specular interface with varying acoustic impedances Z 1 and Z 2. The diffuse reflection 4r that is reflected back to the ultrasound transducer is dependent on the angle of incidence θ: 4 r(Z1, Z2, θ) = (cos θ)( Z2 − Z1 Z2 + Z1 )2 (4.1) If we ignore refraction, the transmitted intensity is not dependent on the angle of incidence and thus is given by: t(Z1, Z2) = 1− ( Z2 − Z1 Z2 + Z1 )2 (4.2) The X-ray attenuation µ, represented in Hounsfield units is approxi- mately proportional to the tissue density. Because of the relationship be- tween acoustic impedance and tissue density, we can use the change in CT intensities as a way to represent the change in the acoustic impedance. While 56 4.1. Registration via the Simulated Ultrasound this approach remains a simplified model of the actual complex interaction of ultrasound propagation in tissue, it is sufficient for the purpose of registra- tion. Thus, we can replace the change in acoustic impedance Z 2 − Z 1 with the change in X-ray attenuation 5µ(x,y,z). The reflected and transmitted energy can then be modeled by the following equations: 4 r(x, y, z, d) = (dT 5 µ(x, y, z)) | 5 µ(x, y, z)| (2µ(x, y, z))2 ; (4.3) t(x, y, z) = 1− ( | 5 µ(x, y, z)| (2µ(x, y, z)) )2; (4.4) r(x, y, z) = I(x, y − 1, z)4r(x, y, z, d); (4.5) I(x, y, z) = I(x, y − 1, z)t(x, y, z). (4.6) where d is the direction of ultrasound propagation, µ is the measured X-ray attenuation in the CT, ∆r is the reflection coefficient, r is the simulated reflection intensity, t is the transmission coefficient, and I is the final inten- sity of the simulated US. The angular dependency represented by cos θ is replaced by the scalar product of d with the normed CT gradient vector. In Gill et al.’s implementation for bone [25], the gradient values at each voxel were monitored and values above specific thresholds caused total reflection of the US beam. Because their work was primarily on bone, they had to represent areas of occlusion. Because this is not necessary for soft tissue, no tuning of threshold values was done for the phantom data. However, for 57 4.1. Registration via the Simulated Ultrasound the in vivo patient data, it was added to account for the possible presence of the spine in the images. A log compression is applied in order to amplify smaller reflections. This is important when working with soft tissue because there are very small changes in acoustic impedance between the layers of tissue. In this case, the value of a was set to 15. r(x, y, z) = 1− log(1 + aI(x, y, z)) log(1 + a) (4.7) Different tissue types have specific echogenicity. Because there is no simple relationship between tissue echogenicity and the X-ray attenuation, an intensity mapping was added inWein’s original implementation [65] based on correspondences of CT intensities to tissue echogenicity in ultrasound of several tissue types including liver, kidney and gall bladder. Thus, the final simulated US is the sum of the reflections and mapped CT intensities. A linear function that approximates the curve used by Wein et al. [65] is used [25]. p(x, y, z) = 1.36µ(x, y, z) − 1429 (4.8) After the reflection and mapped CT components are calculated, a set of weights is found to define their relative contributions to the final simulation. As described in the equation below, α weights the reflection, β weights the mapped CT and γ is the bias term. The weights are optimized using a least-squares strategy with the objective of creating a simulated US with realistic intensity values. While occluded pixels are not common because our image data is of kidney, there are occasionally occluded voxels due to 58 4.1. Registration via the Simulated Ultrasound small air pockets in the CT. Thus, it is important to mention that occluded voxels are not included as part of the weight calculation. Occluded voxels are handled separately and are set to be the mean of intensity values in the US that overlap the occluded region. The handling of occluded voxels becomes more significant when dealing with clinical cases with abdominal CTs that have both hard and soft tissue. The final values for the simulated US are calculated as follows: f(x, y, z) =   αp(x, y, z) + βr(x, y, z) + γ, if I(x, y, z) > 0 ψ, if I(x, y, z) = 0 (4.9) where f represents the simulated US, ψ is the mean US intensity from the voxels that correspond to the occluded voxels in the CT, and α, β, and γ are the weights. The weights are updated iteratively during the optimiza- tion. The various components of the simulation are depicted visually in Figure 4.3. A and B show the original US and CT images of one of the porcine phantoms. Figure C depicts the US reflections that were calculated from the CT and Figure D illustrates the transmission and gradual loss of energy from the surface to the tissues beneath. Finally, Figure E displays the simulated US which combines these reflections and the mapped CT. Note that for the registration tests on the patient data, the weights are fixed throughout registration to a set of values determined from the bronze standard alignment. During registration, the optimizer is improving the alignment based on the Linear Correlation of Linear Combination (LC2) similarity metric pro- 59 4.2. Registration using Scan Converted US Data posed by Wein et al. [65] and presented in the equation below. LC2 = Σ(U(x, y, z)− f(x, y, z))2 NV ar(U) (4.10) Recall that the real US intensities denoted U are being compared with the simulated US values f. This comparison of more similar images than US to CT is the main reasoning behind using this registration method. The difference between the two images is divided by N which is the number of overlapping voxels. All voxels are used for this calculation including those few that are occluded. 4.2 Registration using Scan Converted US Data A visual summary of this variation on the registration method can be found in Figure 4.1A) above. Pre-scan converted data is collected from the ultra- sound machine. This pre-scan converted data is simply a 3D array of values, each representing the intensity at a position given in the pre-scan converted data coordinate system (Fig. 4.4). The vertical position of the sample is represented by s, the angle along the curvilinear array is denoted θ and the angle in the elevational direction is φ. Because the CT is in physical coordinates, x, y and z, it is necessary to convert the US data into the same coordinate system for registration. This is done by using the known geometry of the 3D probe and is described in Pospisil et al. [55]. Figures 4.5 and 4.6 label the angles and distances used 60 4.2. Registration using Scan Converted US Data in the scan conversion equations shown below: Hφ = h cos θ; (4.11) x = Hφ tan θ; (4.12) y = (Hφ −D) cosφ+D; (4.13) z = (Hφ −D) sinφ. (4.14) where h is the distance from the origin to the point, θ is the angle between the center line of the sector and the point in the xy plane, and φ is the angle between the center line of the sector and the point in the yz plane. Both θ and φ can be negative if the point is on the left side of the center line. The term D is the difference between the probe radius and the motor radius. The equations listed are used to find the physical coordinates of each US intensity in transducer the pre-scan converted coordinate system. Once this is done, an ideal spacing is determined. Next, for each point in physical coordinates, trilinear interpolation is performed on the US volume to determine the corresponding US intensity value. When scan conversion is completed, the new volume fills a 3D shape that will appear like Figure 4.7. When registration is done with scan converted data, the US volume in physical coordinates is aligned with the CT at the gold standard. After the CT is perturbed, the volumes are registered using image-based registration, the direction of the US beam is simulated perpen- dicular to the y-axis. This approach is easy to implement but the simulated US beam is not being generated along the paths of the actual US. 61 4.3. Registration using Pre-scan Converted US Data 4.3 Registration using Pre-scan Converted US Data An alternative to registering the images in physical coordinates is to register both volumes in the coordinate system of the pre-scan US. This approach allows the simulated US to be generated from the CT exactly along the rays of the actual US. An additional advantage is that interpolation occurs within the CT instead of the US which is more noisy. A visual summary of this variation on the registration method can be found in Figure 4.1B) above. The first step involves finding the corresponding physical coordinates that match the pre-scan US coordinates using the equations listed below (Eqs. 4.15, 4.16, 4.17, and 4.18). These equations are identical to the equations presented above except that an offset in each coordinate direction is determined manually and added such that the CT is aligned to the US at the gold standard. The physical coordinates are then transformed by the six transformation parameters provided by the optimizer. Next, the new physical coordinates are used to find the intensity in the CT image at that point by linear interpolation. For the purposes of this work, the new CT shall be referred to as the inverse scan converted CT, since it was converted from physical coordinates to pre-scan converted US coordinates. As a result, the pre-scan US image does not change. At this point, the simulated US is generated from the inverse scan converted CT and compared with the pre-scan US. This process is repeated at each optimization step. Figure 4.8 shows the pre-scan converted US accompanied by the inverse scan converted 62 4.3. Registration using Pre-scan Converted US Data CT. These images have been aligned at the gold standard using the fiducial markers. Hφ = h cos θ; (4.15) x = Hφ tan θ + offsetx; (4.16) y = (Hφ −D) cosφ+D + offsety; (4.17) z = (Hφ −D) sinφ+ offsetz. (4.18) Fig. 4.9 is a diagrammatic explanation of how the inverse scan converted CT is generated. The cube represents the CT volume in physical coordinates. The US probe is shown at a particular pose and the 3D volume that would be created is illustrated. The points along each scanline of every frame of the US are found within the CT, and the values at those points are interpolated. 63 4.3. Registration using Pre-scan Converted US Data Figure 4.3: The images above depict the intermediate steps in generating the simulated US from CT. Images were taken from porcine phantom data. A) and B) show the original US and CT. The reflection is illustrated in C) and the transmission in D). The final simulated US image is shown in E).64 4.3. Registration using Pre-scan Converted US Data Figure 4.4: Format of the pre-scan converted US data, where s is the vertical position of the sample, θ is the angle along the curvilinear array, and φ is the angle in the elevational direction. 65 4.3. Registration using Pre-scan Converted US Data Figure 4.5: Depicts how a point in the pre-scan converted coordinate system is converted to physical coordinates in the xy plane. The distance from the origin to the point is denoted h, rprobe is the probe radius, θ is the angle between the center line of the sector and the point and s is vertical position of the sample. 66 4.3. Registration using Pre-scan Converted US Data Figure 4.6: Depicts how a point in pre-scan converted coordinate system is converted to physical coordinates in the yz plane. The distance from the origin to the point is denoted h, rmotor is the probe radius, φ is the angle between the center line of the sector and the point and s is vertical position of the sample. 67 4.3. Registration using Pre-scan Converted US Data Figure 4.7: The shape of a 3D scan converted US volume. Figure 4.8: Shows the pre-scan converted US and its corresponding inverse scan converted CT. Both images are in pre-scan converted coordinates. 68 4.3. Registration using Pre-scan Converted US Data Figure 4.9: Shows the 3D US scan at a particular pose overlayed with the CT. The positions from the US are found in the CT, and the values are interpolated to create the inverse scan converted CT. 69 Chapter 5 Registration Results and Discussion This section discusses the results generated by the registration algorithm. The US simulation-based approach is shown to produce accurate alignments when registering US to CT volumes of kidney phantom data. This is com- pared to a variation of the method which dynamically simulates the US from the CT along the rays of the actual US beam. Consequently, the pre-scan converted US data, which does not have to undergo interpolation, can be registered to the modified CT. From the tests performed on our phantom data, there does not appear to be a significant difference between the two approaches. Finally, the registration approach is tested on a patient data set to evaluate its performance on clinical data. 5.1 Porcine Kidney Phantom Data 5.1.1 Registration using Scan Converted US Phantom Data Registering the scan converted US volumes with the CT volumes of seven phantom data sets produced the following results listed in Table 5.1. Recall 70 5.1. Porcine Kidney Phantom Data that the CT volume was perturbed by a transform selected from a uniform distribution of 10◦ rotation about each axis and 10 mm translation along each axis. Fifty tests with different initial transforms were performed. Of the tests that were run, there was 100% convergence for 5 out of the 7 phantoms. The second data set had 96% of the tests converge and the fourth data set had 90%. Each data set had a slightly different mean error and some of them performed much better than others, for example phantom 2 outperformed phantom 4 in terms of accuracy. This can be partially attributed to how well the contrast agent was able to highlight the structures in the CT. These US volumes have been scan converted from the pre-scan converted data which means that the data has been interpolated and there is decimation close to the top of the image. The dimensions of the US data set and the spacing in millimeters are shown in Table 5.1. The average run time of each test for all phantoms is 746 seconds. Kidney Mean STD US Size US Spacing Avg. Phantom TRE (mm) (mm) Time/Test (mm) (s) 1 3.5 0.10 (168, 93, 181) (0.7, 0.4, 0.3) 593 2 2.2 0.07 ( 186, 96, 162) (0.7, 0.4, 0.3) 547 3 2.7 0.07 (181, 97, 197) (0.7, 0.4, 0.3) 679 4 3.9 0.06 (173, 107, 204) (0.7, 0.4, 0.3) 847 5 2.4 0.07 (184, 94, 175) (0.7, 0.4, 0.4) 689 6 1.8 0.07 (203, 106, 203) (0.7, 0.4, 0.4) 1025 7 1.9 0.10 (203, 105, 218) (0.7, 0.4, 0.4) 844 Table 5.1: Results of registration for scan converted US and CT. The table shows the mean TRE and standard deviation acquired over 50 tests. The size of the data sets, the spacing information, and the average time taken per test are also listed. 71 5.1. Porcine Kidney Phantom Data 5.1.2 Registration Results using Pre-scan Converted US Phantom Data By directly registering the pre-scan converted data, the actual geometry of the US beam can be considered. Recall that given a particular transforma- tion, the CT was resampled along the paths where the US beam would have traveled. The image constructed would then be used to simulate the US in the registration process. The results for the 50 tests run on correspond- ing US and CT images are shown in Table 5.2. For 6 out of 7 data sets, 100% of the registration tests converged. The fourth data set had 90% of the tests converge. In Table 5.2, the bold data sets performed significantly better than the corresponding scan-converted US images described above. This was determined by using a two-sample independent means t-test at a 95% confidence interval. The dimensions of each pre-scan converted data set are shown in the table. In addition, the size of each voxel in the s, θ, and φ coordinate system is listed in the appropriate units. Notice that the sizes of the US data sets are all less than the corresponding scan converted images discussed above. The decimation factor was selected to make a fair comparison. The average time per test taken was 487 seconds. 5.1.3 Discussion Our original hypothesis was that using pre-scan converted US data would obtain better results because no interpolation occurs in the US, which is the noisier modality compared to CT. This was supported by previous research presented by Brooks et al. [11] who investigated deformable 3D US to MR 72 5.1. Porcine Kidney Phantom Data Kidney Mean STD US Size US Spacing Avg. Phantom TRE (mm) (mm, degrees, Time/Test (mm) degrees) (s) 1 2.6 0.15 (128, 134, 85) (0.3, 0.61, 0.73) 418 2 2.6 0.11 (128, 144, 91) (0.3, 0.61, 0.73) 494 3 2.5 0.09 (128, 156, 87) (0.3, 0.61, 0.73) 527 4 4.2 0.09 (128, 155, 97) (0.3, 0.61, 0.73) 553 5 1.5 0.10 (128, 154, 81) (0.3, 0.61, 0.73) 483 6 1.8 0.09 (128, 153, 77) (0.3, 0.61, 0.73) 454 7 1.4 0.19 (128, 164, 83) (0.3, 0.61, 0.73) 486 Table 5.2: Results of registration for pre-scan converted US and inverse scan converted CT. The table shows the mean TRE and standard deviation acquired over 50 tests. The dimensions of the data sets, the size of the voxels, and the average time taken per test is listed. The bolded entries indicate that the result is significantly better than the results acquired registering the scan converted US volume to the CT for that particular phantom data set. registration. In a similar vein as scan conversion, the authors suggested that directly registering the set of 2D US slices to the MR volume obtains better speed and accuracy than reconstructing a 3D US volume prior to registration. However, our results show that the two methods for simulat- ing US from the CT produce similar results in the phantom data. While there is a difference in the speeds given in Tables 5.1 and 5.2 above, it is difficult to assess whether the faster computation time is a sole result of the registration algorithm. The difference in sizes of the data sets and the varying interpolation strategies may have also contributed to differences in computation time. There are several potential sources of error in registration of the porcine phantoms. First, the renal capsule of the kidneys had to be removed so that 73 5.1. Porcine Kidney Phantom Data it would not trap air, which creates artifact in both the US and the CT. Because the phantoms were prepared the night before, contrast was free to leak out of the kidney tissue into the surrounding agar causing slight blur- ring around the boundaries of the organ. This reduced the image quality slightly which could possibly affect registration. It would be preferable to inject contrast at the time of the CT. It has been suggested that tubing be stitched to the artery so that it can be accessed from outside the agar layer. In this way, injection just prior to CT imaging would result in more crisp boundaries between tissue and background. Second, it is conceivable that enhancing the quality of US simulation from the CT may also improve registration results. For instance, the simulated-based approach employed does not generate speckle in the images. While this phenomenon can be simulated using software packages such as Field II, [35] it is extremely time consuming and may not always be realistic depending on scatterer density. Even creating a more realistic ultrasound simulation such as the work done by Shams et al. [59] is not practical for registration because it uses the Field II simulator to create the scattering effect. Thus, it may be reasonable to suggest an improvement of the p-curve that controls the mapped CT com- ponent of the simulation. Currently, the p-curve is static and was derived from Wein’s original method [65]. However, not all phantom images appear the same especially in CT angiography. This is also true of patient CT angiography images. Perhaps, it would be beneficial if the p-curve was dy- namic and was updated throughout registration based on the CT volumes supplied. Wein also suggests that in place of a p-curve, the original values of US and CT be used and the weights be calculated locally, in a small region 74 5.2. Patient Data around the voxel. Finally, for any registration application with US, errors could arise from the localization of fiducials and the distortions caused by speed of sound variations. 5.2 Patient Data Although the phantoms created realistic depictions of the kidney, the sur- rounding tissues in real patient images may introduce challenges in the reg- istration. In particular, the enclosing fat and tissue changes the relative brightness of the kidney to its surroundings. For example, in US, the kid- ney tissue is darker than the surrounding tissue. However, when aligned at the bronze standard, the simulated US is generated correctly (Fig. 5.1). The kidney tissue appears darker than its exterior and there are highlights at the tissue interfaces. The realism of the simulation is controlled by in part the weights (Eq. 4.9), and the weights are influenced by the p-curve. By monitoring the registration, it was observed that when the volumes are misaligned, a realistic simulation was not always generated, and the kid- ney appeared brighter than the surrounding tissue. Thus, the weights were fixed throughout registration to those calculated from the bronze standard alignment to ensure proper simulation of the US image. Recall that eighty registration tests in total were performed. These tests were divided into categories based on the initial misalignment. Twenty tests for each range of 0 - 5 mm, 5 - 10 mm, 10 - 15 mm, and 15 - 20 mm was carried out. Figure 5.2 shows the percentage of tests for each category that succeeded for post-scan converted data and Figure 5.3 for pre-scan converted 75 5.2. Patient Data Figure 5.1: The left image shows the US image of the kidney and the right image shows the corresponding simulated US. data. Success is defined as the registration algorithm converging to a final TRE that is less than 10 mm. As stated in the literature [3], a tumour margin of 5 - 10 mm is considered sufficient. If we consider the capture range to be where 90% of the tests are successful, then the first method using post- scan converted data has a capture range of less than 5 mm. However, the second method has a capture range of 10 mm. In contrast to the phantom experiments, the results show that the registration method using pre-scan converted data outperforms the method using post-scan conversion. For those registration tests that converged to a final TRE of under 15 mm, the final TRE is plotted against the initial misalignment in Figure 5.4. The initial misalignment is displayed on the x-axis. It is found by perturbing a set of surface points on the kidney by the initial transform of the test and comparing it to the bronze standard alignment. The final TRE is plotted on the y-axis and is determined by transforming the set of surface points by the final transform parameters generated by the optimizer. The means of 76 5.2. Patient Data 0 − 5 5 − 10 10 − 15 15 − 20 0 10 20 30 40 50 60 70 80 90 100 Initial Misalignment (mm) Pe rc en ta ge  o f S uc ce ss fu l T es ts Figure 5.2: Graph shows the percentage (y-axis) of tests in each range of initial misalignment that converge to a final TRE of less than 10 mm when using the simple registration approach with post-scan converted US data.. both the initial misalignment and final TRE across the set of surface points is reported. It can be observed that registration with the post-scan con- verted US performs well for small initial misalignments. However, once the misalignment is greater than 5 mm, many of the tests result in final TRE of greater than 10 mm. In contrast, the registration with the pre-scan con- verted data performs consistently until the initial misalignment approaches 15 mm. Thus, while registration on the phantom data sets yielded similar results from both registration techniques, initial validation on patient data demonstrates that registration via the pre-scan converted US results in an improvement in registration accuracy. In comparison to previous work, Wein et al. [65] performed US to CT registration on a mixture of kidney and liver data sets. Across the twenty 77 5.2. Patient Data 0 − 5 5 − 10 10 − 15 15 − 20 0 10 20 30 40 50 60 70 80 90 100 Initial Misalignment (mm) Pe rc en ta ge  o f S uc ce ss fu l T es ts Figure 5.3: Graph shows the percentage (y-axis) of tests in each range of initial misalignment that converge to a final TRE of less than 10 mm when using the approach with pre-scan converted US data. five patients, the TRE ranged from 3.0 to 22.1 mm, with a mean of 9.0 mm. Thus, we were able to obtain similar results. 78 5.2. Patient Data Figure 5.4: The graphs show the final TRE in mm plotted against the initial misalignment in mm. The results from registration with the post- scan converted US is on the left whereas the results from registration with the pre-scan converted US is on the right. 79 Chapter 6 Conclusion Obtaining accurate and efficient registration of CT to US images is a clinical problem. For many interventional procedures, a diagnostic CT is obtained and used for planning of the surgery. Intraoperative guidance often provides valuable information to the surgeon. Ultrasound is a suitable modality be- cause it is portable, real-time, and does not expose the patient to harmful radiation. The registration of the intraoperative US data with the large high quality diagnostic CT would be useful for robot-assisted partial nephrec- tomy. The challenges of registering CT and US is that the two modalities produce images that are distinctly different from one another. This thesis investigates the use of a simulation-based registration approach that uses basic physics to generate a simulated US from the CT, which is then used for registration of kidney. The advantages of this approach is that it is fully automatic and does not require the segmentation of surfaces or selection of landmarks. In addition, it does not require prior knowledge of the direc- tion of US scanning. Validation was performed on porcine phantom data and also tested on a patient data set. The results showed that directional simulation of US using pre-scan converted data produces similar accuracy to registration using post-scan converted data on the phantom data. How- 80 6.1. Summary of Contributions ever, the initial testing on clinical data suggest that using the directional simulation of US with pre-scan converted data yields a better registration accuracy. This section summarizes the contributions and findings in this thesis and suggests future directions for this research. 6.1 Summary of Contributions An overview of the contributions discussed in this thesis are listed below: • Phantom Construction: Developed a detailed recipe for construct- ing soft tissue phantoms for the purpose of evaluating registration on kidneys. The porcine kidneys were prepared with contrast agent so that they would produce images that were representative of in vivo human CT angiograms. The US images generated were also realistic. The porcine kidneys were encased in agar with steel ball fiducials that were visible in both the US and CT. These served as the gold standard alignment. • Simulation-Based Registration with the CMA-ES Optimizer: Simulation-based registration was tested on the phantom data using the stochastic optimizer CMA-ES. This optimizer was successful be- cause its stochastic approach helped the algorithm to avoid local min- ima in a noisy search space. • Evaluation of the Influence of Directional Simulation-Based 81 6.2. Future Work Registration on Phantom Data: Two variations of simulation- based registration were tested. First a simple implementation where the direction of simulation is vertical in the columns of the CT was tested. In addition, an improvement was made to the US simulation- based registration method by interpolating the CT exactly along the rays of the US beam and generating the simulated US from that im- age. This image was then registered to the pre-scan converted US data eliminating the need for interpolation of the US. From the seven phantom data sets used for validation in this thesis, the two methods produced similar accuracy. • Simulation-Based Registration with Patient Data: CT and US data sets of a patient were registered using the two simulation- based registration techniques. Initial validation on the patient data set demonstrated an improvement in registration accuracy for regis- tration of the pre-scan converted data with a directionally simulated US as compared with simulating the US vertically and registering it to post-scan converted data. 6.2 Future Work While it has been demonstrated that the proposed simulation-based registra- tion algorithm has the potential to register US to CT for the robot-assisted partial nephrectomy application, further improvement and validation is nec- essary in preparation for clinical use. The primary focus of future work 82 6.2. Future Work should be on validating the method on patient data and modifying the work flow to facilitate that objective. This includes addressing issues such as patient data collection and the speed of the existing implementation. • GPU acceleration of the registration method: To achieve the ultimate goal of real-time CT to US registration during the surgical procedures, the registration process needs to be sped up. This can be attained by implementing a GPU accelerated version of the program. Because of the parallel processing of simulation generation, a speed-up of up to 200 times has been reported [38]. This would also speed up the experimental testing and thus advancements of the method can be made and tested more efficiently and across larger data sets. • Determine the gold standard for patient data accurately and efficiently: Further investigation should be made into another method for obtaining a gold standard alignment for future patient data sets. Currently, the use of PCA requires that both data sets be segmented which is time consuming and prone to error. Other approaches such as tracking the US probe or using internal landmarks in the images should be explored. • Extensively validate registration of CT to US with patient data: As more patient data is acquired, further analysis of the reg- istration method should be performed on patient data as additional challenges may be presented with numerous data sets. 83 6.2. Future Work • Investigate the possible need for deformable registration: The patient data set presented in this thesis was acquired while the patient was on breath hold. Thus, there was not a significant deformation be- tween the CT and US images. However, during the surgical procedure, the patient will be breathing and the kidney will move as the surgeon conducts the procedure. Thus, further analysis of kidney motion along with the incorporation of deformable registration may contribute to the utility of this approach in the OR. • Compare simulation-based registration to other registration approaches: Finally, a comparison between simulation-based regis- tration and other registration methods would be useful in determining the best choice for this specific application. In particular, it is possible that a feature based approach would produce good results. Unlike the porcine phantom data, there is a considerable amount of structural information contained in the patient data. Depending on what side the tumourous kidney is on, the spleen or liver may be captured. The spine is present in both the US and the CT and it is possible that large vessels could be used to guide the registration. 84 Bibliography [1] A. Antonelli, A. Cozzoli, M. Nicolai, D. Zani, T. Zanotelli, L. Peruc- chini, S. C. Cunico, and C. Simeone. Nephron-sparing surgery versus radical nephrectomy in the treatment of intracapsular renal cell carci- noma up to 7 cm. European Urology, 53:803–809, 2008. [2] T. Arbel, X. Morandi, R. M. Comeau, and D. L. Collins. Auto- matic non-linear MRI-ultrasound registration for the correction of intra- operative brain deformations. In Proceedings of the 4th International Conference on Medical Image Computing and Computer-Assisted In- tervention, volume 2208, pages 913–922, 2001. [3] M. Aron, G. Haber, and I. S. Gill. Laparoscopic partial nephrectomy. British Journal of Urology International, 99:1258–1263, 2007. [4] Canadian Urological Association. Kidney Tumours. Canadian Urolog- ical Association, Inc., Montreal, QC, 2007. [5] M. A. Audette, F. P. Ferrieb, and T. M. Peters. An algorithmic overview of surface registration techniques for medical imaging. Medical Image Analysis, 4:201–217, 2000. [6] D. C. Barratt, G. Penney, C. S. K. Chan, M. Slomczykowski, T. J. 85 Bibliography Carter, P. J. Edwards, and D. J. Hawkes. Self-calibrating ultrasound- to-CT bone registration. In Proceedings of the 8th International Con- ference on Medical Image Computing and Computer-Assisted Interven- tion, volume 3749, pages 605–612, 2005. [7] A. B. Benincasa, L. W. Clements, S. D. Herrell, and R. L. Galloway. Feasibility study for image-guided kidney surgery: assessment of re- quired intraoperative surface for accurate physical to image space reg- istrations. Medical Physics, 35(9):4251–4261, 2008. [8] P. J. Besl and N. D. McKay. A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 14(2):239–256, 1992. [9] B. Brendel, S. Winter, A. Rick, M. Stockheim, and H. Ermert. Reg- istration of 3D CT and ultrasound datasets of the spine using bone structures. Computer Aided Surgery, 7(3):146–155, 2002. [10] B. Brendel, S. Winter, A. Rick, M. Stockheim, and H. Ermert. Bone reg- istration with 3D CT and ultrasound data sets. International Congress Series, 1256:426–432, 2003. [11] R. Brooks, D. L. Collins, X. Morandi, and T. Arbel. Deformable ultra- sound registration without reconstruction. In Proceedings of the 11th International Conference on Medical Image Computing and Computer- Assisted Intervention, volume 5242, pages 97–140, 2008. [12] L. G. Brown. A survey of image registration techniques. ACM Com- puting Surveys, 24(4):325–376, 1992. 86 Bibliography [13] L. Carrat, J. Tonetti, P. Merloz, and J. Troccaza. Percutaneous com- puter assisted iliosacral screwing: Clinical validation. In Proceedings of the 3rd International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 1935, pages 97–140, 2000. [14] R. C. Chan, S. Sokka, D. Hinton, S. Houser, R. Manzke, A. Hanekamp, V. Y. Reddy, M. R. Kaazempur-Mofrad, and V. Rasche. Non-rigid registration for fusion of carotid vascular ultrasound and mri volumetric datasets. In J. M. Reinhardt and J. P. W. Pluim, editors, Medical Imaging: Image Processing, volume 6144 of Proceedings of SPIE, pages 61442E–1–61442E–8, 2006. [15] T. K. Chen, A. D. Thurston, R. E. Ellis, and P. Abolmaesumi. A real- time freehand ultrasound calibration system with automatic accuracy feedback and control. Ultrasound in Medicine & Biology, 35(1):79–93, 2009. [16] C.L. Cheung, C. Wedlake, J. Moore, S.E. Pautler, A. Ahmad, and T.M. Peters. Fusion of stereoscopic video and laparoscopic ultrasound for minimally invasive partial nephrectomy. In M. I. Miga and K. H. Wong, editors, Medical Imaging: Visualization, Image-Guided Procedures, and Modeling, volume 7261 of Proceedings of SPIE, pages 726109–1–726109– 10, 2009. [17] R. V. Clayman, L. R. Kavoussi, N. J. Soper, S. M. Dierks, S. Meretyk, M. D. Darcy, F. D. Roemer, E. D. Pingleton, P. G. Thomson, and S. R. 87 Bibliography Long. Laparoscopic nephrectomy: initial case report. The Journal of Urology, 146(2):278–282, 1991. [18] Canadian Cancer Society’s Steering Committee. Canadian Cancer Statistics 2010. Canadian Cancer Society, Toronto, ON, 2010. [19] C. Dekomien, M. Mildenstein, K. Hensel, S. Hold, and S. Winter. Regis- tration of intraoperative 3D ultrasound with mr data for the navigated computer based surgery. In Advances in Medical Engineering, chapter 3, pages 252–257. Springer, Berlin, Germany, 2007. [20] E. A. Firle, W. Chena, and S. Wesarga. Registration of 3D US and CT images of the prostate. In CARS 2002: Proceedings of the 16th In- ternational Congress and Exhibition Computer Assisted Radiology and Surgery, pages 527–532, 2002. [21] E. A. Firle, S. Wesarg, and C. Dold. Mutual information based reg- istration for ultrasound and CT datasets. In J. M. Fitzpatrick and M. Sonka, editors, Medical Imaging: Image Processing, volume 5370 of Proceedings of SPIE, pages 1130–1138, 2004. [22] J. M. Fitzpatrick, J. B. West, and C. R. Maurer. Derivation of ex- pected registration error for point-based rigid-body registration. In K. M. Hanson, editor, Medical Imaging: Image Processing, volume 3338 of Proceedings of SPIE, pages 16–27, 1998. [23] A. Gee, R. Prager, G. Treece, C. Cash, and L. Berman. Processing and visualizing three-dimensional ultrasound data. The British Institute of Radiology, 77:S186S193, 2004. 88 Bibliography [24] S. Gill, P. Mousavi, G. Fichtinger, E. Chen, J. Boisvert, D. Pichora, and P. Abolmaesumi. Biomechanically constrained groupwise US to CT registration of the lumbar spine. In Proceedings of the 12th Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention, volume 5761, pages 803–810, 2009. [25] S. Gill, P. Mousavi, G. Fichtinger, D. Pichora, and P. Abolmaesumi. Group-wise registration of ultrasound to CT images of human verte- brae. In M. I. Miga and K. H. Wong, editors, Medical Imaging: Vi- sualization, Image-Guided Procedures, and Modeling, volume 7261 of Proceedings of SPIE, pages 72611O–1–72611O–9, 2009. [26] T. Glatard, X.Pennec, and J. Montagnat. Performance evaluation of grid-enabled registration algorithms using bronze-standards. In Pro- ceedings of the 9th International Conference on Medical Image Com- puting and Computer-Assisted Intervention, volume 4191, page 152160, 2006. [27] G. Haber and I. S. Gill. Laparoscopic partial nephrectomy: Contem- porary technique and outcomes. European Urology, 49:660–665, 2006. [28] I. Hacihaliloglu, A. J. Hodgson, R. Abugharbieh, and R. N. Rohling. Assessing surface localization accuracy in 3D local phase ultrasound images using CT reference images. In 9th Annual meeting of the In- ternational Society for Computer Assisted Orthopaedic Surgery, pages 137–140, 2009. [29] N. Hansen, S. D. Muller, and P. Koumoutsakos. Reducing the time com- 89 Bibliography plexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary Computation, 11(1):1–18, 2003. [30] J. L. Herring, B. M. Dawant, C. R. Maurer Jr., D. M. Muratore, R. L. Galloway, and J. M. Fitzpatrick. Surface-based registration of CT im- ages to physical space for image-guided surgery of the spine: a sensi- tivity study. IEEE Transactions on Medical Imaging, 17(5):743–752, 1998. [31] D. L. G. Hill, P. G. Batchelor, M. Holden, and D. J. Hawkes. Medical image registration. Physics in Medicine and Biology, 46:R1–R45, 2001. [32] B. K. Horn, H. M. Hilden, and S. Negahdaripour. Closed form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America, 4:629–642, 1987. [33] X. Huang, N. A. Hill, J. Ren, and T. M. Peters. Rapid registration of multimodal images using a reduced number of voxels. In K. R. Cleary and R. L. Galloway Jr., editors, Medical Imaging: Visualization, Image- Guided Procedures, and Display, volume 6141 of Proceedings of SPIE, pages 614116–1–614116–10, 2006. [34] B. F. Hutton and M. Braun. Software for image registration: Algo- rithms, accuracy, efficacy. Seminars in Nuclear Medicine, 33:180–192, 2003. [35] J. A. Jensen. FIELD: A program for simulating ultrasound systems. In 10th Nordic-Baltic Conference on Biomedical Imaging, volume 4, pages 351–353, 1996. 90 Bibliography [36] J. A. Jensen and N. B. Svendsen. Calculation of pressure fields from arbitrarily shaped, apodized, and excited ultrasound transduc- ers. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, 39(2):262–267, 1992. [37] M. Keil, P. J. Stolka, M. Wiebel, G. Sakas, E. R. McVeigh, R. H. Taylor, and E. Boctor. Ultrasound and CT registration quality: Elastography vs. classical b-mode. In Proceedings of the 6th IEEE International Sym- posium on Biomedical Imaging, pages 967–970, 2009. [38] O. Kutter, R. Shams, and N. Navab. Visualization and gpu-accelerated simulation of medical ultrasound from CT images. Computer Methods and Programs in Biomedicine, 94(3):250–266, 2009. [39] T. Lange, N. Papenberg, S. Heldmann, J. Modersitzki, B. Fischer, H. Lamecker, and P. M. Schlag. 3D ultrasound-CT registration of the liver using combined landmark-intensity information. International Journal of Computer Assisted Radiology and Surgery, 4(1):79–88, 2009. [40] A. Leroy, P. Mozer, Y. Payan, and J. Troccaz. Intensity-based reg- istration of freehand 3D ultrasound and CT-scan images of the kid- ney. International Journal of Computer Assisted Radiology and Surgery, 2(1):31–41, 2007. [41] K. Lesagea, S. Joniaua, K. Fransisa, and H. V. Poppel. Comparison between open partial and radical nephrectomy for renal tumours: Peri- operative outcome and health-related quality of life. European Urology, 51(3):593–595, 2006. 91 Bibliography [42] E. L. Madsen, M. A. Hobson, H. Shi, T. Varghese, and G.R. Frank. Tissue-mimicking agar/gelatin materials for use in heterogeneous elas- tography phantoms. Physics in Medicine and Biology, 50:5597–5618, 2005. [43] J. B. A. Maintz and M. A. Viergever. A survey of medical image registration. Medical Image Analysis, 2:1–36, 1998. [44] G. Martorana, A. Bertaccini, S. Concetti, A. Franceschelli, R. Schi- avina, E. Severini, F. Sanguedolce, C. Giberti, E. Belgrano, and G. Carmignani. Nephron-sparing surgery for renal cell carcinoma: State of the art and 10 years of multicentric experience. European Urology Supplements, 5:600–609, 2006. [45] S. Milko, E. L. Melvaer, E. Samset, and T. Kadir. A novel method for registration of US/MR of the liver based on the analysis of US dynam- ics. In Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 5761, pages 771–778, 2009. [46] M. H. Moghari and P. Abolmaesumi. A novel incremental technique for ultrasound to CT bone surface registration using unscented kalman filtering. In Proceedings of the 8th International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 3750, pages 197–204, 2005. [47] D. M. Murator, J. H. Russ, B. M. Dawant, and R. L. Galloway Jr. Three-dimensional image registration of phantom vertebrae for 92 Bibliography image-guided surgery: A preliminary study. Computer Aided Surgery, 7(6):342–352, 2000. [48] R. Narayanan, P. N. Werahera, A. Barqawi, E. D. Crawford, K. Shi- nohara, A. R. Simoneau, and J. S. Suri. Adaptation of a 3D prostate cancer atlas for transrectal ultrasound guided target-specific biopsy. Physics in Medicine and Biology, 53(20):N397N406, 2008. [49] J. Olesch, N. Papenberga, T. Langec, M. Conradd, and B. Fischera. Matching CT and ultrasound data of the liver by landmark constrained image registration. In M. I. Miga and K.H. Wong, editors, Medical Imaging: Visualization, Image-Guided Procedures, and Modeling, vol- ume 7261 of Proceedings of SPIE, pages 72610G–1–72611O–9, 2009. [50] J. Patard, A. J. Pantuck, M. Crepel, J. S. Lam, L.Bellec, B. Albouy, D. Lopes, J. Bernhard, F. Guille, B. Lacroix, A. Taille, L. Salomon, C. Pfister, Soulie, J. Tostain, J. Ferriere, C. C. Abbou, M. Colombel, and A. S. Belldegrun. Morbidity and clinical outcome of nephron- sparing surgery in relation to tumour size and indication. European Urology, 52:148–154, 2007. [51] G. P. Penney, D. C. Barratt, C. S. K. Chan, M. Slomczykowski, T. J. Carter, P. J. Edwards, and D. J.Hawkes. Cadaver validation of intensity-based ultrasound to CT registration. Medical Image Analysis, 10:385–395, 2006. [52] M. Peycelon, V. Hupertan, E. Comperat, R. Renard-Penna, C. Vaessen, P. Conort, M. O. Bitker, E. Chartier-Kastler, F. Richard, and 93 Bibliography M. Roupret. Long-term outcomes after nephron sparing surgery for re- nal cell carcinoma larger than 4 cm. The Journal of Urology, 181(1):35– 41, 2009. [53] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever. Mutual- information-based registration of medical images: a survey. IEEE Transactions on Medical Imaging, 22(8):986–1004, 2003. [54] B. C. Porter, D. J. Rubens, J. G. Strang, J. Smith, S. Totterman, and K. J. Parker. Three-dimensional registration and fusion of ultrasound and mri using major vessels as fiducial markers. IEEE Transactions on Medical Imaging, 20(4):354–359, 2001. [55] E. Pospisil, R. Zahiri-Azar, R. N. Rohling, and S. E. Salcudean. Filter- ing and scan conversion of 3D displacement vectors from a 4D curvi- linear transducer. In Proceedings of IEEE International Ultrasonics Symposium, page in press, 2009. [56] T. Reichl, J. Passenger, O. Acosta, and O. Salvado. Ultrasound goes gpu: real-time simulation using cuda. In M. I. Miga and K.H. Wong, editors, Medical Imaging: Visualization, Image-Guided Procedures, and Modeling, volume 7261 of Proceedings of SPIE, pages 726116–1–726116– 10, 2009. [57] A. Roche, X. Pennec, G. Malandain, and N. Ayache. Rigid registration of 3-D ultrasound with MR images: A new approach combining inten- sity and gradient information. IEEE Transactions on Medical Imaging, 20(10):1038–1049, 2001. 94 Bibliography [58] J. D. Schiff, M. Palese, E. D. Vaughan Jr, R. E. Sosa, D. Coll, and J. J. Del Pizzo. Laparoscopic vs open partial nephrectomy in con- secutive patients: the cornell experience. British Journal of Urology, 96:811–814, 2005. [59] R. Shams, R. Hartley, and N. Navab. Real-time simulation of medical ultrasound from CT images. In Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted In- tervention, volume 5242, pages 734–741, 2008. [60] W. Shao, R. Wu, K. V. Ling, C. H. Thng, H. S. Sien Ho, C. W. S. Cheng, and W. S. Ng. Evaluation on similarity measures of a surface- to-image registration technique for ultrasound images. In Proceedings of the 9th International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 4191, pages 742–749, 2006. [61] L. M. Su, B. P. Vagvolgyi, R. Agarwal, C. E. Reiley, R. H. Taylor, and G. D. Hager. Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology, 73:896–900, 2009. [62] J. Tonetti, L. Carrat, S. Blendea, P. Merloz, J. Troccaz, S. Lavalle, and J. Chirossel. Clinical results of percutaneous pelvic surgery. computer assisted surgery using ultrasound compared to standard fluoroscopy. Computer Aided Surgery, 6(4):204–211, 2001. [63] P. Viola and W. M. Wells III. Alignment by maximization of mutual 95 Bibliography information. International Journal of Computer Vision, 24(2):137–154, 1997. [64] J. von Berg, J. Kruecker, H. Schulz, K. Meetz, and J. Sabczynski. A hybrid method for registration of interventional CT and ultrasound images. In Proceedings of the 18th International Congress on Computer Assisted Radiology and Surgery, volume 1268, pages 492–497, 2004. [65] W. Wein, S. Brunke, A. Khamene, M.R. Callstrom, and N. Navab. Au- tomatic CT-ultrasound registration for diagnostic imaging and image- guided intervention. Medical Image Analysis, 12:577–585, 2008. [66] W. Wein, A. Khamene, D. Clevert, O. Kutter, and N. Navab. Simula- tion and fully automatic multimodal registration of medical ultrasound. In Proceedings of the 10th International Conference on Medical Image Computing and Computer-Assisted Intervention, volume 4791, pages 136–143, 2007. [67] W. Wein, B. Roper, and N. Navab. Automatic registration and fusion of ultrasound with CT for radiotherapy. In Proceedings of the 8th Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention, volume 3750, pages 303–311, 2005. [68] S. Winter, B. Brendel amd A. Rick, M. Stockheim, K. Schmieder, and H. Ermert. Registration of bone surfaces, extracted from CT-datasets, with 3D ultrasound. Biomedizinische Technik, 47(1):57–60, 2002. [69] S. Winter, B. Brendel, I. Pechlivanis, K. Schmieder, and C. Igel. Reg- istration of CT and intraoperative 3-d ultrasound images of the spine 96 using evolutionary and gradient-based methods. IEEE Transactions on Evolutionary Computation, 12(3):284–296, 2008. [70] J. Xiang, S. Gill, C. Nguan, P. Abolmaesumi, and R. N. Rohling. Reg- istration of ultrasound to CT angiography of kidneys: a porcine phan- tom study. In M. I. Miga and K.H. Wong, editors, Medical Imaging: Visualization, Image-Guided Procedures, and Modeling, volume 7625 of Proceedings of SPIE, pages 762518–1–762518–8, 2010. [71] W. Zhang, J. A. Noble, and J. M. Brady. Real time 3-d ultrasound to mr cardiovascular image registration using a phase-based approach. In 3rd IEEE International Symposium on Biomedical Imaging, pages 666 –669, 2006. [72] W. Zhang, J. A. Noble, and J. M. Brady. Adaptive non-rigid regis- tration of real time 3D ultrasound to cardiovascular mr images. In Proceedings of the 20th International Conference on Information Pro- cessing in Medical Imaging, volume 4584, pages 50–61, 2007. [73] Z. Zhang. Adaptive region intensity based rigid ultrasound and CT im- age registration. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1–7, 23-28 2008. [74] B. Zitova and J. Flusser. Image registration methods: a survey. Image and Vision Computing, 21:977–1000, 2003. 97 Appendix A UBC Research Ethics Board Certificate This appendix contains the UBC Research Ethics Board Certificate of Ap- proval required for the acquisition of clinical data for the Real-time Image Guidance for Robot-Assisted Laparoscopic Partial Nephrectomy study. 98 Appendix A. UBC Research Ethics Board Certificate The University of British Columbia Office of Research Services Clinical Research Ethics Board – Room 210, 828 West 10th Avenue, Vancouver, BC V5Z 1L8 ETHICS CERTIFICATE OF FULL BOARD APPROVAL PRINCIPAL INVESTIGATOR: INSTITUTION / DEPARTMENT: UBC CREB NUMBER: Christopher Nguan UBC/Medicine, Faculty of/Urologic Sciences H08-02798 INSTITUTION(S) WHERE RESEARCH WILL BE CARRIED OUT: Institution Site Vancouver Coastal Health (VCHRI/VCHA) Vancouver General Hospital Other locations where the research will be conducted: N/A CO-INVESTIGATOR(S): Tim S.E. Salcudean David G. Lowe Robert N. Rohling SPONSORING AGENCIES: - Natural Sciences and Engineering Research Council of Canada (NSERC) - "Real-time image guidance for robot-assisted laparoscopic surgery" PROJECT TITLE: Pilot Study: Real-time Image Guidance for Robot-Assisted Laparoscopic Partial Nephrectomy THE CURRENT UBC CREB APPROVAL FOR THIS STUDY EXPIRES:  April 28, 2010 The full UBC Clinical Research Ethics Board has reviewed the above described research project, including associated documentation noted below, and finds the research project acceptable on ethical grounds for research involving human subjects and hereby grants approval. REB FULL BOARD MEETING REVIEW DATE: April 28, 2009 DOCUMENTS INCLUDED IN THIS APPROVAL: DATE DOCUMENTS APPROVED: Document Name Version Date Protocol: Research Proposal 3.1 April 8, 2009 Consent Forms: Kidney Consent 3.1 April 8, 2009 Advertisements: Advertisement 3.1 April 8, 2009 Other Documents: Health Canada device certificates 1 April 8, 2009 CSA device approval 1 April 8, 2009 NSERC Strategic Grant Award Reviewer Comments 1 April 8, 2009 May 6, 2009 CERTIFICATION: In respect of clinical trials: 1. The membership of this Research Ethics Board complies with the membership requirements for Research Ethics Boards defined in Division 5 of the Food and Drug Regulations. 2. The Research Ethics Board carries out its functions in a manner consistent with Good Clinical Practices. 3. This Research Ethics Board has reviewed and approved the clinical trial protocol and informed consent form for the trial which is to be conducted by the qualified investigator named above at the specified clinical trial site. This approval and the views of this Research Ethics Board have been documented in writing. The documentation included for the above-named project has been reviewed by the UBC CREB, and the research study, as presented in the documentation, was found to be acceptable on ethical grounds for research involving human subjects and was approved by the UBC CREB. 99 Appendix A. UBC Research Ethics Board Certificate Approval of the Clinical Research Ethics Board by one of: Dr. Peter Loewen, Chair Dr. James McCormack, Associate Chair Dr. John Russell, Associate Chair Dr. Caron Strahlendorf, Associate Chair Dr. Stephen Hoption Cann, Associate Chair 100 Appendix A. UBC Research Ethics Board Certificate EXPIRY DATE OF THIS APPROVAL:  April 21, 2011 The University of British Columbia Office of Research Services Clinical Research Ethics Board – Room 210, 828 West 10th Avenue, Vancouver, BC V5Z 1L8 ETHICS CERTIFICATE OF EXPEDITED APPROVAL: RENEWAL PRINCIPAL INVESTIGATOR: DEPARTMENT: UBC CREB NUMBER: Christopher Nguan UBC/Medicine, Faculty of/Urologic Sciences H08-02798 INSTITUTION(S) WHERE RESEARCH WILL BE CARRIED OUT: Institution Site Vancouver Coastal Health (VCHRI/VCHA) Vancouver General Hospital Other locations where the research will be conducted: N/A CO-INVESTIGATOR(S): Tim S.E. Salcudean David G. Lowe Robert N. Rohling SPONSORING AGENCIES: - Natural Sciences and Engineering Research Council of Canada (NSERC) - "Real-time image guidance for robot-assisted laparoscopic surgery" PROJECT TITLE: Pilot Study: Real-time Image Guidance for Robot-Assisted Laparoscopic Partial Nephrectomy APPROVAL DATE:  April 21, 2010 CERTIFICATION: In respect of clinical trials: 1. The membership of this Research Ethics Board complies with the membership requirements for Research Ethics Boards defined in Division 5 of the Food and Drug Regulations. 2. The Research Ethics Board carries out its functions in a manner consistent with Good Clinical Practices. 3. This Research Ethics Board has reviewed and approved the clinical trial protocol and informed consent form for the trial which is to be conducted by the qualified investigator named above at the specified clinical trial site. This approval and the views of this Research Ethics Board have been documented in writing. The Chair of the UBC Clinical Research Ethics Board has reviewed the documentation for the above named project. The research study, as presented in the documentation, was found to be acceptable on ethical grounds for research involving human subjects and was approved for renewal by the UBC Clinical Research Ethics Board. Approval of the Clinical Research Ethics Board by one of: Dr. Peter Loewen, Chair Dr. James McCormack, Associate Chair 101

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0071184/manifest

Comment

Related Items