Open Collections

UBC Faculty Research and Publications

Automatic C-arm pose estimation via 2D/3D hybrid registration of a radiographic fiducial. Moult, Eric; Burdette, Everette Clif; Song, Danny Y.; Abolmaesumi, Purang; Fichtinger, Gabor; Fallavollita, Pascal 2011

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-Abolmaesumi_SPIE_7964_79642S.pdf [ 1.23MB ]
JSON: 52383-1.0107558.json
JSON-LD: 52383-1.0107558-ld.json
RDF/XML (Pretty): 52383-1.0107558-rdf.xml
RDF/JSON: 52383-1.0107558-rdf.json
Turtle: 52383-1.0107558-turtle.txt
N-Triples: 52383-1.0107558-rdf-ntriples.txt
Original Record: 52383-1.0107558-source.json
Full Text

Full Text

Automatic C-arm Pose Estimation via 2D/3D Hybrid Registration of a Radiographic Fiducial E. Moult1, E.C. Burdette2, D. Y. Song3, P. Abolmaesumi4, G. Fichtinger1, P. Fallavollita1 1  Queen’s University, Canada Acoustic MedSystems Inc, Illinois 3 Johns Hopkins Hospital, Baltimore 4 University of British Columbia, Vancouver 2  ABSTRACT Motivation: In prostate brachytherapy, real-time dosimetry would be ideal to allow for rapid evaluation of the implant quality intra-operatively. However, such a mechanism requires an imaging system that is both real-time and which provides, via multiple C-arm fluoroscopy images, clear information describing the three-dimensional position of the seeds deposited within the prostate. Thus, accurate tracking of the C-arm poses proves to be of critical importance to the process. Methodology: We compute the pose of the C-arm relative to a stationary radiographic fiducial of known geometry by employing a hybrid registration framework. Firstly, by means of an ellipse segmentation algorithm and a 2D/3D feature based registration, we exploit known FTRAC geometry to recover an initial estimate of the C-arm pose. Using this estimate, we then initialize the intensity-based registration which serves to recover a refined and accurate estimation of the C-arm pose. Results: Ground-truth pose was established for each C-arm image through a published and clinically tested segmentation-based method. Using 169 clinical C-arm images and a ±10° and ±10 mm random perturbation of the ground-truth pose, the average rotation and translation errors were 0.68° (std = 0.06°) and 0.64 mm (std = 0.24 mm). Conclusion: Fully automated C-arm pose estimation using a 2D/3D hybrid registration scheme was found to be clinically robust based on human patient data. Keywords: Prostate Brachytherapy, Pose Estimation, Registration, Fluoroscopy  1. INTRODUCTION The clinical motivation for this work is prostate cancer, the second most common cancer in men, diagnosed in 200,000 new patients every year in North America alone. Brachytherapy, which is a definitive treatment of early stage prostate cancer, boasts excellent long-term disease-free survival and is chosen by over 60,000 men each year. The procedure entails permanent implantation of small radioactive isotope capsules (a.k.a. seeds) into the prostate in an effort to eliminate the cancer via radiation. The success of the procedure hinges on achieving the precise placement of the implants needed to provide the required dose distribution. Unfortunately, primarily due to tissue motion, organ deformation and needle deflection, implants rarely turn out to be as planned. Dynamic dosimetry optimization during the procedure would enable the physician to account for deviations from the plan, tailoring the dose so as to eradicate the cancer while minimizing harm to the surrounding healthy tissues. Such a function requires localization of both the prostate and the implanted seeds which is a highly coveted function that is currently unavailable [1]. Prostate brachytherapy is performed using transrectal ultrasound guidance, which provides adequate real-time visualization of the prostate but not of the implanted seeds. At the same time, C-arm fluoroscopy is widely used for visual assessment and occasionally for 3D reconstruction of the implanted seeds, but it cannot show the prostate and other relevant structures. It follows that the fusion of these two complementary imaging modalities is a vital step in achieving dynamic dosimetry. Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, edited by Kenneth H. Wong, David R. Holmes III, Proc. of SPIE Vol. 7964, 79642S © 2011 SPIE · CCC code: 1605-7422/11/$18 · doi: 10.1117/12.877713 Proc. of SPIE Vol. 7964 79642S-1 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  There have been a variety of fluoroscopic implant reconstruction and fusion techniques investigated [2-5]. However, these methods share one common requirement: the relative poses of the fluoroscopy images must be known prior to reconstruction. Pose recovery on C-arm fluoroscopic systems is a major technical problem that at present does not have a clinically practical solution in many areas of application. The relative poses of fluoroscopy images are generally determined in one of following three ways: (i) electronic joint encoders, (ii) optical or electromagnetic trackers and (iii) radiographic fiducials. Fully encoded C-arms are very expensive and thus virtually non-existent in the brachytherapy setting. Optical tracking1,2 requires line of sight which imparts alterations in clinical setup and workflow. Electromagnetic tracking3 overcomes these issues but is susceptible to field distortion from metal objects, such as the Carm itself, and thus compromises on accuracy. The approach of employing fiducial-based radiographic tracking has been explored by several researchers in efforts to mitigate some of the problems inherent in using the other methods of C-arm tracking. In specific, for prostate brachytherapy, Jain et al. developed a fluoroscope tracking (FTRAC) fiducial [6] and validated the device clinically [5]. In addition to spherical beads, straight lines and ellipses were employed due to their invariance under to projection (in the fact that they project as straight line and ellipses). Such parametric curves segment accurately and constrain the optimization during pose recovery, allowing for a mean accuracy of 0.56 ± 0.33 mm in translations and 0.33 ± 0.21° in rotations. The FTRAC design has small dimensions (3x3x5cm), no special proximity requirements to the anatomy and is relatively inexpensive. Jain et al. utilized the FTRAC fiducial by mounting it over the seed insertion needle template using a mechanical connector (see Figure 1). With the goal of achieving a treatment planning system for dosimetric evaluation, after semi-automatic segmentation of the FTRAC fiducial in all C-arm images, Jain et al. reconstructed the implanted seeds in 3D and registered them with the transrectal ultrasound space. In this process, the single point of failure was the pose estimation phase, with sequential semi-automated segmentation of FTRAC features proving fragile in actual field practice. In order to obtain pose estimates while avoiding segmentation, Fallavollita et al. proposed an automatic intensity-based registration scheme which recovered the C-arm pose by generating a binary DRR of the 3D fiducial coordinates and comparing it to a line enhanced C-arm image [7]. The technique was capable of pose estimation which was accurate in 5 of 6 degrees of freedom and had an execution time clocking a slow 90 seconds per C-arm image.  Figure 1. Prostate brachytherapy clinical setup. (Left) TRUS and C-arm fluoroscopy imaging modalities used intra-operatively to guide the intervention. Note the severely limited rotation angle for the C-arm, of approximately 20 degrees about the AP axis. (Topright) The FTRAC fiducial depicted as a wireframe, including ellipses, ball bearings and lines. (Bottom-right) Final semi-automatic segmentation of the FTRAC features within the C-arm image.  1  VectorVision® Navigation System, Brainlab, Inc., Heimstetten, Germany. StealtStation®, Medtronic Surgical Navigation Technologies, Louisville, CO, USA. 3 OEC 9800 FluoroTrak™, GE Healthcare, Waukesha, WI, USA. 2  Proc. of SPIE Vol. 7964 79642S-2 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  To solve the pose estimation problem and hence allow for accurate implant reconstruction, we propose a 2D/3D hybrid registration algorithm comprising of a feature-based registration phase followed by an intensity-based registration phase. The feature-based registration exploits FTRAC geometry in order to provide an initial C-arm pose estimate which is in turn used to initialize the intensity-based registration phase. This hybrid algorithm can thus take advantage of geometrical considerations while remaining largely insensitive to the accuracy of segmentation. The result of such a strategy is a robust and accurate algorithm for pose estimation. In this manner, we have constructed an intuitive and practical solution to a longstanding clinical problem. Our solution blends seamlessly with the current clinical install base meaning it could be rapidly introduced to community care with minimal costs, making a positive impact in the very near future.  2. METHOD We implemented the 2D/3D hybrid registration algorithm using MATLAB R2008a. As we estimated C-arm pose using a single C-arm image, we termed this single-view registration. Metric: Two cost functions were used during the 2D/3D hybrid registration. The first cost function was employed to exploit known FTRAC geometry, measuring the similarity between the segmented ellipses and the projected FTRAC ellipses. The second cost function was an intensity-based metric, implemented by calculating the cross-correlation of two images as a function of position. Transform: As the FTRAC is a rigid mechanical structure, rigid registration suffices. We implemented a transformation of six parameters, with three for Euler angles and three for translation. Optimizer: A Covariance Matrix Adaptive Evolutionary Strategy (CMA-ES) was used. The CMA-ES has several invariance properties. Two of them, inherited from the plain evolution strategy, are (i) invariance to order preserving (i.e. strictly monotonic) transformations of the objective function value and (ii) invariance to angle preserving (rigid) transformations of the search space (including rotation, reflection, and translation), if the initial search point is transformed accordingly. Additionally, the CMA-ES does not require tedious parameter tuning for its application as with the exception of population size, adjustments of the internal parameters are not left to the user. The 2D/3D hybrid registration scheme follows the sequence of ten steps outlined below: STEP 1: C-arm image is filtered using a morphology operator (see Figure 2).  Figure 2. The morphological filtering process developed by Jain et al. prepares the C-arm image for ellipse segmentation. As can be seen, the noise in the original C-arm image has been eliminated leaving only the FTRAC features. Unfortunately, such intensive filtering occasionally leaves the filtered image containing only a small portion of the FTRAC which can complicate segmentation in STEP 2.  STEP 2: First ellipse segmentation phase. An ellipse-fit segmentation method is applied to extract the FTRAC ellipses. Only one ellipse needs to be segmented for this stage to be considered successful (see Figure 3).  Proc. of SPIE Vol. 7964 79642S-3 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  Figure 3. The segmented ellipse (outlined in red in the right image) extracted by running the ellipse segmentation algorithm developed by Jain et al. The left image is once again the morphologically filtered C-arm discussed in STEP 1.  STEP 3: Execution of the Frangi line enhancement filter followed by binarization using an empirically determined threshold level (see Figure 4). The threshold value was obtained using a testbed of 10 random X-ray images.  Figure 4. From left to right, the original C-arm image, the Frangi filtered and Gaussian blurred image and the final production after thresholding.  STEP 4: Determine the validity of the ellipse segmented in STEP 2. We first compute the sum of the pixel intensities (on the Frangi filtered and binarized image) along the contour parameterized by the segmented ellipse. This computed sum is then compared to an empirically determined threshold level to assess the validity of segmented ellipse. The threshold value was obtained using a testbed of 10 random X-ray images and was set to be particularly strict in order to reject any invalid segmentations. Due to the fact that in its clinical environment, the C-arm is limited to a rotation angle of approximately 20 degrees about the AP axis, additional geometrical constraints on the lengths of the major and minor axis were added to further strengthen the validation process. It should be noted that this ellipse validation process provided a 100% success rate by rejecting all erroneous segmentations. If at least one segmented ellipse is deemed valid, the algorithm skips to STEP 8. Otherwise, the algorithm proceeds to STEP 5. STEP 5: Second ellipse segmentation phase. The CMA-ES intensity-based matching coupled with an adaptive cropping technique is executed (see Figure 5). For an initial optimizer estimate, ground truth is perturbed by a ±10° and ±10 mm random perturbation. Following each iteration, the adaptive cropping algorithm masks the C-arm image around the DRR ellipses which are projected from the known 3D coordinates of the FTRAC using the current best pose estimate from the CMA-ES optimizer. This masked image is then used as the input for the ellipse segmentation algorithm described in STEP 2. The masking significantly constrains the search space considered by the ellipse segmentation algorithm, which in turn allows the parameters of the morphological filter to be adjusted for less severe filtering. This ultimately leads to a greater portion of the FTRAC appearing in the filtered image. Furthermore, due to the constrained search space the  Proc. of SPIE Vol. 7964 79642S-4 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  ellipse validation methods (described in STEP 6) can be relaxed. Together these changes increase the likelihood of successful ellipse segmentation. After an ellipse is segmented or the prescribed number of optimizer iterations are executed, the algorithm proceeds to STEP 6.  Figure 5. The adaptive cropping algorithm. From left to right, after each iteration of the optimizer, the 3D FTRAC coordinates are projected to form a DRR image using the current best pose estimate supplied by the CMA-ES optimizer. Next, the C-arm image is cropped around the DRR projection to form the image on the far right (true FTRAC ellipses outlined in red). This cropped image is then used as input for the ellipse segmentation algorithm described in STEP 2.  STEP 6: Ellipse validation stage very similar to the validation procedure described in STEP 4. However, due to the constrained search space provided by the DRR cropping, the pixel intensity threshold, discussed in STEP 4, can be relaxed. Furthermore, since this constrained search space intrinsically encodes the FTRAC geometry, the geometrical constraints can be removed from the ellipse validation, avoiding redundancy. If at least one ellipse is deemed valid, the algorithm skips to STEP 8. Otherwise, the algorithm proceeds to STEP 7. STEP 7: With no ellipses segmented, the algorithm reverts back to the intensity-based registration algorithm developed by Fallavollita et al. to recover a pose estimate. [7] STEP 8: This stage employs a geometrical matching algorithm (see Figure 6) which takes advantage of the known geometry of the FTRAC and segmented ellipses. To achieve feature-based matching, a CMA-ES algorithm coupled with a geometrical metric is utilized. For an initial optimizer estimate, ground truth is perturbed by a ±10° and ±10 mm random perturbation. The geometrical metric computes the similarity between the lengths of the major and minor axis as well as the center coordinates of the segmented ellipse and the lengths of the major and minor axis and the center coordinates of the corresponding DRR ellipse. This feature-based registration step serves two purposes. First, it provides an accurate pose estimate which is used to initialize the intensity-based registration in STEP 10. This pose estimate allows for the population size and number of iterations of the CMA-ES optimizer to be significantly decreased. Furthermore, the supplied pose estimate increases the likelihood that the optimizer will converge. The second advantage that the geometrical matching step provides is that it allows for the Frangi filtered C-arm image to be closely cropped around the FTRAC features. This cropping greatly reduces the image size and thus the computation time of the crosscorrelation calculation performed during the intensity-based registration in STEP 10. STEP 9: Cropping stage in which the image used in STEP 10 is prepared. The image is formed by cropping the Frangi filtered image (prepared in STEP 3) tightly around the DRR projected FTRAC formed by using the pose estimate obtained in STEP 8. The ability to crop the image precisely is a direct consequence of the accuracy of pose estimate achieved in STEP 8. STEP 10: Intensity-based registration algorithm developed by Fallavollita et al. is initialized with the pose estimate recovered in STEP 8 and run on the image prepared in STEP 9 to provide the final, refined, pose estimate.  Proc. of SPIE Vol. 7964 79642S-5 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  Figure 6. Geometrical matching algorithm. The left image is the FTRAC overlay with the ±10° and ±10 mm random ground truth perturbation used to initialize the optimizer. The right image is the overlay after the geometrical matching algorithm has been executed.  We tested our method on the FTRAC fiducial by Jain et al [6]. Following precise segmentation of the FTRAC motifs, their algorithm allowed for an average registration accuracy of 0.56 mm and 0.33º, which we validated in controlled phantom experiments and accepted as ground-truth. As a bound on the accuracy of the C-arm pose estimate required for successful implant reconstruction, we used the results of a study by Jain et al [8]. In the study, Jain et al. found that implant reconstruction required curtailing of the pose estimation error to ± 4° in rotation and to ±2 mm in lateral translation. In 2D/3D registration, it is generally the case that the optimizer and cost metric have difficulties successfully “driving” the depth component of the pose. In implant reconstruction, however, the exact same effect is working for our advantage because the reconstruction metric is similarly insensitive to the depth component of the C-arm pose. Jain et al. found that “reconstruction error is insensitive to miscalibration in origin and focal length errors of up to 50 mm”, inferring that even large depth errors are permissible if all image poses shift together [9]. What follows is that if the prostate is kept near the isocenter, projection and implant reconstruction are largely insensitive to depth.  3. RESULTS The proposed algorithm was tested on a total of 169 clinical images from seven patients. A total of 4225 simulations were performed (twenty five per clinical image). The optimizer failed to converge in 0.5% of the total trials and these failed simulations were removed from the analysis. The proposed hybrid registration workflow was executed in 98.6% of the total trials with STEP 7 (i.e. direct application of the intensity-based approach in [7]) being executed in the remaining 1.4%. Population size and iterations of the CMA-ES optimizer were 50 and 25 respectively. The average rotation and translation errors in all axis were 0.68° (std = 0.06°) and 0.64 mm (std = 0.24 mm). More importantly, the depth parameter value, Tz, was 1.20 ± 0.47 mm (see Table 1). In conclusion, errors in all recovered C-arm pose parameters were found to be well below the values required for accurate 3D seed reconstruction as determined by Jain et al [8]. Lastly, the run time of the algorithm was measured to be an average of 31 seconds per registration. Future implementation in C++ will yield a real-time algorithmic solution for C-arm pose recovery. Table 1. Mean and standard deviations for rotational and translational errors in recovered C-arm pose estimates. ±10° and ±10 mm Tx (mm)  Ty (mm)  Tz (mm)  Rotations (degrees)  0.32 ± 0.14  0.42 ± 0.11  1.2 ± 0.47  0.68 ± 0.06  Proc. of SPIE Vol. 7964 79642S-6 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  4. DISCUSSION Pose reconstruction algorithms relying solely on segmentation and subsequent geometrical reconstructions are inherently susceptible to errors in the segmentation process. In particular, the depth component of the pose, Tz, is particularly sensitive to even slight imperfections in segmentation. Due to the fact that the hybrid registration scheme does not rely explicitly on accurate ellipse segmentation, the presented strategy obtains significant advantages in robustness. This follows directly from the fact that errors stemming from inaccuracies during the segmentation phase are corrected for during the intensity-based registration phase. This advantage in robustness is clearly illustrated in depth value parameter, Tz, being recovered well below 2 mm. Such accuracy in the recovered depth is further underscored by the fact that 37 out of the 169 images had FTRAC lines or ellipses that were partially occluded (see Figure 7). The hybrid registration algorithm also holds significant advantages over algorithms that rely on intensity-based registration alone. The primary enhancement arises due to the accurate initialization and massively reduced search space that is secured before the intensity-based registration phase employed. The combination of these two factors allows the optimizer to converge despite significant amounts noise and/or occlusion of the FTRAC components. This fact was clearly showcased in the executed simulations as the optimizer converged in 99.5% of total trials.  Figure 7. Examples of occluded FTRAC components. As highlighted by the two X-ray images above, many of the tested C-arm images had significant occlusion of the FTRAC lines and ellipses. The ability of the proposed algorithm to recover accurate pose parameters despite this occlusion serves as a further acknowledgement of the robustness of the technique.  5. CONCLUSION In this work, a 2D/3D hybrid registration scheme to accurately recover the pose of a C-arm image was proposed. Final C-arm pose results are well within clinical tolerances as defined in [9]. Operator-free segmentation is utilized in combination with intensity-based registration so that the resulting algorithm can exploit geometrical considerations while remaining largely insensitive to the accuracy of the segmentation. Thus, the proposed algorithm allows for accurate and robust C-arm pose estimation for subsequent 3D seed reconstruction intraoperatively; a critical function of vital importance to the much sought out dynamic dosimetry phase for prostate brachytherapy interventions.  ACKNOWLEGEMENTS Eric Moult was supported by the NSERC Undergraduate Student Research Awards program, Gabor Fichtinger was supported as a Cancer Care Ontario Research Chair and Pascal Fallavollita was supported as an Ontario Ministry of Research Postdoctoral Fellow.  Proc. of SPIE Vol. 7964 79642S-7 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  REFERENCES [1] Nag, S., Ciezki, J.P., Cormack, R. et al., "Intraoperative planning and evaluation of permanent prostate brachytherapy: report of the American Brachytherapy Society," Int J Radiat Oncol Biol Phys 51(5), 1422-30 (2001). [2] Orio, P.F., Tutar, I.B., Narayanan, S. et al., "Intraoperative ultrasound-fluoroscopy fusion can enhance prostate brachytherapy quality," Int J Radiat Oncol Biol Phys 69(1), 302-7 (2007). [3] Westendorp, H., Hoekstra, C.J., van't Riet, A. et al., "Intraoperative adaptive brachytherapy of iodine-125 prostate implants guided by C-arm cone-beam computed tomography-based dosimetry, " Brachytherapy 6(4), 231-7 (2007). [4] Su, Y., Davis, B.J., Furutani, K.M. et al., "Seed localization and TRUS- fluoroscopy fusion for intraoperative prostate brachytherapy dosimetry, " Computer Aided Surgery 12(1), 25-34 (2007). [5] Jain, A.K., Deguet, A., Iordachita, I. et al., "Intra-operative Guidance in Prostate Brachytherapy Using an Average Carm," Tenth International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Lecture Notes in Computer Science 4792, 9-16 (2007). [6] Jain, A.K., Mustufa, T., Zhou, Y. et al., "FTRAC--a robust fluoroscope tracking fiducial," Medical Physics 32(10), 3185-98 (2005). [7] Fallavollita, P., Burdette, C., Song, D., Abolmaesumi, P., and Fichtinger G., "C-arm tracking by intensity-based registration of a fiducial in prostate brachytherapy," 1st International Conference on Information Processing in Computer-Assisted Interventions, Lecture Notes in Computer Science, 45-55 (2010). [8] Jain, A., Fichtinger, G., "C-arm Tracking and Reconstruction without an External Tracker," Ninth International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Proceedings Lecture Notes in Computer Science 4191, 494-502 (2006). [9] Jain, A., Kon, R., Zhou, Y. et al., "C-arm calibration - is it really necessary?" Eighth International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Proceedings in Lecture Notes in Computer Science 3749, 639-646 (2005).  Proc. of SPIE Vol. 7964 79642S-8 Downloaded from SPIE Digital Library on 15 Aug 2011 to Terms of Use:  


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items