Open Collections

Generalization performance of deep models for assessing echo image quality in different ultrasound machines

University of British Columbia

Background: In comparison with other advanced imaging techniques (e.g. computed tomography, or magnetic resonance imaging), cardiac ultrasound interpretation is less accurate with higher prevalence of low quality images. The problem can be more severe when non-experts use point-of-care ultrasound (PoCUS) to acquire and interpret images. Artificial intelligence (AI) models that provide image quality rating and feedback can help novice users to identify suboptimal image quality in real-time. However, such models have only been validated on cart-based ultrasound systems typically used in echocardiography labs. In this study, we examined the performance of a AI deep learning image quality feedback model trained on cart-based ultrasound systems when applied to PoCUS devices. Methods: We enrolled 107 unselected patients from an out-patient echocardiography facility at the Vancouver General Hospital. A single sonographer obtained 9 standard image views with a cart-based system and with a hand-held PoCUS device. All the images obtained were assigned image quality ratings by the AI model and by 2 expert physician echocardiographers. Image quality was graded based on percent endocardial border visualization (poor quality = 0-25%; fair quality = 26-50%; good quality = 51-75%; excellent quality = 76-100%). Statistical methods were used to compare the model’s classification performance on cart-based vs. PoCUS data with respect to echocardiographer opinion: percent agreement, weighted kappa, positive predictive value (precision), negative predictive value, sensitivity (recall), and specificity. Results: Percent agreement and weighted kappa were comparable on PoCUS and cart-based ultrasound clips. Overall, the model’s positive predictive value, negative predictive value, sensitivity, and specificity were neither better nor worse on either machine type. Conclusions: We conclude that AI based image quality feedback models designed for cart-based systems can perform well when applied to hand-held PoCUS devices. Researchers may consider using cart-based ultrasound data to train models for PoCUS to overcome data collection and labelling barriers.

Text

2021-04-16

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/77813

Experimental Medicine

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/