Open Collections

Hallajiyan, Mohammadreza

University of British Columbia

The growing integration of Artificial Intelligence (AI) and Machine Learning (ML) in medical systems calls for effective security measures to address emerging threats. However, much of the current research underestimates the risk of adversaries introducing false inputs through peripheral devices during inference, causing mispredictions in patient diagnostics and wrong treatments. These risks are challenging to anticipate and address in the design phase, as the system assembly typically occurs during actual use by end users. To address this gap, we introduce SAM, a technique that enables security analysts to perform System-Theoretic Process Analysis for Security (STPA-SEC) on AI/ML-enabled medical devices during the design phase. SAM models the medical system as a control structure, with the ML engine as the controller and peripheral devices as well as their interactions as potential points for false data injection. It interfaces with state-of-the-art vulnerability databases and Large Language Models (LLM) to automate the discovery of vulnerabilities and generate a list of possible attack paths. We demonstrate the usefulness of SAM through case studies on five FDA-cleared medical devices: a blood glucose management system, a bone mineral density measurement software, an automatic diabetic retinopathy detection software, an automatic embryos analysis software, and a noninvasive pulse and breathing rate measurement software. Our results demonstrate that SAM accurately identifies vulnerable points linked to different technologies in these case studies, retrieves known vulnerabilities, and automatically generates potential adversarial steps to exploit them with an average accuracy of 95% across three different well-known LLMs. SAM enables security analysts and device manufacturers to streamline the security assessment of AI/ML-enabled medical devices during both the design and post-deployment phases. This proactive strategy helps mitigate potential patient harm while reducing costs linked to post-deployment security measures.

Text

2024-12-05

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/89791

Electrical and Computer Engineering

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/