Artificial Intelligence Security: Threats and Countermeasures

In the data collection phase of the AI life cycle, main security threats include databases, fake data, data breaches, and sensor attacks; defense strategies include data sanitization and data government.

“(1) data collection phase—main security threats include databases, fake data, data breaches, and sensor attacks; defense strategies include data sanitization and data government;”

Hu et al present an overall framework of attacks and defense strategies based on five phases of the AI life cycle.

“Similarly, Hu et al [57] present an overall framework of attacks and defense strategies based on the following five phases of the AI life cycle:”

In the data processing phase of the AI life cycle, image scaling is the main threat; recommended defense strategies include image reconstruction and data randomization.

“(2) data processing phase—image scaling is the main threat; recommended defense strategies include image reconstruction and data randomization;”

Adversarial training uses carefully crafted adversarial examples to attack the training model to check for the integrity and robustness of outputs.

“The method uses carefully crafted adversarial examples to attack the training model to check for the integrity and robustness of outputs [57,79].”

Malicious parties also intensively explore adversarial training and use adversarial examples to attack training models to generate incorrect outcomes.

“By contrast, malicious parties also intensively explore this method and use adversarial examples to attack training models to generate incorrect outcomes [57].”

Modifying training processes and models as well as training methods may help to control AI risks in the model training and building phase.

“Modifying training processes and models as well as training methods may also help to control AI risks in this phase [57].”

Data standardization improves the quality of data collection.

“Data provenance and authentication metrics can be used to prevent collecting data from untrustworthy sources; detecting and filtering methods can be used to identify and filter poisoned data; and data standardization improves the quality of data collection [57].”

Health care AI systems, especially generative AI systems, are susceptible to code extraction and information extraction, leading to security and privacy breaches.

“Similar to other AI systems, health care AI systems, especially generative AI systems, are susceptible to code extraction and information extraction (eg, black-box, gray-box, and white-box attacks), leading to security and privacy breaches [57].”

Availability attacks such as denial of service can be used to flood conversational health AI applications to assess their resilience and availability before deployment, and integrity attacks with poisoned data can be used to test the stability of model performance and generalizability.

“For example, availability attacks such as denial of service can be used to flood conversational health AI applications to assess their resilience and availability before deployment, and integrity attacks with poisoned data can be used to test the stability of model performance and generalizability [57].”

AI systems interfacing with other hardware and software may face new security and privacy threats that have never existed before.

“In addition to AI-specific security and privacy threats, AI systems interfacing with other hardware and software may face new security and privacy threats that have never existed before [57].”