Military Application of Artificial Intelligence: Identifying and Reducing Security Threats

In the realm of technology, artificial intelligence (AI) is becoming increasingly prevalent, influencing decisions about what to eat, wear, and purchase. However, its impact extends far beyond our daily lives, reaching into the sphere of great power competition.

The Army Cyber Institute's Competition in Cyberspace Project (C2P) is shedding light on this crucial issue. One of the key concerns identified is the potential for reverse image search attacks on AI systems.

Reverse image search involves gaining access to an AI system, often during maintenance or storage, or through a network intrusion or battlefield capture, to understand and exploit its workings. This can be particularly concerning in the context of competition, as it could facilitate the compromise of classified intelligence.

To conduct an inference attack, an adversary needs the ability to send inputs to a model and observe the outputs. The goal is to extract what data the AI system used in its learning process, a subtle distinction with significant implications for models trained on sensitive or classified data.

Adversarial methods can be used to attack AI systems, particularly in the development, operation, and maintenance phases. Direct knowledge of how an AI is making its decisions can enable an adversary to predict or avoid responses, posing a significant threat to operational security.

Reverse image search could enable an adversary to learn what the target identification model considers to be a threat and develop its own version of the AI system. This could potentially be used to gain an advantage in the ongoing competition.

Protecting systems against reverse image search can be challenging due to mission requirements that may necessitate many queries or weighted outputs. Managing risks associated with inference attacks will involve policy decisions about when to use sensitive or classified data in the training of AI systems.

It is crucial to understand the risks introduced by AI systems and their potential strategic advantages in the era of great power competition. As AI continues to evolve and permeate various aspects of our lives, it is essential to approach its development and deployment with a keen understanding of the potential threats it poses, and to take steps to mitigate those risks.

In addition to the concerns surrounding reverse image search, other methods such as poisoning and evasion are also being used to compromise AI systems. Poisoning involves altering the data the AI system uses in training, leading to flawed learning, while evasion targets how the AI's learning is applied, often by slightly modifying image pixels to cause misclassification.

The use of AI in warfare is another area of concern, with reports suggesting that an autonomous, AI-augmented rifle was used in the November 2020 assassination of a top Iranian nuclear scientist. Countries like Russia and China are rapidly developing and deploying AI-enabled irregular warfare capabilities, adding another layer of complexity to the competition.

The author of the influential article "Compete and Win: Envisioning a Competitive Strategy for the Twenty-First Century" is John A. Quelch. His work underscores the importance of understanding the strategic implications of AI in the context of great power competition. As we move forward, it is essential that we continue to explore these issues and develop strategies to protect our interests in this rapidly evolving landscape.