The threat frontier is shifting toward generative AI
New findings from research firm Gartner reveal that attacks centered on generative AI are expanding globally. In a recent survey, 62% of organizations reported experiencing at least one incident that leveraged generative AI to facilitate wrongdoing. The pattern crosses industries, geographies, and attack surfaces, signaling a need for AI-aware security strategies.
How attackers use generative AI
Adversaries deploy generative AI to automate social engineering, craft convincing phishing messages, generate realistic deepfakes, or tailor malware and prompts to bypass standard defenses. Prompt injection can manipulate chatbots into revealing sensitive information or performing unintended actions. Attacks may occur at multiple layers, from endpoints to the cloud, and can be blended with traditional software vulnerabilities to maximize impact.
Common attack modalities
- Phishing and social engineering enhanced by natural language generation
- Deepfake audio and video to impersonate executives or customers
- Prompt-injection and jailbreak techniques to override guardrails
- Data exfiltration via AI-generated steganography or covert channels
- Model poisoning or data poisoning in training pipelines
Who is at risk?
All sectors are potential targets, but financial services, healthcare, and critical infrastructure show heightened risk due to the value of data and the potential for disruption. Smaller organizations may be particularly vulnerable to highly targeted social engineering, while larger enterprises face complex supply-chain and cloud-attacks that leverage AI to scale.
Defensive playbook for an AI-first threat landscape
Organizations should adapt with an AI-first security program. Key steps include:
- Adopt zero-trust principles and enforce strong identity controls, MFA, and device security
- Implement prompt hygiene: guardrails, input validation, and continuous monitoring of AI tools
- Apply least-privilege access (RBAC) to AI workflows and data stores
- Use AI-enabled threat detection to identify anomalous language, behavior, or data flows
- Conduct regular red-teaming and tabletop exercises focusing on AI-enabled scenarios
- Strengthen incident response with playbooks for AI-driven incidents and rapid containment
Gartner’s take and the path forward
Industry analysts emphasize that the rise of generative AI attacks is not a one-time spike but a sustained shift in the threat landscape. Organizations that invest in governance, model risk management, and proactive monitoring will be better prepared to detect, deter, and respond to AI-driven threats. The takeaway is clear: security must evolve in parallel with AI innovation, embedding AI risk management into every layer of IT and business operations.
Preparing for an AI-first threat landscape
As generative AI becomes more capable, attackers will continue to refine their methods. By combining technical controls with user awareness and executive-level governance, companies can reduce exposure and resilience against AI-enhanced attacks. The time to act is now.
