.png){kind=logo}
AI-generated phishing mimics trusted communications, increasing success rates against targets significantly.
Cybercriminals use AI to automate attacks, reconnaissance, impersonation, and deception.
Strong governance, awareness training, and detection tools remain critical defenses today.
As AI becomes increasingly integrated into business processes, cyber criminals have begun to weaponize this technology. The way that these cyber criminals typically use phishing techniques has dramatically changed over time, moving from poorly written emails sent en masse (generalized) to very specific/targeted email campaigns that seem very convincing. In addition to this evolution in phishing attack techniques, a new type of phishing attack has begun to appear on the market; that new type of phishing attack is thought to be “vibe coded” phishing. As organizations around the world prepare for this unknown and more advanced threat, the cybercriminal landscape continues to look increasingly difficult to navigate.
Vibe coded phishing is a type of AI generated phishing email, designed to mirror the tone, your organization’s communication style and content, to a specific person or business. Instead of creating mass mailers with visible red flags associated with them (due to poor English grammar usage), the cyber criminals are taking publicly available information (including any social media activity posts and communications about their target) and utilizing AI tools to create an email that appears to be a legitimate communication sent from a trusted source (such as the target’s manager, vendor(s), colleagues, and/or business partner(s)). Individuals who receive these types of emails are significantly more prone to trust the information in the email, click on the malicious link(s), divulge personally identifiable information (PII), and/or authorize unauthorized transactions.
One of the biggest problems surrounding the use of AI is that use of this technology is providing significantly greater access for the cybercriminal community to perpetrate advanced cyber-attacks on their victims. Individuals do not need to be technically savvy to generate a phishing email or create a fake website or carry out a social engineering attack anymore. Generative AI can assist in the mass production of tailored email content, automate reconnaissance processes, and generate realistic audio and/or video impersonations, thereby allowing cyber criminals to greatly expand their businesses in terms of volume while still providing a high degree of individual personalization increasing their chances of being successfully able to target and execute their attacks on unsuspecting individuals and/or organizations.
Risks go beyond malware phishing. The advent of artificial intelligence means that AI systems themselves are now targets for attackers who are trying new methods, such as attack prompt injection, data poisoning, model manipulation or manipulating an AI application to allow them entry to unauthorized access or more broadly AI-enabled applications. Organizations deploying AI applications without appropriate oversight/safety nets or governance could inadvertently expose risking their sensitive corporate data, trade secrets, and/or their customers' private data. In addition, employees who are using unauthorized AI applications (often referred to as “Shadow AI”) also contribute to the risk by providing confidential proprietary or sensitive data to non-securely hosted and/or unregulated (for example: publically available cloud) services.
Organizations should develop a multifunctional strategy for defending against AI-based cyber threats by leveraging appropriate technology, implementing effective Policies that govern how employees will interact with and implement AI, and implementing employee security awareness training programs applicable to emerging threats. To appropriately protect themselves against emerging AI threats, organizations need to invest in cyber threat detection systems that provide advanced threat detection capabilities that can identify AI-generated attacks and/or anomalous behaviors.
Organizations should also continue to articulate emerging phishing techniques to all employees through continued security awareness training (for example: ongoing employee training that includes defense against deep fake videos or examples of AI generated social engineering attempts), to reinforce and clearly define their Governance standards for use of AI. Organizations will need to position themselves to stay ahead of the evolving nature of cyber threats by implementing proactive security strategies, having a continually vigilant approach to cyber security, and developing a strong understanding of the impacts that AI could have on cybercriminal activities.
