Cybercriminals are increasingly turning to artificial intelligence (AI) to perfect their phishing attacks. A recent report by Microsoft shows how attackers are using the same technologies as security researchers – but with the opposite objective.

The campaign observed started via a compromised email account belonging to a small business. The perpetrators used this account to send deceptively genuine-looking messages, purportedly from a file-sharing service. Attachments that looked like normal document files directed recipients to fake websites in order to steal their login details. The ‘self-addressed’ tactic was particularly sophisticated: the sender and recipient addresses were identical, while the actual targets remained hidden in the BCC field. This allowed simple security mechanisms to be circumvented.

AI as a cloak of invisibility

The type of concealment was particularly striking: the malicious code was hidden in an SVG file that had apparently been generated or modified using AI. Microsoft determined that the code had a synthetic structure – extremely complex, meaningless in terms of content, but peppered with business terms to feign legitimacy. Security researchers used their own Security Copilot to analyse the file and concluded that it would have been almost impossible for a human to create such a construct manually.

Arms race between attackers and defenders

The incident highlights a growing arms race: While companies are using AI to detect and stop threats more quickly, cybercriminals are experimenting with the same tools to create credible bait, disguise malicious code and imitate legitimate content. The campaign is seen as an example of the new era of cybercrime, in which AI significantly increases the effectiveness, reach and concealment of attacks.

Recommendations for action for companies

The report shows that AI has long been a key factor in cyber security – both on the attacker and defender sides. Companies should continuously modernise technical protection systems, train employees in dealing with phishing and raise awareness of AI-supported threats. Only the combination of advanced technology and human vigilance offers long-term protection against the new generation of AI-supported attacks.