The fake Rolex problem: How AI turned amateur attackers into nation-state threats
Have you ever owned a really good fake Rolex? Not the forty dollar beach version. The kind that makes a jeweler think.
The movement is Swiss. The glass is sapphire. The bracelet is made from 904L steel, the same alloy that Rolex actually uses. Each component is authentic, from real suppliers, assembled with real know-how.
Article continues below
Alan LeFort is CEO of StrongestLayer.
What sets this moment apart from all previous threat evolutions is this: AI tools haven’t just made these attacks more convincing. This made the expertise needed to build them almost free.
The research was spot on. The prediction was wrong.
Advances in AI-automated spear phishing
In November 2024, researchers at the Harvard Kennedy School published a study that should have redefined how the industry views AI-based attacks.
Led by Fred Heiding and co-authored with Bruce Schneier, the paper found that fully automated AI spear phishing emails achieved a 54% click-through rate, statistically identical to emails created by human experts and 350% higher than generic phishing. Cost per attack: about four cents.
AI doesn’t just make phishing cheaper. This makes it profitable on a large scale for almost everyone. Researchers have calculated that AI-enhanced phishing can increase attacker profitability by 50 times compared to traditional methods.
The hypothesis behind this change was that attackers would rely on AI for personalization: better writing, more convincing lures, deeper context. This turned out to be only part of the story.
The most important change is structural.
Analysis of thousands of real-world attacks that bypassed major enterprise email security systems shows that the biggest change isn’t in improving the language. It’s a better build.
Attackers don’t just personalize emails. They customize kill chains.
Expanding skills, not just cutting costs
Harvard’s framework, that AI is a collapsing cost, was accurate to the extent possible. But cost reduction is the first-order effect. The second-order effect is more substantial. When you eliminate the expertise barrier, you not only get more phishing. You get a skill ceiling collapse.
The type of kill chain engineering that now appears in common attacks – dozens of evasion techniques, hundreds of unique combinations, tailored to the specific behaviors of Microsoft Defender or Google Workspace – once required nation-state resources. It required operators who knew the company’s security stacks well enough to design custom paths through them.
This level of ability is no longer rare. The floor rose. Medium attackers now produce what was once considered an advanced persistent threat.
This is a different problem than “AI makes phishing emails more convincing.”
And that requires a different response.
What a custom kill chain actually looks like
In the observed data, more than half of the attacks use four or more evasion techniques simultaneously. The average attack combines a little more than four. Combined attacks are increasing rapidly from year to year.
A representative chain: a QR code attack targeting a Google Workspace tenant. The email contains no URL, only an embedded QR code. Automated scanners have nothing to scan. The QR resolves to a CAPTCHA gate, blocking sandbox environments. Behind this is a multi-hop redirect through trusted cloud providers – AWS, then Cloudflare – leading to a credentials collection page masquerading as a Microsoft MFA prompt. The language reflects real authentication requests that users see regularly.
Each stage destroys a different layer of defense.
No URL goes against link analysis.
CAPTCHA blocks sandboxing.
The redirect chain escapes reputation filtering.
MFA spoofing bypasses human judgment.
Now compare that to an entirely different attack model, like smuggling HTML into a PDF where the payload assembles in the browser and never exists as a file in transit. There is almost no overlap in detection logic between the two.
Rules that catch one are blind to the other.
The combinatorial space is too large. And the AI is expanding it faster than defenders can keep up.
Why rules can’t solve a skills problem
The Harvard study found that AI models still needed human intervention to compete with expert attackers in 2023. By 2024, this gap had closed. Fully automated systems have reached parity. They are expected to be squarely beyond human expertise.
Traditional secure email gateways were designed for a different threat model.
– Pattern matching works when patterns repeat.
– Signature detection works when attackers cannot continually mutate.
– Reputation filtering works when malicious infrastructure appears different from legitimate infrastructure.
None of these assumptions are valid anymore.
Consider the same QR-based attack evaluated using different approaches.
A rules-based system sees no URLs, no attachments, no known flags. Verdict: clean.
A machine learning system flags a recently registered domain and assigns it a medium trust level. In most environments this is effectively ignored.
A reasoning-based approach asks a different question: why does a new domain send an MFA flow through a CAPTCHA-secured redirect chain to a user without any prior authentication activity linked?
That’s the difference.
The coins are legit. The intention is not.
What to do with it
AI has not only increased the volume of attacks. This increased the base ability of attackers.
The question is not whether existing defenses were built for this environment. This was not the case.
The question is how detection evolves when attacks are no longer defined by individual indicators, but by the way in which these indicators are assembled.
Three things are worth pressure testing.
– How well does your detection approach handle multi-stage attacks where each attack component is “not suspicious” in isolation?
– Can your systems evaluate intent, so that seemingly unproven attacks can be detected when traditional security reputation checks fail (bad URL, bad file, bad domain).
– How quickly can you scan, verify and contain these advanced threats with your current stack as response times compress rapidly?
It’s no longer about catching bad emails. The question is whether you can recognize the intention before it’s too late.
We ranked the best antivirus software.
This article was produced as part of TechRadar Pro Insightsour channel that features the best and brightest minds in today’s tech industry.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you would like to contribute, find out more here: https://www.techradar.com/pro/perspectives-how-to-submit
