Gemini hit with 100,000+ prompts in cloning attempt

Google says its flagship artificial intelligence chatbot, Gemini, has been flooded with “commercially motivated” actors who are trying to clone it by prompting it repeatedly, sometimes with thousands of different queries – including one campaign that prompted Gemini more than 100,000 times.

In a report released Thursday, Google said it was increasingly falling victim to “distillation attacks,” or repeated questions intended to trick a chatbot into revealing its inner workings. Google described the activity as “pattern mining,” in which potential imitators probe the system for the patterns and logic that make it work. Attackers appear to want to use this information to create or strengthen their own AI, he says.

The company believes that the culprits are mainly private companies or researchers seeking to gain a competitive advantage. A spokesperson told NBC News that Google believed the attacks came from around the world, but declined to share additional details about what was known about the suspects.

The scale of the attacks against Gemini indicates that they are very likely or soon will also be common against small businesses’ custom AI tools, said John Hultquist, chief analyst at Google’s Threat Intelligence Group.

“We’re going to be the canary in the coal mine for a lot more incidents,” Hultquist said. He declined to name any suspects.

The company considers distillation to be intellectual property theft, she said.

Tech companies have spent billions of dollars developing their AI chatbots, or large language models, and view the inner workings of their best models as extremely valuable proprietary information.

Although they have mechanisms to attempt to identify distillation attacks and block the people behind them, major LLMs are inherently vulnerable to distillation because they are open to anyone on the Internet.

OpenAI, the company behind ChatGPT, last year accused Chinese rival DeepSeek of carrying out distillation attacks to improve its models.

Many of the attacks were designed to uncover the algorithms that help Gemini “reason” or decide how to process information, Google said.

Hultquist said that as more companies design their own custom LLMs trained on potentially sensitive data, they become vulnerable to similar attacks.

“Let’s say your LLM was formed on 100 years of secret thinking about how you trade. In theory, you could distill some of that,” he said.