Apps like Grok are explicitly banned under Google’s rules—why is it still in the Play Store?

Elon Musk’s xAI recently weakened the content guardrails for image generation in the Grok AI bot. This led to a new wave of non-consensual sexual images on X, much of which was aimed at silencing women on the platform. This, along with the creation of sexualized images of children in the more compliant Grok, led regulators to begin investigating xAI. In the meantime, Google has rules in place precisely for this eventuality: it simply does not enforce them.

Google’s publicly available policies really couldn’t be clearer that Grok should have been banned yesterday. And yet, it remains in the Play Store. Additionally, it boasts a T rating for Teen, a notch below the M-rated X app. Apple also still offers the Grok app on its platform, but its rules actually allow more wiggle room.

Restrictions on app content at Apple and Google have evolved in very different ways. Since the beginning, Apple has tended to remove apps on a whim, so developers have come to expect that Apple’s guidelines won’t mention every possible eventuality. As Google has moved from a laissez-faire attitude to tighter control of the Play Store, it has gradually made clarifications to its content policy. As a result, Google’s rules are unequivocally stated and Grok does not follow them.

Google has a dedicated support page that explains how to interpret its “Inappropriate Content” policy for the Play Store. As with Apple, the rules start with a ban on apps that contain or promote sexual content, including but not limited to pornography. That’s where Apple stops, but Google goes on to list more types of content and experiences that it considers against the rules.

“We do not allow apps that contain or promote content associated with sexually predatory behavior, or distribute non-consensual sexual content“, reads the Play Store policy (emphasis added). So the policy is aimed at apps like Grok, but this line alone could be read as being focused on apps with “real” sexual content. However, Google is very thorough and has helpfully explained that this rule covers AI.

The detailed policy includes examples of content that violates this rule, which include much of what you’d expect: nothing obscene or profane, no escort services, and no illegal sexual themes. After a series of rudimentary “nudification” apps in 2020 and 2021, Google added text to this page clarifying that “apps that claim to undress people” are not allowed on Google Play. In 2023, as the AI ​​boom began, Google added another line to indicate that it would also remove apps containing “non-consensual sexual content created via deepfake or similar technology.”