Elon Musk's xAI recently weakened content guard rails for image generation in the Grok AI bot. This led to a new spate of non-consensual sexual imagery on X, much of it aimed at silencing women on the platform. This, along with the creation of sexualized images of children in the more compliant Grok, has led regulators to begin investigating xAI. In the meantime, Google has rules in place for exactly this eventuality—it's just not enforcing them.
It really could not be more clear from Google's publicly available policies that Grok should have been banned yesterday. And yet, it remains in the Play Store. Not only that—it enjoys a T for Teen rating, one notch below the M-rated X app. Apple also still offers the Grok app on its platform, but its rules actually leave more wiggle room.
App content restrictions at Apple and Google have evolved in very different ways. From the start, Apple has been prone to removing apps on a whim, so developers have come to expect that Apple's guidelines may not mention every possible eventuality. As Google has shifted from a laissez-faire attitude to more hard-nosed control of the Play Store, it has progressively piled on clarifications in the content policy. As a result, Google's rules are spelled out in no uncertain terms, and Grok runs afoul of them.
Google has a dedicated support page that explains how to interpret its "Inappropriate Content" policy for the Play Store. Like Apple, the rules begin with a ban on apps that contain or promote sexual content including, but not limited to, pornography. That's where Apple stops, but Google goes on to list more types of content and experiences that it considers against the rules.
"We don’t allow apps that contain or promote content associated with sexually predatory behavior, or distribute non-consensual sexual content," the Play Store policy reads (emphasis ours). So the policy is taking aim at apps like Grok, but this line on its own could be read as focused on apps featuring "real" sexual content. However, Google is very thorough and has helpfully explained that this rule covers AI.
Recent additions to Google's Play Store policy explicitly ban apps like Grok.
Credit:
Google
The detailed policy includes examples of content that violate this rule, which include much of what you'd expect—nothing lewd or profane, no escort services, and no illegal sexual themes. After a spate of rudimentary "nudify" apps in 2020 and 2021, Google added language to this page clarifying that "apps that claim to undress people" are not allowed in Google Play. In 2023, as the AI boom got underway, Google added another line to note that it also would remove apps that contained "non-consensual sexual content created via deepfake or similar technology."
Sound like any apps you know?
The archetype of a bannable app, approved for teens
Taken together, Google's description of bannable apps describes Grok's app to a tee. Google made these additions as new threats became apparent, knowing that developers would try to publish AI-undressing apps in the Play Store. The company did not, apparently, think the world's richest person would be the one pushing digital humiliation tools on its platform. And Google's response to this situation so far has been to do nothing.
The backlash to xAI's loosened restrictions prompted the company to limit access to image editing slightly. You can no longer edit images on X without paying for a premium plan. However, the Grok app does not have that limitation. Anyone who downloads Grok can use it to create non-consensual sexual content.
Since the app is cleared for teens, even devices with parental controls enabled will permit 13- to 17-year-olds to download Grok. There is no paywall, and you don't even have to log in before editing your first image. The app does ask the user to confirm their birth year, but teenagers would never lie about that, right?
This is not xAI's first problem with non-consensual sexual content. Last year, the AI was widely used to create fake Taylor Swift nudes. However, in that case, users were simply prompting the bot with the singer's name—Grok can create entirely new images of famous people because the training data includes real images of them. Grok's newer ability to "edit" images of people is a different and more insidious feature because it can turn anyone into an AI plaything.
Ars has reached out to Google to ask why Grok has not been removed and why it has retained a Teen rating. The company has declined to make a statement at this time. So we're left with a policy that explicitly bans apps like Grok, but Google is taking no action to enforce those policies, allowing impressionable teenagers and unsavory weirdos to use it to sexualize real people.
This is how pcTattletale used to describe its install process.