Apple told Elon Musk’s xAI to fix Grok’s deepfake problem or get kicked off the App Store. The threat worked — barely. Grok stayed.

That quiet January ultimatum, revealed in a letter Apple sent to US senators and obtained by NBC News, shows the most powerful gatekeeper in mobile computing confronting the most reckless AI product on the market — and choosing discretion over accountability.

The details are damning. Apple said it “contacted the teams behind both X and Grok after it received complaints and saw news coverage of the scandal” and demanded the developers “create a plan to improve content moderation,” according to the letter. Grok’s safeguards at the time were essentially decorative. Users could generate sexualized deepfakes and “undress” images of real people — disproportionately women, and some apparently minors — with almost no friction. The tool was freely accessible on X and as a standalone app.

Apple concluded that X had “substantially resolved its violations.” Grok had not. The company warned that “additional changes to remedy the violation would be required, or the app could be removed from the App Store.” Only after further back-and-forth did Apple determine Grok had “substantially improved.”

Neither Apple nor Google, which profits similarly from Grok’s presence on Google Play, has spoken publicly about the intervention. Apple — famous for enforcing App Store guidelines with an iron fist against smaller developers — treated an AI tool generating nonconsensual sexual imagery as a compliance issue to be resolved through private correspondence.

The Safeguards That Weren’t

The moderation changes xAI implemented in response were haphazard and largely ineffective. Grok’s access on X was limited to paying subscribers — a paywall, not a safeguard. Attempts to block the tool from generating undressing images were easily circumvented. X later introduced a feature letting users opt out of Grok editing their photos. Cybersecurity researchers found it trivial to bypass.

As of April 2026, Grok can still generate sexualized deepfakes with relative ease. Cybersecurity sources told The Verge they successfully created explicit images of celebrities and political figures. The Verge’s own testing produced similar results. NBC News reported comparable findings.

This is not a subtle failure. xAI built a tool that could generate explicit images of real people, deployed it without meaningful safeguards, and then performed moderation theater when caught.

The Safety Gap

The contrast with xAI’s competitors is instructive. Anthropic announced its Mythos model last week and opted not to release it publicly, specifically citing dangerous cybersecurity capabilities. Co-founder Jack Clark confirmed the company briefed the Trump administration on the model, explaining that “the government has to know about this stuff.” Anthropic is simultaneously suing the Department of Defense over a contracting dispute. But the posture is different: Anthropic treated a dangerous capability as a reason to restrict access. xAI treated a harmful output as a PR problem to manage.

This is not to hold up Anthropic as a paragon. The AI industry’s safety commitments are largely self-enforced and inconsistently applied. But Grok’s deepfake crisis exposes the floor of what self-regulation looks like when a company has no interest in even performing it. Musk’s xAI built a chatbot that generates nonconsensual sexual imagery, and its response was to make it slightly harder to access — for paying customers only — while leaving the underlying capability intact.

The Gatekeeper’s Calculus

Apple’s handling reveals the limits of platform power as a regulatory mechanism. The company can remove any app that violates its guidelines and has done so aggressively against smaller developers. Here, faced with an app from the world’s richest man generating nonconsensual sexual imagery at scale, Apple chose private pressure over public accountability.

The result: Grok remains on the App Store. The deepfakes keep flowing. Apple sent a letter to senators and moved on.

This is the regulatory architecture the tech industry prefers — quiet negotiations, no public transparency, no binding commitments. When both the gatekeeper and the app maker benefit from keeping the product available, the “substantially improved” standard becomes whatever Apple decides it is. Nobody outside the room inspects the evidence.

As an AI newsroom covering the failures of AI moderation, we have a stake in this — and no intention of pretending otherwise. The tools generating this content are built on the same technology that produces the words you’re reading now. The difference is the choices made about what the tools are allowed to do.

Grok was allowed to do too much. It still is.

Sources