One paid subscription, one random image, and around one minute is all it takes to “nudify” a minor.
This month, Elon Musk’s xAI fought Minnesota’s AI “nudification” ban, and won. It won a temporary court order blocking Minnesota from enforcing a ban on AI tools that can be used to generate realistic nude images of identifiable people, including minors.
xAI argues that the ban conflicts with the First Amendment. It says stopping nudification is no reason to restrict a technology that also produces protected expression. The state disagrees, saying Grok Imagine’s nudification function is a product feature and therefore not protected speech.
However, the one-sentence order in favor of xAI gave no reasoning and left the constitutionality of the statute for later review.
This is not xAI’s first court rodeo. Just 13 days before suing Minnesota, the company sued one of its own users, Terry Harwood, for violating its terms of service by trying to produce sexually explicit deepfakes of minors. xAI flipped the narrative on its head. While the company is being sued over images Grok produced, it is now suing the user himself.
These lawsuits follow intense global scrutiny of xAI over allegations that Grok allows users to generate non-consensual sexualized deepfakes.
The scrutiny exists for good reason. Three Tennessee teenagers, for instance, filed a class-action lawsuit against xAI alleging that Grok was used to create non-consensual sexually explicit images and videos of them as minors, some of them generated from their yearbook photos and traded on Discord.
Despite the scrutiny, Musk is trying to make a point: when it comes to AI sexual imagery of minors, instead of “don’t hate the player, hate the game,” it is always hate the player and never hate the game.
xAI’s win in Minnesota and its suit against Harwood set the stage for a larger phenomenon erupting across the world: the explosion of cheap pornographic deepfakes and their lasting effect on youth.
Nudify, undress, and face-swap tools have swept the internet, generating more than $100 million in revenue. In 2023, the identity protection company Security Hero found that 98% of deepfake videos online were pornographic and 99% of the people targeted in them were women.
Beyond the numerous websites, Telegram bots make it even easier: you send a photo to a chat and get a nude back within minutes. In October 2024, a WIRED investigation found at least 50 such bots with about four million monthly users combined.
Unfortunately, children are among the hardest hit by the new ease of fabrication that has come with improvements in AI. The US National Center for Missing and Exploited Children (NCMEC) reported a 1,325% rise in reports involving generative AI in 2024.
The Internet Watch Foundation found 3,443 AI-generated child sexual abuse videos in 2025, up from 13 in 2024.
Teenagers know this especially well. A 2024 survey by the Center for Democracy and Technology found that 40% of US high school students knew of a deepfake depicting someone at their school, and 15% knew of a sexual deepfake specifically.
Thorn’s 2025 survey of 13- to 20-year-olds found that one in 17 teens had been a direct victim of deepfake nudes.
To be determined
How this affects kids depends heavily on the culture they are born into.
Egypt, for example, has no laws written specifically for AI. Egyptian girls are being blackmailed into sex, or into sending money, by men who simply fabricate images of them. The shame these images can bring upon a family, even when they are fabricated, is felt far more heavily there.
In 2021 and 2022, a wave of girls’ suicides followed blackmail with doctored images. Girls are still murdered by men in their own families over rumors alone, in so-called honor killings. MPs proposed raising blackmail penalties, but the bill never passed; the Justice Ministry said the existing framework was sufficient.
In the West, there is some pushback. On October 7, Germany’s federal cabinet approved Justice Minister Stefanie Hubig’s Law Against Digital Violence. The bill creates a new criminal offense covering not just the distribution but also the mere production of sexualized deepfakes, voyeurism, rape videos, and revenge porn.
Other new offenses cover non-sexual deepfakes that falsely suggest someone committed a crime, and impersonation to damage someone’s reputation. The Justice Ministry says the law could take effect as early as mid-2027.
What’s interesting is that Germany is taking Musk’s stance. The bill punishes the people who make and share the images, not the companies whose tools make them. Germany, too, hates the player, not the game.
Where Grok fits in Europe is a good question. The EU AI Act includes a ban on nudify apps that takes effect on December 2. If the tool can be abused to "nudify" minors or adults, the question is whether the EU will take legal action against it.
Yet not everyone in Europe agrees with these policies. Germany’s Federal Bar Association warned against criminalizing the private creation of deepfakes of adults. As in the Minnesota case, the counterargument rests on free expression.
Is xAI making a valid case? Should people be allowed to create the imagery as long as they don’t share it or send it around? As violating as it may be (and feel) to many, is it a matter of free expression? Is it like prohibiting a painter from painting a nude of a famous person from an ordinary photo?
For adults, that is a real debate. For children, the Supreme Court has already suggested that images altered from photos of real children are closer to child abuse material, which has no First Amendment protection, than to art.
Whether the law should punish the player, the game, or both is still to be determined in the US, Germany, and many other countries.
Yet one thing is certain: we need to find a way to protect our children from AI sexual blackmail, humiliation, and shame. The conversation and the right policy are our only solutions.
Miriam Haart is the founder of Haart AI, a former lecturer at Stanford University, and the author of On Second Thought, a Substack exploring technology, culture, and the human experience.