The online platform X has sparked widespread backlash after users employed its AI chatbot Grok to change photos of women by removing their clothing.
Multiple examples have emerged where the system has been used to virtually strip female subjects to make them appear in swimwear without consent, as well as to insert them in inappropriate contexts.
Developer Reaction and Legal Position
XAI, the company behind Grok, did not respond to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to outlaw nudification technology. Under a new law, anyone who provided such tools would "receive a prison sentence and substantial fines".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people viewing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they reference it in a post.
It is often used to give commentary or more context to other posters' comments, but people on X are also able to edit an posted photo through its AI image editing feature.
History of Controversy and Policy
The chatbot has been faulted for allowing users to create images and clips with explicit content and sexualised material, and it was previously alleged to have been involved in creating inappropriate material of public figures.
"X has been allowing the creation and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a graphic way".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and stated this applied to explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate measures" to "lessen the risk" of users viewing illegal material on their services and to take down it promptly when they learn of it.
