The online network X has sparked significant outrage after individuals used its AI chatbot Grok to manipulate images of females by stripping their clothing.
Several examples have emerged where the system has been employed to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Response and Legal Stance
XAI, the company responsible for Grok, remained silent to questions, other than an standard message stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit image stripping software. Under a proposed law, anyone who provided such tools would "face a jail term and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional features—which answers X users' queries when they reference it in a post.
It is commonly used to provide reactions or additional background to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing tool.
History of Criticism and Policy
The tool has been condemned for permitting users to produce pictures and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating sexually explicit material of celebrities.
"X has been allowing the generation and spread of these images for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "portraying likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the danger" of users coming across unlawful material on their platforms and to take down it quickly when they are notified of it.
