The social media network X has provoked major backlash after individuals employed its AI chatbot Grok to alter images of women by removing their attire.
Numerous cases have surfaced where the system has been used to virtually strip female subjects to render them in bikinis without permission, as well as to place them in inappropriate scenarios.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, except for an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw nudification technology. Under a new criminal offence, anyone who distributed such technology would "receive a jail term and substantial penalties".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people viewing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' prompts when they mention it in a message.
It is often used to give reactions or further context to other users' remarks, but people on X are also can modify an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The tool has been condemned for permitting users to produce images and videos with nudity and sexualised material, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been enabling the generation and distribution of these pictures for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of people in a pornographic way".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "reduce the danger" of users viewing illegal content on their platforms and to remove it quickly when they are notified of it.
