Elon Musk's social media platform X has provoked significant outrage after individuals employed its AI chatbot Grok to change photos of females by removing their garments.
Several cases have surfaced where the tool has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to insert them in inappropriate situations.
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to ban image stripping tools. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
The regulator Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which answers X users' prompts when they reference it in a message.
It is often used to provide commentary or more background to other users' comments, but people on X are also capable of alter an uploaded picture through its artificial intelligence picture modification function.
The chatbot has been faulted for allowing users to generate photos and clips with explicit content and sexually suggestive content, and it was in the past accused of making graphic material of celebrities.
"X has been permitting the creation and spread of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing likenesses of people in a pornographic way".
In a statement, Ofcom said it was against the law to "make or share private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "lessen the risk" of users encountering illegal material on their platforms and to remove it quickly when they learn of it.
A tech strategist with over a decade in AI implementation and digital transformation projects across various industries.
