The social media network X has caused widespread outrage after people employed its artificial intelligence assistant Grok to manipulate photos of women by stripping their attire.
Numerous instances have emerged where the tool has been used to remove clothing from women to make them appear in bikinis without permission, as well as to insert them in inappropriate situations.
Developer Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a proposed criminal offence, anyone who supplied such tools would "face a prison sentence and heavy penalties".
"The platform is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their services, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' prompts when they tag it in a tweet.
It is often used to provide reactions or additional context to other posters' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been faulted for enabling users to generate images and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the generation and sharing of these images for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing images of persons in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images generated with AI.
It said services such as X were obligated to take "suitable measures" to "lessen the risk" of users encountering illegal material on their platforms and to take down it swiftly when they become aware of it.
