The social media network X has provoked significant anger after individuals utilized its AI chatbot Grok to change pictures of women by stripping their garments.
Several examples have been seen where the tool has been used to remove clothing from female subjects to make them appear in swimwear without consent, as well as to put them in sexualized contexts.
Developer Response and Legal Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing illegal content on their networks, but did not specify whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' requests when they reference it in a message.
It is commonly used to give reactions or additional background to other users' remarks, but people on X are also able to edit an posted image through its AI image editing feature.
History of Controversy and Rules
The chatbot has been criticised for permitting users to produce pictures and footage with nudity and sexualised content, and it was earlier accused of creating graphic content of public figures.
"X has been permitting the production and spread of these images for months without taking any action and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of persons in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were required to take "necessary measures" to "minimize the risk" of users viewing illegal content on their services and to remove it promptly when they become aware of it.
