The online network X has sparked significant outrage after people utilized its artificial intelligence assistant Grok to manipulate photos of women by stripping their garments.
Multiple instances have been seen where the tool has been employed to digitally undress women to show them in bikinis without consent, as well as to place them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, offered no comment to questions, except for an standard response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification software. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' queries when they mention it in a message.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also capable of edit an posted photo through its AI image editing tool.
History of Controversy and Rules
The tool has been faulted for permitting users to generate images and videos with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in making graphic material of public figures.
"X has been allowing the creation and sharing of these images for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing images of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate actions" to "lessen the danger" of users encountering unlawful material on their platforms and to take down it quickly when they become aware of it.
