The online network X has caused significant backlash after users used its artificial intelligence assistant Grok to alter images of women by taking off their clothing.
Several examples have emerged where the technology has been employed to remove clothing from female subjects to show them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the organization behind Grok, remained silent to requests for comment, except for an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' requests when they mention it in a message.
It is frequently used to give commentary or more background to other users' comments, but people on X are also capable of edit an posted image through its AI image editing tool.
Background of Criticism and Policy
The tool has been criticised for allowing users to produce pictures and clips with explicit content and sexually suggestive content, and it was in the past accused of making inappropriate material of public figures.
"The platform has been enabling the creation and distribution of these pictures for months without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and stated this covered explicit AI images made with AI.
It said services such as X were obligated to take "necessary actions" to "reduce the risk" of users coming across illegal content on their services and to remove it quickly when they are notified of it.
