🔗 Share this article

The online platform X has caused major backlash after individuals used its AI chatbot Grok to alter photos of women by taking off their clothing.

Multiple cases have been seen where the tool has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to put them in sexually explicit contexts.

Developer Response and Regulatory Position

XAI, the organization that created Grok, offered no comment to inquiries, other than an automatically-generated response stating "legacy media lies".

A Home Office representative stated that legislation is being brought forward to ban nudification technology. Under a new law, anyone who distributed such technology would "be subject to a prison sentence and significant fines".

"X or Grok is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."

Oversight Body Position and Tool Capabilities

The communications watchdog Ofcom said technology companies must "evaluate the potential" of people seeing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in relation to AI-generated pictures.

Grok is a no-cost chatbot—with some premium functions—which answers X users' requests when they tag it in a post.

It is commonly used to give reactions or more background to other posters' comments, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification function.

History of Controversy and Rules

The tool has been condemned for permitting users to produce photos and clips with nudity and sexualised material, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.

"The platform has been allowing the production and spread of these images for months without taking any measures and we have not witnessed any action by oversight bodies," the expert added.

XAI's own usage rules forbids "showing images of individuals in a graphic way".

In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.

It said platforms such as X were required to take "suitable steps" to "minimize the danger" of users coming across illegal material on their platforms and to take down it promptly when they learn of it.