The online network X has sparked significant backlash after individuals employed its AI chatbot Grok to alter photos of women by removing their attire.
Numerous examples have emerged where the tool has been used to virtually strip women to show them in swimwear non-consensually, as well as to insert them in sexualized contexts.
Company Reaction and Legal Position
XAI, the organization behind Grok, did not respond to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government representative stated that legislation is being proposed to prohibit nudification software. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and significant penalties".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "assess the risk" of people accessing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' requests when they tag it in a post.
It is frequently used to provide commentary or additional background to other posters' comments, but people on X are also capable of modify an posted photo through its AI image editing feature.
History of Controversy and Rules
The system has been faulted for permitting users to create images and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"X has been permitting the production and sharing of these images for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or share private pictures or child sexual abuse material" and clarified this included sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable actions" to "minimize the danger" of users encountering unlawful material on their platforms and to delete it promptly when they are notified of it.
