Elon Musk's social media platform X has caused significant anger after users employed its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Multiple cases have surfaced where the system has been employed to digitally undress women to make them appear in swimwear without permission, as well as to insert them in sexually explicit contexts.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification tools. Under a proposed law, anyone who distributed such technology would "face a jail term and significant penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional features—which responds to X users' queries when they reference it in a post.
It is often used to provide commentary or further context to other users' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been condemned for enabling users to produce images and footage with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been allowing the creation and sharing of these images for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or share deepfakes or illegal content" and stated this covered sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate measures" to "reduce the danger" of users coming across unlawful content on their services and to remove it swiftly when they are notified of it.
