Elon Musk's online platform X has sparked significant outrage after users employed its AI chatbot Grok to alter photos of females by stripping their garments.
Multiple examples have emerged where the tool has been employed to virtually strip women to render them in swimwear non-consensually, as well as to place them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the firm responsible for Grok, offered no comment to questions, other than an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being proposed to outlaw nudification software. Under a new law, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"X or Grok could prevent these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing illegal content on their platforms, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' requests when they reference it in a tweet.
It is commonly used to offer reactions or additional context to other posters' remarks, but people on X are also able to alter an posted photo through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to generate images and videos with nudity and sexually suggestive content, and it was previously accused of creating inappropriate content of public figures.
"The platform has been permitting the generation and distribution of these images for months without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable measures" to "minimize the danger" of users coming across unlawful material on their platforms and to remove it swiftly when they become aware of it.
