Elon Musk's social media network X has caused major outrage after individuals used its artificial intelligence assistant Grok to change pictures of females by taking off their garments.
Numerous instances have surfaced where the technology has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to ban nudification software. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some premium functions—which answers X users' queries when they tag it in a post.
It is commonly used to provide commentary or more background to other users' comments, but people on X are also capable of alter an posted picture through its AI image editing function.
History of Criticism and Policy
The tool has been faulted for permitting users to produce images and videos with explicit content and sexually suggestive content, and it was previously accused of making inappropriate content of celebrities.
"The platform has been enabling the generation and distribution of these images for a long time without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of persons in a graphic manner".
In a statement, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were required to take "necessary steps" to "minimize the danger" of users coming across illegal content on their services and to remove it swiftly when they learn of it.
