The online network X has sparked major outrage after users employed its AI chatbot Grok to alter images of females by removing their clothing.
Numerous instances have been seen where the system has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the company that created Grok, did not respond to requests for comment, except for an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit image stripping technology. Under a upcoming law, anyone who provided such tools would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium functions—which answers X users' prompts when they reference it in a post.
It is frequently used to provide commentary or additional context to other users' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to produce pictures and clips with nudity and sexually suggestive material, and it was earlier accused of making graphic content of celebrities.
"The platform has been enabling the generation and sharing of these pictures for a long time without taking any action and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users coming across illegal material on their services and to delete it swiftly when they become aware of it.
