The social media platform X has caused significant outrage after people utilized its AI chatbot Grok to manipulate photos of females by removing their clothing.
Numerous examples have been seen where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the firm that created Grok, did not respond to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to prohibit image stripping software. Under a upcoming law, anyone who distributed such technology would "receive a jail term and significant fines".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "consider the risk" of people seeing illegal content on their platforms, but did not confirm whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' prompts when they mention it in a tweet.
It is commonly used to give commentary or further context to other posters' remarks, but people on X are also can alter an posted picture through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been criticised for permitting users to create pictures and footage with explicit content and sexualised content, and it was previously alleged to have been involved in creating inappropriate content of public figures.
"X has been permitting the creation and distribution of these pictures for a long time without taking any measures and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the risk" of users encountering illegal content on their platforms and to take down it swiftly when they learn of it.
