[ad_1]
The UK communications regulator Ofcom launched a proper investigation into Elon Musk’s social media platform X concerning its AI chatbot, Grok following experiences that Grok has been used to generate nonconsensual sexual deepfakes.
Leon Neal/Getty Pictures
cover caption
toggle caption
Leon Neal/Getty Pictures
Indonesia and Malaysia quickly blocked X’s chatbot, Grok, over the weekend after it made scores of faux photos publicly sexualizing principally ladies and, in some situations, youngsters late final 12 months.
Governments world wide are additionally launching investigations. The most recent got here on Monday because the UK media regulator, Ofcom, launched a probe into the social media platform, which may lead to a ban.
Grok had been producing sexually specific photos of individuals for a while. However the concern obtained widespread consideration in late December as individuals used the chatbot to edit a excessive quantity of current photos by tagging the bot in feedback and giving it prompts reminiscent of “put her in a bikini.” Whereas Grok didn’t reply to the entire requests, it obliged in lots of circumstances. In some circumstances, Bellingcat senior investigator and researcher Kolina Koltai famous, customers can get Grok to generate frontal nudes.
Untold numbers of girls and in some circumstances, youngsters, as Reuters first reported, have had their likenesses sexualized on-line by Grok with out their permission, together with one of many moms of X proprietor Elon Musk’s youngsters.
It is uncommon for therefore many governments to take motion in opposition to a social media firm however this case is totally different, stated Riana Pfefferkorn, a coverage fellow at Stanford College. “Making baby sexual abuse [material] is flagrantly unlawful, just about in every single place on Earth.”
By final Friday, X had restricted Grok’s AI picture technology characteristic to make it solely accessible to paying subscribers. Non-paying customers can nonetheless put individuals in bikinis publicly with just some clicks, however they’ll solely put in a couple of such requests earlier than being prompted to join a premium membership, which prices $8 a month.
NPR reviewed Grok’s publicly accessible photos generated earlier this month and located it had stopped making photos of scantily clad ladies a number of days into 2026. Nevertheless, it typically nonetheless affords up bikini-clad males.
xAI, the mother or father firm of X, has been pushing grownup content material with Grok since final 12 months. In Might, Koltai first famous that the chatbot would generate sexually specific photos in response to requests on X like “take off her garments.” This previous summer season, Grok launched “spicy mode” in its standalone app, which allowed customers to place bikinis on AI-generated characters.
Ben Winters, director of AI and privateness on the advocacy group Client Federation of America, stated that Grok not solely permits modifying photos of actual individuals, but it surely additionally offers a simple distribution platform through X. “It is a additional and important escalation,” he stated.
Governments are outraged over X’s transfer to limit entry to the picture technology perform to subscribers. British Expertise Secretary Liz Kendall advised Sky Information “it’s insulting to say that you would be able to nonetheless entry this service when you pay for it.”
The Indonesian authorities discovered that Grok lacked efficient guardrails to cease customers from making nonconsensual pornographic content material based mostly on actual Indonesian residents, the Related Press first reported.
“The federal government sees nonconsensual sexual deepfakes as a severe violation of human rights, dignity and the protection of residents within the digital house,” Indonesian Communication and Digital Affairs Minister Meutya Hafid stated in an announcement.
The AP additionally reported Grok will keep blocked in Malaysia till efficient guardrails in opposition to misuse are put in place.
In response to NPR’s questions, X spokesperson Victoria Gillespie pointed to a assertion posted on January 3 that stated “anybody utilizing or prompting Grok to make unlawful content material will undergo the identical penalties as in the event that they add unlawful content material.” The assertion echoed Musk’s personal submit earlier that day.
Such an method is an “try and abdicate duty, ” stated Winters.
“It actually is not only the consumer that’s prompting it alone,” he stated. “It’s the truth that the picture wouldn’t be created if not for … the software they made.”
Earlier than the wave of Grok-generated nonconsensual specific deepfakes hit X, different AI makers had added related capabilities to their chatbots. In November, Google launched a brand new picture producing mannequin, Nano Banana Professional. In December, OpenAI up to date its mannequin ChatGPT Pictures. Each may also edit photos to place individuals in bikinis, Wired reported final month. A thread on Reddit distributing such photos has been taken down.
In one other X submit, Musk steered that since instruments from different AI firms even have this undressing perform, the strain from governments is a type of censorship directed at his platform.
Koltai famous the development to make AI-generated nonconsensual intimate media has been within the making for the previous 12 months or two. “You may see these adverts in Instagram even – it is like, add a photograph of you and your crush and also you guys can kiss,” she stated. “In order that’s kissing and hugging, and you then see there’s that extra excessive spectrum.”
“There’s clearly an enormous concern throughout the tech trade as a result of we have seen this throughout a number of platforms, and [there is] not at all times nice tips or boundaries or regulation,” stated Koltai.
The criticism within the U.S. has been much more muted than in different nations. Sen. Ted Cruz, R-Tex., posted on X Wednesday that the pictures “ought to be taken down and guardrails ought to be put in place.” He additionally stated that he’s “inspired that X has introduced that they are taking these violations critically and dealing to take away any illegal photos and offending customers from their platform.”
Grok has generated frequent controversy prior to now 12 months. Final summer season, the chatbot referred to itself as “MechaHitler” and spewed antisemitic conspiracy theories.
Apps with so-called nudifying capabilities have existed for years, principally within the shadows of the web. Winters stated officers have to do extra to police a few of X’s options. “There are misrepresentations in regards to the security of their merchandise. There are violations of their phrases of service,” he stated.
“We’ve not seen actually any important motion from any U.S. businesses, whether or not it is state or federal, which have the authority to implement the legislation,” Winters stated.
[ad_2]
