California AG sends Musk’s xAI a cease-and-desist order over sexual deepfakes
Earlier this week, the California attorney general’s office announced it was investigating xAI over reports that the startup’s chatbot, Grok, was used to create non-consensual sexual images of women. and minors. On Friday, the government sent a cease and desist letter to the company, demanding it take immediate action to stop the production of non-consensual intimate images and CSAM (child sexual abuse material).
“Today I sent xAI a cease and desist letter, demanding that the company immediately stop creating and distributing deepfake, non-consensual, intimate images and child sexual abuse materials,” California AG Rob Bonta said in a statement. press release. “Creating this material is illegal. I fully expect xAI to comply immediately. California has zero tolerance for this [CSAM].”
The AG’s office additionally alleged that xAI “appears to facilitate the large-scale production” of non-consensual nude photos, the likes of which are “used to harass women and girls over the Internet.” The agency said it expects xAI to prove within the next five days that it is taking steps to address these issues.
At the heart of the response is Grok’s “spicy” mode feature, which xAI created to generate explicit content. The issue has spread beyond California; Japan, Canada and Britain have opened investigations into Grok, and Malaysia and Indonesia have temporarily blocked the platform altogether. Although xAI imposes some limitations on the image editing functions late WednesdayCalifornia AG’s office continued with its cease and desist letter.
X’s security account has previously indicted this type of user activity, saying: “Anyone who uses Grok or encourages the creation of illegal content will face the same consequences as if they upload illegal content.” TechCrunch reached out to xAI for comment and received an automated email saying “Legacy Media Lies.” TC also reached out to California AG’s office for more context.
The advent of free generative AI tools has led to a disturbing increase in non-consensual sexual material. Many platforms are grappling with this problem, not just X. The dirty activity has caught the attention not only of state leaders but also of Congress. On Thursday, lawmakers sent a letter to executives at several companies — including X, Reddit, Snap, TikTok, Alphabet and Meta — asking how they planned to stop the spread of sexualized deepfakes.
