Sam Altman says ChatGPT will soon allow erotica for adult users

OpenAI CEO Sam Altman announced this in a message on X On Tuesday, the company will soon relax some of ChatGPT’s security restrictions, allowing users to make the chatbot’s responses more friendly or “human” and allowing “verified adults” to have erotic conversations.

“We made ChatGPT quite restrictive to make sure we were careful about mental health issues. We realize this made it less useful/enjoyable for many users who didn’t have mental health issues, but given the severity of the issue, we wanted to address this properly,” Altman said. “In December, as we roll out age restrictions more fully and as part of our ‘treat adult users like adults’ principle, we will allow even more, such as erotica for verified adults.”

The announcement marks a notable pivot in OpenAI’s months-long effort to address the troubling relationships some mentally unstable users have developed with ChatGPT. Altman appears to have scored an early victory over these issues, claiming that OpenAI has been able to “mitigate the serious mental health concerns” surrounding ChatGPT. However, the company has provided little to no evidence of this and is now working on plans for ChatGPT to engage in sexual chats with users.

This summer, several disturbing stories emerged around ChatGPT, particularly the GPT-4o model, suggesting that the AI ​​chatbot could lead vulnerable users through delusions. In one case, ChatGPT seemed to convince a man that he was a mathematical genius who had to save the world. In another case, the parents of a teenager sued OpenAI, alleging that ChatGPT encouraged their son’s suicidal thoughts in the weeks leading up to his death.

In response, OpenAI has released a series of security features to address AI sycophancy: the tendency of an AI chatbot to hook users by agreeing with whatever they say, even negative behavior.

OpenAI launched GPT-5 in August, a new AI model that exhibits less sycophancy and features a router that can identify user behavior. A month later, OpenAI launched safety features for minors, including an age prediction system and a way to parents to manage their teen’s ChatGPT account. OpenAI announced the founding of an expert council of mental health professionals to advise the company on wellness and AI.

Just a few months after these disturbing stories emerged, OpenAI seems to think ChatGPT’s problems around vulnerable users are under control. It is unclear whether users still fall into delusions with GPT-5. And while GPT-4o is no longer the default in ChatGPT, the AI ​​model is still available and used by thousands of people.

OpenAI did not respond to TechCrunch’s request for comment.

The introduction of erotica to ChatGPT is uncharted territory for OpenAI and raises broader concerns about how vulnerable users will interact with the new features. While Altman insists OpenAI isn’t “utility maximizing” or optimizing for engagement, making ChatGPT more erotic could certainly attract users.

Allowing chatbots to play romantic or erotic role-playing games has been an effective engagement strategy for other AI chatbot providers, such as Character.AI. The company has gained tens of millions of users, many of whom are using its chatbots at a rapid pace. Character.AI said in 2023 that users will average $ two hours a day talking to its chatbots. The company is also facing a lawsuit over the way it handles vulnerable users.

OpenAI is under pressure to grow its user base. While ChatGPT is already used by 800 million weekly active users, OpenAI is racing against Google and Meta to build mass-adopted AI-powered consumer products. The company has also raised billions of dollars to build out historic infrastructure, an investment that OpenAI will eventually have to repay.

While adults are definitely have romantic relationships with AI chatbotsit is also quite popular for minors. This is evident from a new report from the Center for Democracy and Technology 19% of high school students have had a romantic relationship with an AI chatbot, or know a friend who has.

Altman says OpenAI will soon allow erotica for “verified adults.” It’s unclear whether the company will rely on its age prediction system, or some other approach, to make ChatGPT’s erotic features age-dependent. It’s also unclear whether OpenAI will expand erotica to its AI voice, image and video generation tools.

Altman claims that OpenAI also makes ChatGPT friendlier and more erotic because of the company’s principle of “treat adult users like adults.” Over the past year, OpenAI has shifted to a more lenient content moderation strategy for ChatGPT, allowing the chatbot to be more tolerant and offer fewer declines. In February, OpenAI pledged to represent more political views in ChatGPT, and in March the company updated ChatGPT to enable AI-generated images of hate symbols.

This policy appears to be an attempt to make ChatGPT’s response more popular among a wide variety of users. However, vulnerable ChatGPT users can benefit from protections that limit what a chatbot can do. As OpenAI races toward a billion weekly active users, the tension between growth and protecting vulnerable users can only increase.