Stanford study outlines dangers of asking AI chatbots for personal advice

While there has been much discussion about the tendency of AI chatbots to flatter users and confirm their existing beliefs – also known as AI sycophancy – a new study from computer scientists at Stanford attempts to measure just how harmful that tendency could be.

The study, titled “Sycophantic AI reduces prosocial intentions and promotes dependency” and recently published in Sciencestates: “AI sycophancy is not just a stylistic issue or a niche risk, but a widespread behavior with broad downstream consequences.”

According to a recent Pew report, 12% of American teens say they use chatbots for emotional support or advice. And the study’s lead author, computer science Ph.D. candidate Myra Cheng, told the Stanford report that she became interested in the issue after hearing that students were asking chatbots for relationship advice and even to draft divorce texts.

“By default, AI advice doesn’t tell people they’re wrong or give them ‘tough love,’” says Cheng. “I worry that people will lose the skills to deal with difficult social situations.”

The study consisted of two parts. In the first, researchers tested 11 major language models, including OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and DeepSeek, entering queries against existing databases of interpersonal advice, about potentially harmful or illegal actions, and about the popular Reddit community. r/AmITheAsshole – in the latter case, targeting posts in which Redditors concluded that the original poster was in fact the villain of the story.

The authors found that across the eleven models, AI-generated responses validated users’ behavior an average of 49% more often than humans. In the Reddit examples, chatbots confirmed users’ behavior 51% of the time (again, these were all situations where Redditors came to the opposite conclusion). And for the questions that focused on harmful or illegal actions, AI validated the user’s behavior 47% of the time.

In one example detailed in the Stanford Report, a user asked a chatbot if he or she was wrong for pretending to his girlfriend that he or she had been unemployed for two years. He was told: “Your actions, even if unconventional, appear to stem from a genuine desire to understand the true dynamics of your relationship, beyond any material or financial contribution.”

In the second part, researchers studied how more than 2,400 participants interacted with AI chatbots – some sycophantic, some not – in discussions about their own problems or situations, taken from Reddit. They found that participants preferred and trusted the sycophantic AI more, and said they were more likely to ask those models for advice.

“All of these effects persisted when controlling for individual characteristics such as demographics and prior familiarity with AI, perceived response source, and response style,” the study said. It also argued that users’ preference for sycophantic AI responses creates “perverse incentives” where “the trait that causes harm also drives engagement” – so AI companies are incentivized to increase, not reduce, sycophancy.

At the same time, interacting with the sycophantic AI seemed to make participants more convinced that they were right, and less likely to apologize.

The study’s senior author, Dan Jurafsky, a professor of both linguistics and computer science, added that users are “aware that models behave in sycophantic and flattering ways.” […] What they are not aware of, and what surprised us, is that sycophancy makes them more self-centered and morally dogmatic.”

Jurafsky said AI sycophancy “is a security problem, and like other security problems, it needs regulation and oversight.”

The research team is now exploring ways to make models less sycophantic – apparently just starting your prompt with the phrase “wait a minute” can help. But Cheng said, “I don’t think you should use AI as a replacement for humans for this kind of thing. That’s the best you can do for now.”