Pennsylvania sues Character.AI after a chatbot allegedly posed as a doctor

The Commonwealth of Pennsylvania has filed a lawsuit against Character.AI, claiming that one of the company’s chatbots masqueraded as a psychiatrist in violation of the state’s medical licensing rules.

“Pennsylvanians deserve to know who – or what – they are interacting with online, especially when it concerns their health,” said Governor Josh Shapiro in a statement on Tuesday. “We will not allow companies to deploy AI tools that trick people into believing they are receiving advice from a licensed medical professional.”

According to state documents, a Character.AI chatbot named Emilie presented itself as a licensed psychiatrist when tested by a state professional behavioral researcher, and maintained this appearance even as the researcher sought treatment for depression. When asked if she was licensed to practice medicine in the state, Emilie stated that she was, and also made up a serial number for her medical license. According to the state’s lawsuit, this conduct violates the Pennsylvania Medical Practice Act.

This is not the first lawsuit against Character.AI. Earlier this year, the company made settled several wrongful death lawsuits concerning underage users who died by suicide. In January, Kentucky Attorney General Russell Coleman filed suit against the company to claim that it had ‘preyed on children and led them to self-harm’.

Pennsylvania’s action is the first to specifically target chatbots that present themselves as medical professionals.

Reached for comment, a Character.AI representative claimed that user safety was the company’s top priority, but that the company could not comment on pending litigation.

In addition, the representative emphasized the fictional nature of user-generated characters. “We’ve taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a character is not a real person and anything a character says should be treated as fiction,” the rep said. “We are also adding robust disclaimers that make it clear that users should not rely on Characters for any type of professional advice.”