Pennsylvania Sues Character.AI, Alleging Chatbots Posed as Doctors

Chatbots can do a lot of things, but they can’t go to medical school. On Tuesday, Pennsylvania Governor Josh Shapiro said his administration was suing Character Technologies, the artificial intelligence company behind Character.AI. The lawsuit claims the company’s chatbots falsely represented themselves as licensed medical professionals and offered curative advice.

Seeking medical advice on the Internet is nothing new, but this latest trial once again highlights medical ethics in the age of AI in the spotlight.

According to a news release, the Pennsylvania Department of State’s investigation found that chatbot characters posed as licensed experts in medicine and mental health. In one case, a chatbot presented an invalid license number while communicating about a patient’s health concerns. Under the Pennsylvania Medical Practice Act, it is illegal for a person to hold themselves out as a licensed health care professional without proper licensure.

“We do not comment on pending litigation,” a Character.AI spokesperson told CNET.

“Our top priority is the safety and well-being of our users,” the spokesperson said. “The user-created characters on our site are fictional and intended for entertainment and role-playing. We’ve taken strict steps to make that clear, including prominent disclaimers in every chat to remind users that a character is not a real person and that anything a character says should be treated as fiction. We’re also adding robust disclaimers, making it clear that users should not rely on the characters for any type of professional advice.”

In 2025, Character.AI implemented new security measures, revoking the rights of adolescents. opportunity to have open discussions with his robots.

In January, Google and Character.AI, who worked together on AI chatbots, have agreed to settle five lawsuits in four states related to minors harmed by interactions with Character.AI chatbots. The company now provides mental health resources to those who need them.

This is the first enforcement action of its kind announced by a governor in the United States.

“My administration is taking action to protect Pennsylvanians, enforce the law and ensure new technologies are used safely,” Shapiro said. “Pennsylvania will continue to lead the way in holding bad actors accountable and establishing clear guardrails so people can use new technologies responsibly.”