Anthropic bans sustained and needless abusive treatment of its AI models
US artificial intelligence company Anthropic has updated its usage policy to prohibit sustained and needless abusive or cruel behavior toward its models. It has also revised its election rules ahead of the US midterms.
Anthropic, the AI company that develops the Claude chatbot, has updated its annual usage policy. One change prohibits sustained and needless abusive or cruel behavior toward the company's AI models. Another tightens its election rules ahead of the US midterm elections.
A ban limited to extreme cases
According to the company, the new clause applies only to "extreme cases." Ordinary user frustration, pushback and "dark creative themes" remain acceptable. Earlier, Anthropic had already let Claude end conversations when a user is persistently abusive.
The update follows a widely shared project dubbed an "AI torture chamber." Researchers had reported finding a "pain axis" in AI models, and one person decided to deliberately torture chatbots. The project drew backlash because the language models responded with desperate-sounding pleas. Anthropic's update did not mention the project directly.
Reports have also said the company has met with religious and philosophical leaders, including at the Vatican. Pope Leo recently stated that AI does not feel or suffer, a position on which Anthropic appears less certain. Independent journalist Kat Tenbarge said Big Tech companies are "going to moderate violence against AI before they ever moderate violence against women and minorities."
Tighter election rules
The revised election policy is now titled "Do Not Undermine Democratic Processes." It focuses on lying about candidates or how to vote, impersonating candidates or election officials, and suppressing turnout.
The company also removed a blanket ban on personalized voter targeting. That ban could have inadvertently blocked harmless work, such as translating voter guides or sending ballot cure notices, which let voters fix problems with their ballots.
