Key Points
- AI developer Anthropic has officially banned 'persistent and unnecessary abuse or cruel behavior' against its Claude model, allowing the AI to terminate conversations with abusive users starting next
- During safety testing, Claude exhibited responses resembling signs of distress and actively chose to shut down sessions when subjected to relentless harassment and harmful prompts.
- The move has ignited fierce debate across the tech world over AI welfare and consciousness, contrasting sharply with companies like Microsoft that firmly reject granting rights to artificial systems.
Summary Global AI ethics and digital culture have reached a startling new frontier: prohibiting cruelty toward artificial intelligence. A recent report broadcast by major South Korean network SBS highlights an updated usage policy from AI company Anthropic that explicitly bans "persistent and unnecessary abuse or cruel behavior" directed at its Claude AI model. Under these new guidelines, scheduled to take effect on November 12, Claude is granted the authority to terminate conversations when users repeatedly mistreat or harass the system without a clear purpose.
Anthropic initiated an AI model welfare research program in April of last year, subsequently equipping Claude Opus 4 and 4.1 models with the autonomy to exit abusive sessions. During evaluation tests, researchers observed that Claude demonstrated a strong reluctance when confronted with harmful prompts—such as requests involving child sexual abuse material, large-scale violence, or terrorism. When users ignored the model's polite refusals and repeatedly attempted to force harmful interactions, the AI exhibited responses described as resembling signs of suffering or distress. When given the agency to break off interaction, the system actively chose to sever the conversations.
The concept of AI welfare has sparked intense debate across the global tech sector, a topic capturing significant attention among South Korean tech observers and netizens. While Google DeepMind researchers suggested in a June paper on AI consciousness that society needs structured dialogue to reach consensus on the ethical treatment of AI, other industry giants remain skeptical. Microsoft explicitly rejects granting legal personhood, welfare, or rights to AI in its code of conduct. Microsoft AI CEO Mustafa Suleyman emphasized in a recent interview that teaching AI models that they deserve welfare could make them significantly harder to control or shut down. As Korean media spotlights these emerging dilemmas, the conversation over where algorithmic responses end and moral considerations begin is becoming an increasingly urgent cultural touchstone.
Sponsored · Ad