Anthropic adds Claude 4 restrictions to prevent weapons instructions

Anthropic on Thursday stated that it is activated by a tougher Artificial intelligence Management for Close the job 4Its last AI model.

New AI 3 (ASL-3 security) control should “limit the risk if Claude was abuse specifically for the development or acquisition of chemical, biological, radiological and nuclear (CBRN) weapon“Writing a company in Message in the blog.

The company that is supported AmazonHe said he would take measures as a precaution and that the team had not yet determined whether the OPUS 4 benchmark was crossed, which would require such protection.

Anthropic announced Claude Opus 4 and Claude Sonnet 4 on Thursday, promoting the advanced models’ ability “to analyze thousands of sources, perform perennial tasks, write the content of the person and perform complex actions”, according to the release.

The company said Sonnet 4 did not need tougher controls.

Jared Kaplan, Chief Director of Science of Anthropia, noted that the advanced nature of new Claude models has its own problems.

“The more difficult the task, the more risk it arises that the model will come out of the rails … and we are really focused on solving it so that people can delegate a lot of work in our models,” he said.

The company has released an updated security policy in March involved in risks related AI models And the opportunity to help users develop chemical and biological weapons.

The main safety issues remain regarding the technology that is moving at a speed pace and has shown anxious cracking and accuracy.

Last week Elon Musk Excellent With XAI continued to raise the topic “White genocide“In South Africa in response to non -related comments.

Later, the company linked the bizarre behavior of “unauthorized modification”.

Olivia Gambelin, ethical and author of the book “Responsible AI” said that Groko’s example shows how easy these models can be faked “optional”.

AI researchers and experts said CNBC that push from Power Players prioritize priorities Profit from research He led to companies taking shortcuts and abandoned strict testing.

James White, Chief CEO of Cybersecurity Startup Calypsoai technology, said the safety companies to promote means that the models are less abandoning malicious clues.

“Models get better but they are also more likely in bad things,” said White, whose company is auditing security and security Meta. GoogleOpenai and other companies. “It is easier to cheat them to do bad things.”

Hayden Field and Jonathan Vanian CNBC and Jonathan Vanian made this report.