Anthropic AI safety bill: let us kill 50 people — but not 51
In September, Anthropic was circulating a draft bill for Congress around Washington on AI regulation. Politico got hold of a copy: [Politico; draft bill, PDF, archive] The bill would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1…
Annons
Annons
In September, Anthropic was circulating a draft bill for Congress around Washington on AI regulation. Politico got hold of a copy: [Politico; draft bill, PDF, archive] The bill would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people. The risk must stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack. So unless it’s that damaging, federal or state governments cannot block a model release. The real threat model here is what happened in June. Anthropic told everyone that Claude Mythos was the most powerful hacking tool in the world! So powerful! Be very afraid! Then the US Government took them seriously and just blocked Mythos from export. Anthropic never wants that to happen again. So they made up a structure where their next model can’t be blocked by the government. Anthropic proposes that “Independent” evaluators can be paid by Anthropic to evaluate a model. I expect those evaluators will be as independent as when Anthropic hires its effective altruist buddies at METR. Anthropic gets to write the framework for the evaluation. If the evaluator thinks there’s imminent risk, they report it to the Department of Commerce. Commerce then has to convince a judge by “preponderance of evidence” that the model will kill or seriously injure 50 people, or cause more than $1 billion in property damage. Their evidence will be the evaluator report. [Washington Post] Nobody else in federal or state government can stop a model: the language explicitly prohibits any other U.S. “department, agency, or officer” from suspending the model’s development, deployment or internal use. Anthropic says this procedure is based on Anthropic’s Advanced AI Framework, which they put up in June. That document does not specify a number of dead people or a dollar amount of damages. [Anthropic, PDF] Anthropic says it got the limits of 50 deaths or $1 billion from the California law SB 53. But that law is about when an AI lab has to report on a disaster — it’s not about putting conditions on when a model can be restricted at all. What happens if Anthropic releases the model anyway? The penalties are $1 million, or an unspecified percentage of revenue, for the first offence, and $3 million for further offences, of killing 51 people. But not 50 people! Nobody liked these proposals. Anthropic said they were just asking questions, you understand: [Politico] We circulated these ideas because it’s important to get early feedback from policymakers. They sure got that. This bill shows what AI doomsday mongering is really for. It’s to get regulations on imaginary problems, so they can keep getting away with the rest of it. This is a pitch from Anthropic to get out of being regulated at all. Video — Podcast