How Amazon and the White House ended Anthropic’s Fable

Anthropic's much-anticipated, powerful Fable 5 AI model lasted just days in the public's hands, after an urgent report from Amazon triggered a scramble inside the White House that ended in a dramatic Friday night takedown. Why it matters: The episode highlights the administration and industry's reactionary approach to a technology that is moving at breakneck speed.It also raises questions about why Amazon would strike such a disruptive blow against a company in which it is a major investor. Behind the scenes: Amazon called administration officials Thursday night to share a report showing how they were able to jailbreak and access portions of Anthropic's powerful new Mythos model that pose a national security threat, sources familiar told Axios.Anthropic had previously notified the government multiple times about the planned June 9 release of Fable — which is a general-use version of Mythos —and the government did not object, a source close to the company said.But calls from Amazon — as well as at least five other companies to a variety of senior administration officials Thursday evening and Friday morning — led to the model being shut down by Friday night.For the record: "As a leading cloud provider that serves a large number of private and public sector customers, it's not uncommon for governments to seek our counsel on potential security risks. When they occur, we don't share the details of these discussions," an Amazon spokesperson said in a statement.Administration officials spoke with Anthropic for hours early Friday, trying to get the company to pull the latest model; those efforts were unsuccessful, an official said.On Friday around 5:20 p.m. ET, the White House sent Anthropic a letter, first reported by Axios, informing the company the Fable and Mythos models would be subject to sweeping export control rules. By about 10 p.m., users lost access to Fable. Under the controls, not only would Anthropic's most advanced models be inaccessible to foreign adversaries — something Anthropic was already preventing on its own — but any U.S. ally or foreign national in the U.S. The implications were immediately felt within the company, where many foreign-born workers need access to Anthropic's models.The Anthropic source says that CEO Dario Amodei and other Anthropic officials spoke with the administration after the company was told the government would impose the export rules. During the conversations, Anthropic officials laid out how the alleged Amazon jailbreak was relatively simple, could be achieved with other models, and did not demonstrate a flaw in Fable 5's safety systems.Between the lines: The government's response "seems way out of line with what's actually in the research report," Luta Security CEO Katie Moussouris, who Anthropic shared the Amazon report with, told Axios.Moussouris said the researchers were able to find security vulnerabilities by asking questions normal defenders would ask AI, which is exactly what the model was intended to do."All AI models need to be able to help defenders in exactly this way, or we won't be able to scale our defense against attackers," Moussouris said.The big picture: The export control letter targets just one company but could have broader implications across industry."This is a de-facto licensing regime," one person familiar told Axios. "Companies will not screw with the White House. That is the ultimate effect."An administration official told Axios they do not view other models as national security threats because they do not surpass the bar that Mythos set.Anything at Mythos level or above would need to go through the administration to ensure the government's national security apparatus is hardened enough, the official added. The bottom line: The source familiar with the government's thinking said there was a "lack of seriousness" that Anthropic was applying to the release of Fable. "Had Anthropic taken it seriously and, rather than dismissing as isolated, moved to fix or pause access, this would have never happened," the source said, adding "they were overly confident."