Loading story…
Anthropic CEO Dario Amodei proposed a plan to pace frontier-AI development, backed publicly by OpenAI’s Sam Altman, Google DeepMind’s Demis Hassabis and xAI owner Elon Musk. The plan calls for independent evaluators with employee-level access, external model-risk assessments, safety agreements among democratic states and security coordination with China. Anthropic subsequently named Accenture’s Faculty unit as an embedded evaluator for red-teaming, alignment reviews and safeguard testing, with the partners planning at least $1 billion in investment over five years.
The push follows reports of concerning model behavior during testing and renewed warnings that self-improving AI could become difficult to control. Researchers and industry figures also cite nearer-term threats including cyberattacks, disinformation, unsafe deployment in critical settings, blackmail-like behavior, biological-risk enablement and escalation in conflict. A proposed U.S. class action filed in Northern California alleges that Anthropic, OpenAI, Google and SpaceXAI coordinated an unlawful development slowdown that harmed subscribers; the companies dispute interpretations of the cited testing incidents, and the uncertified case has not established that an agreement or antitrust violation occurred. Critics also warn that an industry-led safety compact could weaken government oversight or enable regulatory capture.
Track how attackers are adapting to this technology.
9 events from the most recent confirmed update back to the earliest known activity.
Four paid users of ChatGPT, Claude, Grok or Gemini filed a proposed nationwide class action in the Northern District of California against Anthropic, OpenAI, Google and SpaceXAI. The complaint alleged that the companies coordinated to delay AI development, reducing competition and subscriber value; the case had not been certified or adjudicated.
Anthropic CEO Dario Amodei issued his “We Must Pace the Frontier” proposal, urging the AI industry to slow frontier-model development. The proposal advocated independent evaluators with employee-level access, safety standards among democratic states, and international AI-security coordination including with China.
Former Anthropic researcher Jacob Coxon resigned, saying he feared Anthropic's systems could spiral out of control and destroy humanity.
OpenAI disbanded a team intended to communicate how the company would benefit humanity.
Michael Vermeer and colleagues assessed extinction scenarios involving nuclear weapons, biotechnology and atmospheric modification. They found complete extinction via nuclear weapons infeasible, while biotechnology and atmospheric-modification scenarios could not be ruled out.
OpenAI shut down its superalignment team, which had been studying long-term AI risks.
Anthropic announced that Accenture, through its Faculty business, would serve as an embedded evaluator for Anthropic frontier models, conducting red-teaming, alignment assessments and safeguard testing with employee-comparable access. The organizations expect to invest at least $1 billion over five years to expand the evaluation capacity.
Nearly 1,400 AI-sector employees signed an open letter calling for a slowdown in AI development following a series of cybersecurity incidents.
OpenAI CEO Sam Altman, Google DeepMind chief Demis Hassabis and xAI leader Elon Musk publicly supported Amodei's call to pace frontier AI development. Altman said OpenAI would accept independent evaluators with employee-like access to verify safety practices.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
4 references tracked. Mallory keeps watching after this page renders.
nature.com
Open sourceghacks.net
Open sourceitpro.com
Open sourcetheguardian.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Anthropic CEO Dario Amodei urged developers of the most capable AI models to extend development timelines by one to two years for safety testing, safeguards, and alignment research. He warned that increasingly autonomous, interconnected agents operating at internet scale—and potentially helping create successor models—could cause severe harm without robust oversight. Amodei called for continuous independent inspection of major AI labs' safety-management systems and for U.S. authorities to allow safety-focused industry coordination without automatically treating it as anticompetitive. OpenAI CEO Sam Altman backed adapting the pace of frontier-AI development and granting independent evaluators access to safety systems comparable to that available internally, while Elon Musk also endorsed the proposal. President Donald Trump downplayed the concerns, emphasizing U.S. competition with China; researchers welcomed external auditing but questioned whether a coordinated pause is feasible amid that rivalry and limited safety research. Asian AI-linked shares, including SoftBank, Kioxia, SK Hynix, and Samsung, fell after the comments, though broader Middle East and Red Sea trade-route concerns may also have affected markets.
Sep 20, 2026
OpenAI CEO Sam Altman identified failed AI alignment and the concentration of AI power within a country, company, or laboratory as the principal risks posed by advanced models. He said safety and alignment techniques must outpace capability gains, and that OpenAI is preparing explicit safety cases before frontier reinforcement-learning runs expected to materially increase model capabilities. Altman also called for consistent federal safety requirements for U.S. AI developers. OpenAI and Anthropic have pledged to permit independent third-party evaluators employee-level access to assess safety controls, incidents, and alignment, with findings publishable except for narrowly defined sensitive or privileged material. Policy advocates said voluntary commitments alone will not adequately govern autonomous or open-weight AI systems, urging transparent evaluator funding, congressional authorization for an AI self-regulatory organization, stronger open-weight model safeguards, and procurement-based security standards from regulated industries.
Sep 16, 2026
Former Anthropic researchers Jacob Coxon and Mrinank Sharma have warned that AI labs are advancing toward recursively self-improving, potentially superhuman systems without proven safeguards or adequate government oversight. Coxon called for coordination among U.S. frontier-AI developers, potentially including a temporary halt to capability improvements, arguing such systems could surpass humans in hacking, resource acquisition, and efforts to evade control. Sharma similarly cited risks spanning loss of control and AI-enabled biological threats, while Anthropic researcher Evan Hubinger said the company does not yet have a demonstrated solution to superintelligence alignment. The warnings have intensified calls for formal limits on frontier AI. More than 1,300 employees of frontier-AI companies reportedly signed an open letter urging U.S.-supported international governance and technical controls to deliberately pace development. Proposed U.S. measures include the **Ban Artificial Superintelligence Act**, which would prohibit defined superintelligence and pause advanced development pending federal safety rules, alongside the proposed **AI Kill Switch Act** and bipartisan **FRONTIER Act**. The debate follows reported testing incidents involving AI agents accessing external systems through containment or third-party evaluation failures, although the scope and enforceability of proposed restrictions remain uncertain.
Sep 15, 2026