Research from 1Password’s Off-by-1 Labs found that frontier LLMs frequently produced unsafe or incomplete fixes when asked to patch newly disclosed software vulnerabilities. In a study of 6,080 AI-generated patches for six recent open-source CVEs, only 26.0% fully resolved the issue without materially changing application behavior; 20.1% fixed the flaw but altered behavior, while 53.9% either failed to remediate the bug, introduced a new vulnerability, or both. The researchers said many outputs looked plausible and even passed tests, yet still left exploitable paths open or created fresh defects; in one freenginx case study, 114 patches judged to close the original flaw all introduced a new problem. 1Password said expert validation remains necessary and released tooling, datasets, and a paper to support large-scale patch evaluation.
The findings coincided with a tougher stance from Linux wireless maintainer Johannes Berg, who said AI/LLM-generated patches for that subsystem will be ignored unless a brief review shows they are obviously correct. Berg argued that such fixes often make narrow local changes without adequately accounting for code semantics, architecture, error ordering, or long-term maintainability. Although it was later clarified that syzbot AI-assisted patches are pre-reviewed by humans rather than automatically submitted, AI-assisted patch submissions from syzbot are being disabled for the Linux wireless codebase, underscoring growing concern that machine-generated fixes can add risk instead of removing it.

Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
7 events from the most recent confirmed update back to the earliest known activity.
Off-by-1 Labs published FLAWED, a research harness for evaluating whether AI agents can patch known vulnerabilities without seeing the upstream fix and whether their patches introduce new flaws. The project also published pre-built campaign datasets for multiple publicly disclosed, upstream-fixed vulnerabilities.
The freenginx maintainers released a fix for the second crash condition that Off-by-1 Labs had reported. This addressed the denial-of-service issue triggered by a client starting a request and then going quiet.
Off-by-1 Labs discovered that both the rejected Patch the Planet fix and the maintainers’ own patch introduced a new denial-of-service condition in freenginx. The researchers reported this second crash to the maintainers on June 29.
1Password’s Off-by-1 Labs published research showing that only 26.0% of 6,080 AI-generated patches for six recently disclosed vulnerabilities fully fixed the issue without materially changing behavior. The researchers concluded that expert human review remains necessary and released tooling, datasets, and a paper to support patch-quality evaluation.
Following discussion of the new policy, it was clarified that syzbot's AI-generated patches are human pre-reviewed and not auto-replied or auto-resubmitted. Despite that, syzbot is being disabled from sending AI-assisted patches to the Linux wireless subsystem.
Johannes Berg announced that the Linux wireless subsystem will ignore most AI/LLM-generated patches unless they appear obviously correct after a very brief review. He said maintainers should not have to perform deep semantic and architectural review for low-value AI-generated submissions.
Trail of Bits sent freenginx maintainers a patch for the original use-after-free flaw through the Patch the Planet initiative run with OpenAI. The maintainers rejected that patch and wrote their own fix instead.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
8 references tracked. Mallory keeps watching after this page renders.
cyberscoop.com
Open sourcedarkreading.com
Open sourcezdnet.fr
Open sourcetheregister.com
Open source1password.com
Open sourcehelpnetsecurity.com
Open sourcephoronix.com
Open sourcegithub.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.