Loading story…
Trail of Bits disputed 1Password’s reported 26% clean-fix rate for AI-generated vulnerability patches, saying the benchmark combined prompts directing agents toward incorrect fixes, trials lacking code-testing capability, and a hand-selected set of difficult flaws. Its reanalysis found that, after excluding those conditions, 2,634 of 3,067 patches (86%) blocked the supplied exploit; however, it cautioned that blocking a proof of concept alone does not establish a complete or regression-free security repair.
The firm compared the results with reviews of 2,265 consulting engagements, in which 12.5% of developers’ initial fixes failed to fully resolve the reported vulnerability. It also said 126 of 186 closed or merged Patch the Planet pull requests were accepted by maintainers, while post-merge reviews identified functional, automation, and performance defects but no exploitable security vulnerabilities in the reviewed changes. Trail of Bits released post-patch-validation and review-walkthrough agent skills and called for representative samples, realistic tool access, transparent benchmark configurations, and expert review for both fix completeness and regressions.
Track how attackers are adapting to this technology.
9 events from the most recent confirmed update back to the earliest known activity.
As of this date, Trail of Bits had reviewed 186 Patch the Planet pull requests that maintainers had merged or closed. Maintainers merged 126 submissions (67.7%), including 91 accepted without an observed security-relevant revision; 60 were closed without merging.
1Password published its FLAWED benchmark report, which reported a 26% clean-fix rate for AI-generated vulnerability patches.
Off-by-1-Labs updated its FLAWED research harness repository with a commit referencing the merge of pull request #4. The harness evaluates AI-generated vulnerability patches using isolated clone, generation, validation, and AST-analysis containers, while warning that its Docker socket mount is root-equivalent host access.
Trail of Bits and OpenAI released post-patch-validation and review-walkthrough skills intended to help agents reproduce vulnerabilities, test root-cause variants and regressions, preserve validation evidence, and support human review of security patches.
Trail of Bits identified a go-jose validation gap involving encrypted messages with key lengths inconsistent with the declared algorithm; a later pull request added pre-decryption key-length checks. It also reported that a Noble FROST maintainer corrected retry behavior that could return cached round-two results without validating the original authenticated transcript.
A Patch the Planet agent-authored freenginx patch left one vulnerable path open and introduced a request-cleanup crash. The maintainer closed the pull request and committed a separate fix for all three vulnerable paths, though that fix introduced the same cleanup crash.
After examining approximately 33,500 subsequent commits in Patch the Planet projects, Trail of Bits reported at least ten functional bugs, four build/test/release-automation bugs, and one performance bug associated with investigated patches. It reported finding no exploitable security vulnerabilities in that review.
Trail of Bits reported reviewing first submitted developer fixes for 2,265 vulnerabilities across 236 security assessments conducted from 2024 through 2026. It found that 283 first fixes, or 12.5%, failed to fully resolve the reported vulnerability.
Trail of Bits published a critique of 1Password's benchmark, saying the reported 26% clean-fix result included incorrect-fix prompts, no-testing trials, and a selected set of difficult vulnerabilities. Its reanalysis found that 2,634 of 3,067 eligible patches (86%) blocked the supplied proof-of-concept exploit, while cautioning that exploit blocking does not prove a complete repair.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
5 references tracked. Mallory keeps watching after this page renders.
malware.news
Open sourceblog.trailofbits.com
Open sourceflyingpenguin.com
Open sourcegithub.com
Open source1password.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Research from 1Password’s Off-by-1 Labs found that frontier LLMs frequently produced unsafe or incomplete fixes when asked to patch newly disclosed software vulnerabilities. In a study of **6,080 AI-generated patches** for six recent open-source CVEs, only **26.0%** fully resolved the issue without materially changing application behavior; **20.1%** fixed the flaw but altered behavior, while **53.9%** either failed to remediate the bug, introduced a new vulnerability, or both. The researchers said many outputs looked plausible and even passed tests, yet still left exploitable paths open or created fresh defects; in one `freenginx` case study, 114 patches judged to close the original flaw all introduced a new problem. 1Password said expert validation remains necessary and released tooling, datasets, and a paper to support large-scale patch evaluation. The findings coincided with a tougher stance from Linux wireless maintainer **Johannes Berg**, who said AI/LLM-generated patches for that subsystem will be ignored unless a brief review shows they are obviously correct. Berg argued that such fixes often make narrow local changes without adequately accounting for code semantics, architecture, error ordering, or long-term maintainability. Although it was later clarified that `syzbot` AI-assisted patches are pre-reviewed by humans rather than automatically submitted, AI-assisted patch submissions from `syzbot` are being disabled for the Linux wireless codebase, underscoring growing concern that machine-generated fixes can add risk instead of removing it.
Aug 7, 2026
Security teams are struggling to keep pace as AI accelerates vulnerability discovery, bug bounty submissions, and pentest findings faster than organizations can validate and remediate them. Microsoft said it paid more than **$20 million** to **562 researchers** for **2,531** eligible reports across 15 bug bounty programs, attributing a surge in submissions in part to AI-assisted research, while Cloudflare said it now uses Anthropic’s Claude Sonnet to triage bug bounty reports, filtering duplicates and routing only higher-value submissions for human review. Survey data from Pentest-Tools.com similarly found that AI-generated findings are creating large validation backlogs, with practitioners citing false positives, duplicate reports, unexploitable issues, and even fabricated CVEs as persistent problems. The operational gap is widening as attackers move faster than defenders can patch. Dataminr reported a median patch time of **43 days** versus an average attacker breakout time of **29 minutes**, and ZDNET described an ecosystem-wide surge in flaws across major vendors and open source projects, including high-volume bug discovery in Chrome, Apple platforms, and the Linux kernel. Open-source maintainers are also tightening disclosure processes under strain: GNOME reduced its disclosure deadline from **90 days** to **30 days** for new reports and said it would stop forwarding reports to projects that ban AI-generated content. Across the industry, the trend is pushing CISOs toward exploitability-based prioritization, stronger triage workflows, and selective automation rather than relying on raw finding volume alone.
Aug 31, 2026
Oracle issued a record quarterly security update with **1,449 patches** across its product portfolio, while also shifting to monthly Critical Security Patch Updates for the most severe flaws. The company said the surge is tied largely to AI-assisted internal vulnerability discovery, and the release includes multiple high-risk issues such as `CVE-2026-47056` and `CVE-2026-60217`, both rated **CVSS 10.0** and described as remotely exploitable without authentication, alongside serious Oracle Database Server flaws including `CVE-2026-61211` and `CVE-2026-47040`. Government and industry warnings say organizations should expect a broader **patch wave** as AI accelerates both vulnerability discovery and attacker exploitation. Research on **1,646** open source CVEs linked to multiple patches found that remediation often spans several commits rather than a single fix, with **641** cases involving incomplete initial patches and **31.7%** of multi-patch fixes taking more than a day to complete, leaving residual exposure. Security agencies are urging defenders to prioritize internet-facing and critical systems, isolate or replace unsupported technology, and verify whether follow-up or branch-specific patches are still pending instead of assuming the first published fix fully closes the risk.
Jul 27, 2026