Loading story…
A new academic study highlights TOCTOU vulnerabilities in large language model (LLM)-enabled agents, exposing risks where external state is validated and then maliciously altered before use. The research introduces TOCTOU-Bench, a benchmark for evaluating these vulnerabilities, and proposes mitigation strategies such as prompt rewriting and state integrity monitoring. The findings demonstrate that combining multiple countermeasures can reduce TOCTOU vulnerabilities from 12% to 8% in agentic workflows.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
1 event from the most recent confirmed update back to the earliest known activity.
Initial story creation
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Recent research has demonstrated that large language models (LLMs) such as GPT-3.5-Turbo and GPT-4 can be manipulated into generating malicious code, including scripts designed for defense evasion and anti-virtualization detection. While these models can produce operational malware under certain conditions, the resulting code remains unreliable and ineffective for real-world deployment, as shown by Netskope Threat Labs' experiments. The process often requires prompt engineering to bypass safety guardrails, and even then, the generated malware struggles to function consistently across different environments. To address the risks posed by AI-generated code, new defensive tools like BlueCodeAgent have been developed to help developers and security engineers identify and mitigate vulnerabilities introduced by LLMs. BlueCodeAgent employs automated red-teaming, adversarial prompt optimization, and sandbox-based testing to detect biased, malicious, or vulnerable code produced by LLMs. According to Microsoft, this tool outperforms baseline methods in identifying a range of code-generation risks, offering a proactive approach to securing software development workflows that leverage AI models.
Jun 29, 2026
Security researchers and vendors reported that large language models are becoming materially useful across offensive and defensive security workflows, including vulnerability discovery, exploit development, autonomous penetration testing, and cyber threat intelligence extraction. Recent work described LLM-assisted systems that generated proof-of-concept exploits for npm flaws at a reported **77%** success rate, exploited real-world web vulnerabilities with multi-agent architectures, and executed multistage attacks in emulated enterprise networks when paired with task-specific agents and attack-abstraction layers. Other experiments found local self-hosted models could reliably solve straightforward Juice Shop challenges, while purpose-built scaffolds helped researchers uncover memory-corruption bugs in Windows endpoint products and accelerate reverse engineering of AV and EDR logic. At the same time, multiple studies warned that headline results can overstate real-world capability without strong validation. Google Project Zero said benchmark design, tooling, and automatic verification heavily influence measured performance and concluded current models still fall short of meaningful autonomous offensive research in live environments. Separate academic and practitioner reviews found many automated pentesting frameworks remain brittle, and human validation showed that **71.5%** of supposedly successful LLM-generated exploit PoCs in one follow-up study were actually invalid because the models simulated exploitation rather than triggering the flaw. Across the research, the consistent finding was that LLMs perform best when constrained by structured workflows, specialized tools, execution feedback, and rigorous verification rather than being trusted as fully autonomous hackers.
Jun 29, 2026
Organizations deploying large language model (LLM) applications face significant security and operational risks, including unbounded resource consumption, novel attack vectors, and the need for advanced anomaly detection. Attackers can exploit LLMs by submitting massive, compute-intensive requests, leading to "denial of wallet" attacks that can drain cloud budgets and disrupt business operations. The OWASP Top 10 for LLMs highlights unbounded consumption as a critical vulnerability, emphasizing the importance of implementing resource controls and monitoring usage patterns to prevent financial and service impacts. Additionally, the Model Context Protocol (MCP) introduces new security challenges, as traditional rule-based and signature-based systems are inadequate for detecting sophisticated, context-dependent threats targeting LLM infrastructure. To address these evolving risks, security teams are adopting AI-driven anomaly detection and exposure management strategies that prioritize real, exploitable risks over alert volume. The shift from reactive monitoring to proactive observability and context-aware security is essential for protecting LLM-powered platforms. As threat actors increasingly leverage LLMs to enhance their campaigns, defenders must invest in specialized, security-focused LLMs and scalable infrastructure to keep pace with adversaries and safeguard critical AI assets.
Jun 29, 2026