Loading story…
Recent research has demonstrated that large language models (LLMs) such as GPT-3.5-Turbo and GPT-4 can be manipulated into generating malicious code, including scripts designed for defense evasion and anti-virtualization detection. While these models can produce operational malware under certain conditions, the resulting code remains unreliable and ineffective for real-world deployment, as shown by Netskope Threat Labs' experiments. The process often requires prompt engineering to bypass safety guardrails, and even then, the generated malware struggles to function consistently across different environments.
To address the risks posed by AI-generated code, new defensive tools like BlueCodeAgent have been developed to help developers and security engineers identify and mitigate vulnerabilities introduced by LLMs. BlueCodeAgent employs automated red-teaming, adversarial prompt optimization, and sandbox-based testing to detect biased, malicious, or vulnerable code produced by LLMs. According to Microsoft, this tool outperforms baseline methods in identifying a range of code-generation risks, offering a proactive approach to securing software development workflows that leverage AI models.
Track how attackers are adapting to this technology.
2 events from the most recent confirmed update back to the earliest known activity.
A report published on November 20, 2025 said malware produced with large language models is getting better, while noting fully autonomous AI-driven attacks are not expected imminently. This reflects a new assessment of the threat landscape rather than a specific attack incident.
BlueCodeAgent was reported as offering a tool to help developers identify and address security issues in AI-generated code. The reference indicates the product was publicly discussed or announced by November 20, 2025.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Large language models (LLMs) present a dual-use dilemma in cybersecurity, as their capabilities can be leveraged for both defensive and offensive purposes. Security researchers have identified purpose-built malicious LLMs, such as WormGPT and KawaiiGPT, which are designed to facilitate cybercrime by generating convincing phishing content and rapidly producing or modifying malicious code. The thin line between beneficial and harmful use of LLMs is defined largely by developer intent and the presence or absence of ethical safeguards, raising concerns about the proliferation of offensive AI tools in the threat landscape. In addition to malicious use, LLMs face significant challenges in maintaining privacy and security due to contextual integrity failures and regulatory-driven censorship. Research from Microsoft highlights the need for AI agents to respect contextual privacy norms, as current models may inadvertently leak sensitive information. Meanwhile, the DeepSeek-R1 model demonstrates how geopolitical censorship mechanisms can introduce security flaws, such as insecure code generation and broken authentication, especially when handling politically sensitive prompts. These issues underscore the urgent need for robust privacy controls and security-aware development practices in the deployment of LLM-powered systems.
Jun 29, 2026
New research highlights how **large language models (LLMs)** can be operationalized for offensive use, including generating malicious JavaScript and exploit code with limited human involvement. Unit 42 described an *AI-augmented runtime assembly* technique in which a seemingly benign webpage makes client-side API calls to trusted LLM services to obtain code fragments that are then assembled and executed in the victim’s browser, producing a personalized phishing experience. The approach is designed to be evasive by delivering content from trusted LLM domains, producing polymorphic code per visit, and deferring malicious behavior until runtime—reducing the effectiveness of static and network-only detections. Separately, an experiment reported by CybersecurityNews described testing GPT-5.2- and Opus 4.5-based systems against a **zero-day** in the *QuickJS* JavaScript interpreter, resulting in **40+ distinct exploits** across multiple configurations and protection scenarios. The report claims GPT-5.2 solved all presented challenges and that many exploit-generation runs completed in under an hour at relatively modest token costs, suggesting exploit development could increasingly scale with compute and budget rather than scarce expert labor. Together, the reports reinforce that LLMs can be used both for **client-side phishing payload generation** and for **automated vulnerability exploitation**, increasing the speed and variability of attacks defenders may face.
Jun 29, 2026
The rapid adoption of AI-powered code generation tools such as GitHub Copilot, ChatGPT, and Amazon CodeWhisperer has fundamentally changed the software development landscape, introducing new security challenges for enterprise application security teams. Unlike traditional human-written code, AI-generated code often lacks clear provenance, making it difficult to verify its origin or ensure compliance with organizational security policies. This shift has led to the emergence of 'shadow code'—machine-generated code that may bypass standard security reviews and evade detection by traditional static and dynamic analysis tools, increasing the risk of invisible vulnerabilities in production systems. Generative AI models can introduce unique threats, including the creation of 'hallucinated' packages—references to non-existent or malicious libraries that may be inadvertently included in enterprise applications. Additionally, the language-based nature of large language models (LLMs) opens new attack surfaces, such as prompt injection and jailbreaking, where malicious inputs can manipulate model behavior or bypass safety constraints. As organizations accelerate the integration of AI into development workflows, application security programs must adapt to address these novel risks and build trust in the security of AI-powered software.
Jun 29, 2026