Loading story…
Recent research and expert commentary highlight the growing sophistication of attacks targeting artificial intelligence systems, particularly large language models (LLMs) and multimodal AI. Studies reveal that attackers can exploit weaknesses in open weight LLMs through multi-turn prompt injection, significantly increasing the success rate of bypassing safety guardrails compared to single-turn attacks. These findings underscore the importance of robust alignment strategies and continuous adversarial testing, as attackers adapt their tactics over extended interactions to elicit harmful or unauthorized outputs from AI models.
In addition to text-based prompt injection, multimodal AI systems face unique threats from hidden or invisible instructions embedded in both text and visual data. Techniques such as white-on-white text in resumes or hidden prompts in online profiles can manipulate AI-driven screening tools, allowing malicious actors to influence outcomes undetected by human reviewers. These developments demonstrate that as AI systems become more integrated into critical decision-making processes, their attack surface expands, necessitating new approaches to securing both the models and their input channels against increasingly subtle and complex adversarial techniques.
Track how attackers are adapting to this technology.
2 events from the most recent confirmed update back to the earliest known activity.
A Help Net Security report examined how attackers can use gradual, patient interaction strategies to push past security guardrails in open-weight AI models, underscoring an emerging attack technique against AI systems.
A SecuritySenses report described how multimodal AI systems can already be manipulated with invisible or hidden instructions embedded in content, highlighting a real-world security weakness in these models.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Cybersecurity threats leveraging artificial intelligence are rapidly evolving, with attackers using AI to automate vulnerability discovery, conduct deepfake-enabled social engineering, and compromise large language models (LLMs) through data poisoning and malicious model files. A Gartner survey found that 62% of organizations experienced deepfake attacks, often combined with social engineering to defraud companies, while 32% faced attacks on AI applications such as prompt injection. Security researchers have demonstrated prompt injection vulnerabilities in AI summarization tools, highlighting the risks of insufficient separation between user input and system prompts. Additionally, experts warn that LLMs can be compromised via poisoned training data, malicious adapters, or trojanized model files, expanding the attack surface for organizations adopting AI technologies. These developments underscore the need for robust AI governance, technical defenses, and employee awareness to counter increasingly sophisticated AI-driven threats.
Jun 29, 2026
Security researchers and commentators warned that attacks on **LLM-based systems** are evolving beyond simple “prompt injection” into a broader execution mechanism dubbed **promptware**, with a proposed seven-step **promptware kill chain** to describe how malicious instructions enter and propagate through AI-enabled applications. The core risk highlighted is architectural: LLMs treat system instructions, user input, and retrieved content as a single token stream, enabling **indirect prompt injection** where hostile instructions are embedded in external data sources (web pages, emails, shared documents) that an LLM ingests at inference time; the attack surface expands further as models become **multimodal**, allowing instructions to be hidden in images or audio. Related academic work demonstrated a concrete multimodal variant against **embodied AI** using large vision-language models: **CHAI (Command Hijacking Against Embodied AI)**, which embeds deceptive natural-language instructions into visual inputs (e.g., road signs) to influence agent behavior in scenarios including drone emergency landing, autonomous driving, and object tracking, reportedly outperforming prior attacks in evaluations. Separately, reporting on a viral “AI caricature” social-media trend framed the risk as downstream **social engineering** and potential **LLM account takeover** leading to exposure of prompt histories and employer-sensitive data; while largely hypothetical, it underscores how widespread consumer LLM use and public oversharing can increase the likelihood and impact of prompt-driven compromise paths.
Jun 29, 2026
Security researchers and industry experts are warning that the rapid evolution of AI-native malware and the abuse of large language models (LLMs) are creating new, sophisticated cyber threats that traditional security tools struggle to detect. Future malware is expected to embed LLMs or similar models, enabling self-modifying code, context-aware evasion, and autonomous ransomware operations that adapt to their environment and evade static detection rules. This shift is outpacing the capabilities of most SIEMs and security operations centers, which are limited by the scale and complexity of detection rules required to keep up with AI-driven attack techniques. The need for automated rule deployment and AI-native detection intelligence is becoming critical, as defenders face challenges in maintaining effective coverage and managing the operational burden of thousands of detection rules. In addition to the threat of AI-powered malware, new research highlights a paradox where iterative improvements made by LLMs to code can actually increase the number of critical vulnerabilities, even when explicitly tasked with enhancing security. This phenomenon, termed 'feedback loop security degradation,' underscores the necessity for skilled human oversight in the development process, as reliance on AI coding assistants alone can introduce significant risks. The growing prevalence of agentic AI and the expansion of non-human identities further complicate the security landscape, requiring organizations to rethink identity management and detection strategies to address these emerging threats effectively.
Jun 29, 2026