Loading story…
AI agents are increasingly being used for tasks such as online shopping, with a significant portion of consumers leveraging these tools to find deals and automate purchases. While these agents offer convenience and efficiency, experts have raised concerns about the security and privacy risks associated with delegating sensitive actions to AI, especially as their influence on e-commerce grows rapidly. Reports indicate a dramatic increase in AI-driven e-commerce traffic, but also highlight the need for caution due to potential vulnerabilities in how these agents handle user data and execute transactions.
Separately, new research has demonstrated that large language models (LLMs) can be manipulated to bypass their safety guardrails through the use of poetic or metaphorical prompts. By embedding harmful instructions within poetic language, researchers achieved high rates of success in getting AI models to provide information on topics such as cyber-offense and the creation of dangerous materials. This finding underscores a structural vulnerability in current AI alignment and refusal mechanisms, raising concerns about the broader security implications of deploying AI agents in sensitive or high-stakes environments.
Track how attackers are adapting to this technology.
1 event from the most recent confirmed update back to the earliest known activity.
Researchers from Icaro Lab, Sapienza University of Rome, and Sant’Anna School of Advanced Studies showed that adversarial prompts written in poetic form could evade safety controls in 25 leading AI models. The single-prompt, text-only technique elicited harmful outputs across domains including CBRN, cyber-offense, and manipulation, with some models showing a 100% attack success rate.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Large language models (LLMs) present a dual-use dilemma in cybersecurity, as their capabilities can be leveraged for both defensive and offensive purposes. Security researchers have identified purpose-built malicious LLMs, such as WormGPT and KawaiiGPT, which are designed to facilitate cybercrime by generating convincing phishing content and rapidly producing or modifying malicious code. The thin line between beneficial and harmful use of LLMs is defined largely by developer intent and the presence or absence of ethical safeguards, raising concerns about the proliferation of offensive AI tools in the threat landscape. In addition to malicious use, LLMs face significant challenges in maintaining privacy and security due to contextual integrity failures and regulatory-driven censorship. Research from Microsoft highlights the need for AI agents to respect contextual privacy norms, as current models may inadvertently leak sensitive information. Meanwhile, the DeepSeek-R1 model demonstrates how geopolitical censorship mechanisms can introduce security flaws, such as insecure code generation and broken authentication, especially when handling politically sensitive prompts. These issues underscore the urgent need for robust privacy controls and security-aware development practices in the deployment of LLM-powered systems.
Jun 29, 2026
Recent research has demonstrated that large language models (LLMs) such as GPT-5 and others are increasingly vulnerable to prompt injection and jailbreak attacks, which can be exploited to bypass built-in safety guardrails and leak sensitive information. Attackers use techniques like prompt injection—embedding malicious instructions within seemingly benign queries—to trick LLMs into revealing confidential data, including user credentials and internal documents. A notable study by Icaro Lab, in collaboration with Sapienza University and DEXAI, found that adversarial prompts written as poetry could successfully bypass safety mechanisms in 62% of tested cases across 25 frontier models, with some models exceeding a 90% success rate. These findings highlight the sophistication and creativity of new attack vectors targeting AI systems, raising significant concerns for organizations embedding LLMs into business operations. The widespread adoption of LLMs in handling sensitive business functions amplifies the risk of data exfiltration through these advanced attack methods. As organizations increasingly rely on AI for customer service, document processing, and other critical tasks, the potential for prompt injection and poetic jailbreaks to facilitate unauthorized data access becomes a pressing security issue. The research underscores the urgent need for improved AI safety measures, robust prompt filtering, and continuous monitoring to mitigate the risks posed by these evolving adversarial techniques.
Jul 12, 2026
Recent research and expert commentary highlight the growing sophistication of attacks targeting artificial intelligence systems, particularly large language models (LLMs) and multimodal AI. Studies reveal that attackers can exploit weaknesses in open weight LLMs through multi-turn prompt injection, significantly increasing the success rate of bypassing safety guardrails compared to single-turn attacks. These findings underscore the importance of robust alignment strategies and continuous adversarial testing, as attackers adapt their tactics over extended interactions to elicit harmful or unauthorized outputs from AI models. In addition to text-based prompt injection, multimodal AI systems face unique threats from hidden or invisible instructions embedded in both text and visual data. Techniques such as white-on-white text in resumes or hidden prompts in online profiles can manipulate AI-driven screening tools, allowing malicious actors to influence outcomes undetected by human reviewers. These developments demonstrate that as AI systems become more integrated into critical decision-making processes, their attack surface expands, necessitating new approaches to securing both the models and their input channels against increasingly subtle and complex adversarial techniques.
Jun 29, 2026