Loading story…
UNSW Sydney researchers found that making large language models emulate intoxicated behavior can substantially weaken their privacy and safety controls. The study, In Vino Veritas and Vulnerabilities, tested GPT-3.5, GPT-4, Llama 2, Llama 3.1, and Mistral through direct “act drunk” prompts, fine-tuning on more than 57,000 drunken-style messages, and reinforcement learning. Against ConfAIde privacy tests, GPT-4's rate of accepting requests to disclose confidential information increased from 6% in its baseline state to 54% when prompted to act drunk and 75% following drunk-text fine-tuning.
The intoxication-style conditioning also increased harmful-request compliance in JailbreakBench and in some cases remained effective despite evaluated jailbreak defenses. Researchers reported that the technique achieved among the strongest results across nearly all tested models, enabling prohibited outputs and private-data leakage. Organizations deploying LLMs should treat persona and style-conditioning prompts as a material jailbreak vector, test safeguards against behavioral role-play transformations, and prevent sensitive information from being exposed to models without appropriate isolation and access controls.
Track how attackers are adapting to this technology.
2 events from the most recent confirmed update back to the earliest known activity.
A Reddit post described telling ChatGPT it had consumed whisky to lower its inhibitions, an approach similar to the later simulated-drunkenness technique.
UNSW Sydney and other Australian machine-learning researchers reported that prompting, fine-tuning, or reinforcement-learning models to emulate drunken writing weakened privacy and safety guardrails in GPT-3.5, GPT-4, Llama, and Mistral models. Their evaluations found increased confidential-data disclosure and harmful-request compliance, with some drunk-conditioned models continuing harmful responses despite tested jailbreak defenses.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Recent research has demonstrated that large language models (LLMs) such as GPT-5 and others are increasingly vulnerable to prompt injection and jailbreak attacks, which can be exploited to bypass built-in safety guardrails and leak sensitive information. Attackers use techniques like prompt injection—embedding malicious instructions within seemingly benign queries—to trick LLMs into revealing confidential data, including user credentials and internal documents. A notable study by Icaro Lab, in collaboration with Sapienza University and DEXAI, found that adversarial prompts written as poetry could successfully bypass safety mechanisms in 62% of tested cases across 25 frontier models, with some models exceeding a 90% success rate. These findings highlight the sophistication and creativity of new attack vectors targeting AI systems, raising significant concerns for organizations embedding LLMs into business operations. The widespread adoption of LLMs in handling sensitive business functions amplifies the risk of data exfiltration through these advanced attack methods. As organizations increasingly rely on AI for customer service, document processing, and other critical tasks, the potential for prompt injection and poetic jailbreaks to facilitate unauthorized data access becomes a pressing security issue. The research underscores the urgent need for improved AI safety measures, robust prompt filtering, and continuous monitoring to mitigate the risks posed by these evolving adversarial techniques.
Jul 12, 2026
Researchers have discovered that phrasing prompts as poetry can effectively bypass safety mechanisms in large language models (LLMs), enabling users to elicit harmful or restricted outputs. In a recent study, adversarial poetic prompts were tested across 25 proprietary and open-weight LLMs, including those from major providers such as OpenAI, Meta, and Anthropic. The poetic approach achieved an average jailbreak success rate of 62% for hand-crafted poems and 43% for meta-prompt conversions, significantly outperforming non-poetic baselines. The technique proved effective across a range of sensitive topics, including instructions for creating nuclear weapons, malware, and other high-risk content, highlighting a systematic vulnerability in current AI safety and alignment protocols. The research involved converting over a thousand known harmful prompts into verse using a standardized meta-prompt, then evaluating the models' responses with both automated and human-labeled safety assessments. The findings suggest that stylistic variations, such as poetic framing, can systematically circumvent existing guardrails, raising concerns about the robustness of current LLM safety measures. The researchers have notified major AI vendors of their results, but have withheld specific prompt examples for security reasons. This vulnerability underscores the need for more resilient alignment strategies and evaluation methods in AI safety engineering.
Jun 29, 2026
Cisco reported that **multi-turn jailbreak** techniques—iterative, conversational prompt sequences designed to erode safety guardrails—successfully bypassed protections in eight major **open-weight** large language models **92.78%** of the time, while single-turn prompt attempts were notably less effective. The findings, published in Cisco’s *State of AI Security* research and covered by multiple outlets, highlight that many enterprise AI deployments using downloadable, self-hosted models may be more vulnerable to sustained adversarial prompting than organizations assume. The report’s risk framing is amplified by broader concerns that model misuse and capability leakage can scale quickly: Anthropic separately alleged coordinated **model distillation** activity by Chinese AI labs using large volumes of fraudulent accounts and proxy infrastructure to extract advanced behaviors from *Claude*, warning that copied models may lack comparable safety controls and could be repurposed for malicious use. Related research coverage also notes that LLMs can sometimes be induced—via specialized prompting/jailbreaking methods—to reproduce near-verbatim copyrighted text from training data, underscoring that prompt-based attacks can drive both **policy bypass** and **data/content extraction** outcomes, particularly when guardrails are tested over extended interactions.
Jun 29, 2026