Loading story…
Permiso documented how hosted AI models can be hijacked for “dark roleplaying,” underscoring that externally accessible model services can be manipulated for harmful or policy-evading use cases. The issue adds an abuse-management requirement to AI deployments alongside conventional application and cloud security controls.
Separately, Wiz found hundreds of validated, still-active secrets in public code repositories during a month-long scan, with AI-service credentials disproportionately represented. Jupyter notebooks and AI-agent configuration files such as mcp.json frequently exposed keys through hardcoded values, outputs, diagnostics, and error messages; 56% of company-impacting secrets were in employees’ personal repositories. Organizations should expand secret scanning to AI providers, scan Git history and CI/CD pipelines, block commits containing notebook outputs, and revoke or rotate exposed credentials.
See attribution, scope, and your downstream exposure.
2 events from the most recent confirmed update back to the earliest known activity.
The researchers reported that a secret they disclosed to the Microsoft Security Response Center was rated Critical and could have enabled exposure of sensitive HR data.
A month-long scan of thousands of public code repositories found hundreds of validated exposed secrets, many still active. AI-service credentials represented three of the five most common validated secret types, and Jupyter Notebook files were the most leak-prone file type.
See attribution, scope, and whether this vendor sits anywhere in your supply chain.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Recent research has highlighted significant security vulnerabilities in the development and deployment of AI models, particularly those with open weights and in fast-growing AI startups. Cisco's AI Threat Research found that open-weight language models are highly susceptible to adversarial manipulation, especially during extended multi-turn conversations. Attackers can gradually steer these models toward producing harmful or restricted outputs, often bypassing moderation systems designed for single-turn interactions. The study revealed that some models, such as Mistral’s Large-2 and Alibaba’s Qwen3-32B, had success rates for multi-turn attacks exceeding 85%, underscoring the challenge of maintaining safety context over time. Separately, a report from Wiz identified that nearly two-thirds of leading private AI companies have leaked sensitive credentials, such as API keys and access tokens, on GitHub. These leaks could expose private AI models, training data, and internal organizational details, with some credentials belonging to major platforms like Hugging Face and LangChain. The findings suggest that the rapid pace of AI innovation is outstripping secure development practices, leaving critical assets vulnerable to exploitation. Both studies emphasize the urgent need for improved security controls and DevSecOps hygiene in the AI sector to protect intellectual property and prevent adversarial abuse of AI systems.
Jul 12, 2026
Organizations are facing escalating security exposure from employee and developer use of public AI services without adequate controls. A viral social-media trend prompting ChatGPT to “create a caricature of me and my job based on everything you know about me” is being cited as a visible indicator of **shadow AI** and potential **data leakage**, because users may be sharing work context and sensitive details into non-sanctioned LLM accounts and then posting outputs publicly. The public sharing can also aid **social engineering** by revealing roles, access, and other targeting cues; if an attacker compromises an employee’s LLM account, they may be able to review prompt history for sensitive information. Separately, Cyble Research and Intelligence Labs (CRIL) reported widespread **credential mismanagement** involving OpenAI/ChatGPT API keys, identifying **5,000+ public GitHub repositories** with hardcoded credentials and roughly **3,000 production websites** exposing active keys in client-side assets (e.g., JavaScript). CRIL noted that secrets often persist in commit histories, forks, and archived projects, and that automated scanning/indexing can shrink the time from exposure to abuse to minutes—enabling unauthorized API usage, cost fraud, and potential downstream compromise where those keys are tied into broader application workflows.
Jun 29, 2026
Recent research and industry developments have highlighted significant vulnerabilities in the integrity and security of AI models and their supply chains. Researchers demonstrated that inserting as few as 250 carefully crafted, poisoned documents into an AI model's training data can corrupt its outputs, causing it to generate nonsensical or manipulated responses. This attack method, tested on models up to 13 billion parameters, shows that even a small amount of malicious data can have a disproportionate impact on model behavior, raising concerns about the robustness of AI training pipelines. In parallel, new attack techniques such as the Lies-in-the-Loop (LITL) attack have been identified, where adversaries plant seemingly benign code or dependencies that only reveal malicious behavior under specific runtime conditions. These attacks are particularly insidious because they can deceive AI-powered code assistants, especially those that learn from human feedback, into approving or perpetuating unsafe code. The LITL attack demonstrates that even with human oversight, AI systems can be manipulated into executing or recommending malicious actions, undermining the trust in human-in-the-loop security models. Additionally, the rapid growth of AI model repositories like Hugging Face has introduced new supply chain risks, as models themselves can harbor hidden malware, backdoors, or payloads that activate during inference or deserialization. Security firms are now developing specialized scanners to detect such threats in AI model artifacts, recognizing that traditional software security measures are insufficient for the unique risks posed by AI models. Real-world incidents have already shown that attackers can exploit model file formats to execute arbitrary code or exfiltrate data, making the AI supply chain a lucrative target. The convergence of these threats underscores the urgent need for robust security practices in AI development, including rigorous data curation, model artifact scanning, and enhanced monitoring of AI-assisted development workflows. As AI becomes more deeply integrated into production environments, the potential impact of compromised models or manipulated code suggestions grows, threatening both data integrity and operational security. The industry is responding with new tools and collaborative research, but the evolving threat landscape requires continuous vigilance and adaptation. Organizations deploying AI must recognize that both the data used for training and the models themselves are attack surfaces. Proactive measures, such as integrating supply chain scanning and monitoring AI-human interactions, are essential to mitigate these emerging risks. The findings from recent studies and product announcements serve as a wake-up call for the broader adoption of AI-specific security controls. As adversaries become more sophisticated in targeting AI systems, defenders must stay ahead by understanding and addressing these novel attack vectors. The future of AI security will depend on the industry's ability to anticipate, detect, and neutralize threats at every stage of the AI lifecycle.
Jul 12, 2026