Loading story…
The increasing integration of AI-powered applications and personal assistants into enterprise and personal environments has exposed significant risks related to data leakage and trustworthiness. Security experts highlight that current AI systems lack robust integrity controls, making them susceptible to errors, manipulation, and the mishandling of sensitive personal data. The challenge is compounded by the need for AI assistants to access highly intimate information, which demands both accuracy and strict privacy controls—capabilities that existing solutions do not adequately provide. Proposals to separate personal data stores from AI processing aim to address these gaps, but the problem remains unsolved. Additionally, technical analysis points to the growing risk of data leakage from AI applications, emphasizing the need for a defense-in-depth approach to safeguard information as AI becomes more deeply embedded in business processes.
The operational complexity of modern enterprises, especially with the proliferation of AI tools, further increases the risk of uncontrolled data flows and shadow IT. As collaboration tools and AI-powered solutions multiply, sensitive information is more likely to traverse unmonitored channels, exacerbating the potential for leaks. Security leaders are urged to recognize these evolving risks and implement comprehensive strategies that address both the technical and organizational aspects of AI adoption, including the development of trustworthy AI agents and the mitigation of data leakage through layered security controls.
Track how attackers are adapting to this technology.
2 events from the most recent confirmed update back to the earliest known activity.
Bruce Schneier published an essay arguing that current AI assistants lack integrity controls and proposing that personal data stores be separated from AI systems to improve privacy, security, and user control.
CrowdStrike published an analysis warning that AI-powered applications can expose sensitive information through weak security controls and misconfigurations, and recommended a defense-in-depth approach to reduce leakage risk.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
AI assistants and agents are increasingly integrated into both personal and enterprise workflows, raising significant concerns about data privacy and security. Recent research highlights that users often grant AI agents more data access than necessary, sometimes sharing sensitive information out of convenience or misunderstanding, while still withholding highly sensitive data like Social Security numbers in many cases. This over-permissioning and under-permissioning dynamic underscores the challenge of designing AI systems that request and handle data appropriately, balancing user convenience with privacy protection. At the same time, AI assistants have become attractive targets for cyber attackers, who exploit their growing presence in organizations to launch new types of attacks such as prompt injection. These attacks manipulate AI agents through crafted text-based payloads, bypassing traditional email and endpoint security measures. As AI agents expand the enterprise attack surface, security teams face new challenges in defending against threats that target machine reasoning rather than human behavior, emphasizing the urgent need for updated security architectures and user awareness around AI-driven workflows.
Jun 29, 2026
Enterprises are rapidly integrating artificial intelligence (AI) into their core operations, leading to a significant increase in both the scale and complexity of cybersecurity risks. Autonomous AI agents, once limited to providing suggestions, now act independently within enterprise systems, accessing sensitive data, executing transactions, and triggering downstream workflows without human oversight. These agents, often deployed by individual teams or embedded in third-party software, can inadvertently ingest confidential information, such as customer credit card data, even if the data is only briefly accessible. Unlike human users, AI agents lack contextual understanding and ethical judgment, acting continuously and at scale, which introduces a new category of 'Shadow AI' risk. Multimodal AI systems, which process multiple input streams to generate more human-like outputs, further expand the attack surface. Adversaries can exploit these systems by manipulating data inputs, such as subtly altering images or text, to deceive the AI and bypass security controls. Research has demonstrated that these attacks are not merely theoretical; adversarial manipulations can evade detection and cause significant harm, especially in critical sectors like defense, healthcare, and finance. Organizations are increasingly aware of the dangers posed by AI-augmented threats, including deepfakes and AI-driven social engineering, but many lag in implementing effective technical defenses. Surveys indicate that while a majority of firms have experienced deepfake or AI-voice fraud attempts, more than half have suffered financial losses as a result. Despite this, investment in detection and mitigation technologies remains inadequate, and many companies overestimate their preparedness. The surge in AI adoption is reflected in corporate disclosures, with over 70% of S&P 500 firms now reporting AI as a material risk, up from just 12% two years prior. Reputational and cybersecurity risks are the most frequently cited concerns, followed by legal and regulatory challenges as governments move to establish AI-specific compliance requirements. However, only a minority of corporate boards have formally integrated AI oversight into their governance structures, highlighting a gap between risk awareness and actionable governance. The lack of comprehensive frameworks for managing AI risk leaves organizations vulnerable to both technical and compliance failures. As AI becomes more deeply embedded in business processes, the need for robust governance, continuous education, and responsible-use frameworks becomes increasingly urgent. Security and governance leaders must adapt to this new frontier by developing strategies that address the unique risks posed by autonomous and multimodal AI systems. Failure to do so could result in significant financial, operational, and reputational damage as adversaries continue to exploit the evolving AI landscape.
Jun 29, 2026
Security leaders and experts are warning that the rapid adoption of AI technologies in enterprise environments is introducing new and significant cybersecurity risks. While some industry voices downplay the threat of AI-driven attacks as marketing hype, most threat intelligence professionals and practitioners report that adversaries are already leveraging AI to enhance malware, automate social engineering, and bypass traditional defenses. Research highlights that AI agents, when given autonomy to perform tasks, can be manipulated to break established guardrails, and that model size does not necessarily correlate with resistance to such attacks. In industrial settings, organizations like Siemens are adapting their threat models and operational strategies to address the unique risks posed by AI-driven threats, emphasizing the need for adaptive defenses, cross-team collaboration, and the integration of AI-specific security practices. Analysts are also raising alarms about the use of AI-powered browsers, such as ChatGPT Atlas and Perplexity Comet, which can lead to untraceable data loss and expose sensitive enterprise information through prompt injection vulnerabilities and uncontrolled data flows to the cloud. Security agencies and experts stress the importance of adopting secure-by-design principles when integrating AI features into modern applications, advocating for rigorous threat modeling, least privilege, and continuous monitoring to mitigate the heightened risks associated with automated decision-making systems. As AI becomes a core component of business operations, organizations are urged to proactively address these evolving threats to safeguard their data and critical infrastructure.
Jun 29, 2026