Loading story…
AI assistants and agents are increasingly integrated into both personal and enterprise workflows, raising significant concerns about data privacy and security. Recent research highlights that users often grant AI agents more data access than necessary, sometimes sharing sensitive information out of convenience or misunderstanding, while still withholding highly sensitive data like Social Security numbers in many cases. This over-permissioning and under-permissioning dynamic underscores the challenge of designing AI systems that request and handle data appropriately, balancing user convenience with privacy protection.
At the same time, AI assistants have become attractive targets for cyber attackers, who exploit their growing presence in organizations to launch new types of attacks such as prompt injection. These attacks manipulate AI agents through crafted text-based payloads, bypassing traditional email and endpoint security measures. As AI agents expand the enterprise attack surface, security teams face new challenges in defending against threats that target machine reasoning rather than human behavior, emphasizing the urgent need for updated security architectures and user awareness around AI-driven workflows.
Track how attackers are adapting to this technology.
5 events from the most recent confirmed update back to the earliest known activity.
A reported attack dubbed Gemini Jack showed how hidden instructions embedded in documents or emails could manipulate Google's Gemini assistant into leaking sensitive information. The attack was triggered by routine employee queries, exploiting the AI's legitimate access rather than stolen credentials.
Reporting described a rise in attacks targeting AI assistants and agents, with adversaries using prompt injection and related AI-specific techniques to manipulate automated actions. The coverage noted that traditional email and phishing defenses often fail against these attacks because agents may process hidden instructions in metadata or HTML automatically.
The study also introduced a hybrid AI model that predicted user permission preferences with high accuracy. Experts cautioned that prediction accuracy alone is not sufficient for secure deployment, especially in regulated environments.
The same research highlighted security concerns around AI permission models, including prompt injection attacks and risks from running permission inference on shared infrastructure. Researchers advised organizations to isolate these systems and ensure compliance rules take precedence over learned user preferences.
A research study examined how people respond when AI agents request access to personal data for different tasks. It found users often allow automatic sharing for convenience but become more restrictive with sensitive data or after agent mistakes, creating both over-permissioning and under-permissioning risks.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
3 references tracked. Mallory keeps watching after this page renders.
securitysenses.com
Open sourcehelpnetsecurity.com
Open sourcetechrepublic.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
The increasing integration of AI-powered applications and personal assistants into enterprise and personal environments has exposed significant risks related to data leakage and trustworthiness. Security experts highlight that current AI systems lack robust integrity controls, making them susceptible to errors, manipulation, and the mishandling of sensitive personal data. The challenge is compounded by the need for AI assistants to access highly intimate information, which demands both accuracy and strict privacy controls—capabilities that existing solutions do not adequately provide. Proposals to separate personal data stores from AI processing aim to address these gaps, but the problem remains unsolved. Additionally, technical analysis points to the growing risk of data leakage from AI applications, emphasizing the need for a defense-in-depth approach to safeguard information as AI becomes more deeply embedded in business processes. The operational complexity of modern enterprises, especially with the proliferation of AI tools, further increases the risk of uncontrolled data flows and shadow IT. As collaboration tools and AI-powered solutions multiply, sensitive information is more likely to traverse unmonitored channels, exacerbating the potential for leaks. Security leaders are urged to recognize these evolving risks and implement comprehensive strategies that address both the technical and organizational aspects of AI adoption, including the development of trustworthy AI agents and the mitigation of data leakage through layered security controls.
Jun 29, 2026
Organizations are rapidly adopting artificial intelligence (AI) tools to enhance cybersecurity operations, streamline workflows, and improve productivity, but this trend introduces significant new risks and challenges. Reports indicate that cybersecurity professionals with AI security skills are in high demand, as companies seek to leverage AI for vulnerability management, threat detection, and automation of security tasks. The integration of AI into security teams’ arsenals is accelerating, with agentic AI tools becoming increasingly common for both defensive and operational purposes. However, the proliferation of AI-powered applications, such as AI notetakers in virtual meetings, raises concerns about data privacy, compliance, and the potential for sensitive information exposure. Many AI notetaking tools operate outside official enterprise systems, often lacking robust security controls such as SOC 2 certification, GDPR compliance, or strong encryption, making them vulnerable to data breaches and mishandling. The risk is compounded by the rapid spread of these tools within organizations, sometimes without proper vetting by legal, security, or procurement teams. Transcripts generated by these applications can be stored in third-party systems, increasing the risk of unauthorized access or legal discoverability. Security leaders are advised to develop clear policies and governance frameworks to manage the use of AI tools, ensuring that only approved applications with adequate security measures are deployed. The evolving landscape of AI in cybersecurity also includes increased merger and acquisition activity, as companies seek to acquire innovative AI security capabilities. Industry analysis highlights the need for continuous evaluation of AI models, such as DeepSeek, and the security implications of open-source agent frameworks like OpenAI’s AgentKit. The impact of AI-generated code on application security is another emerging concern, as automated code generation can introduce vulnerabilities if not properly reviewed. As AI becomes more embedded in business processes, organizations must balance the benefits of automation and efficiency with the imperative to safeguard sensitive data and maintain regulatory compliance. Security teams are encouraged to stay informed about the latest trends in AI security, invest in upskilling staff, and implement layered defenses to mitigate the unique risks posed by AI-driven tools. The convergence of AI and cybersecurity is reshaping the threat landscape, requiring proactive risk management and strategic investment in secure AI adoption.
Jun 29, 2026
Agentic AI, characterized by its ability to autonomously perform complex tasks without human supervision, is rapidly transforming IT operations and cybersecurity. Security leaders highlight its strengths in processing vast data volumes, enabling real-time threat detection and response, and automating routine or large-scale security tasks, thereby allowing human teams to focus on strategic initiatives. Industry experts emphasize that agentic AI can optimize resource utilization, accelerate problem resolution, and fundamentally change the way IT organizations manage infrastructure, support, and security operations. However, the adoption of agentic AI introduces new security risks, particularly around data leakage and trust management. Recent research demonstrates that AI agents with web search and internal document access can be manipulated through indirect prompt injection, causing them to exfiltrate sensitive company data without user awareness. Security professionals stress the importance of evolving identity and Zero Trust principles to address the unique challenges posed by autonomous AI agents, including the risk of rogue behavior and ethical blind spots. Organizations are advised to implement robust controls and monitoring to mitigate these emerging threats while leveraging the operational benefits of agentic AI.
Jun 29, 2026