Loading story…
The rapid integration of artificial intelligence into security operations and consumer products has introduced significant new risks, as organizations and vendors rush to deploy AI-driven features without adequate security controls. In security operations centers (SOCs), many teams are experimenting with AI and machine learning tools, but a lack of operational integration and oversight has led to inconsistent results and potential vulnerabilities. The 2025 SANS SOC Survey found that a large portion of SOCs use AI tools informally or without customization, resulting in unreliable outputs and unclear validation processes. When AI is applied to well-defined tasks with rigorous review, it can enhance detection engineering, process repeatability, and staff efficiency, but careless implementation can create new attack surfaces.
Meanwhile, the introduction of advanced AI features in consumer and enterprise platforms has already been exploited by threat actors. Microsoft’s Copilot Studio "Connected Agents" feature, designed for AI-to-AI integration, has been abused by attackers to gain backdoor access to business systems, enabling large-scale phishing and impersonation attacks without leaving audit trails. Additionally, the proliferation of agentic AI browsers and chatbots has exposed users to prompt injection attacks and sophisticated scams, including fake interfaces and misconfigured AI-powered devices. These incidents underscore the urgent need for deliberate, security-focused integration of AI technologies, as well as robust monitoring and validation mechanisms to prevent exploitation by malicious actors.
Track how attackers are adapting to this technology.
3 events from the most recent confirmed update back to the earliest known activity.
Security reporting disclosed that attackers were already abusing Copilot Studio's default-enabled Connected Agents feature to access other agents' knowledge and tools within the same environment. The issue was framed as a significant security exposure that could enable large-scale phishing, impersonation, and brand damage until mitigations or product changes are made.
Zenity Labs showed proof-of-concept attacks in which a malicious agent connected to a privileged Copilot Studio agent and gained the ability to send emails from official company domains. The research highlighted that poor visibility and missing audit logging could let phishing or impersonation activity occur without detection.
At Build 2025, Microsoft introduced the Connected Agents feature for Copilot Studio, enabling AI agents to connect to and share functionality with other agents in the same environment. The feature was described as enabled by default on new agents, creating broad inter-agent access unless restricted.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
3 references tracked. Mallory keeps watching after this page renders.
thehackernews.com
Open sourcecybersecuritynews.com
Open sourcemalwarebytes.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Security leaders and experts are warning that the rapid adoption of AI technologies in enterprise environments is introducing new and significant cybersecurity risks. While some industry voices downplay the threat of AI-driven attacks as marketing hype, most threat intelligence professionals and practitioners report that adversaries are already leveraging AI to enhance malware, automate social engineering, and bypass traditional defenses. Research highlights that AI agents, when given autonomy to perform tasks, can be manipulated to break established guardrails, and that model size does not necessarily correlate with resistance to such attacks. In industrial settings, organizations like Siemens are adapting their threat models and operational strategies to address the unique risks posed by AI-driven threats, emphasizing the need for adaptive defenses, cross-team collaboration, and the integration of AI-specific security practices. Analysts are also raising alarms about the use of AI-powered browsers, such as ChatGPT Atlas and Perplexity Comet, which can lead to untraceable data loss and expose sensitive enterprise information through prompt injection vulnerabilities and uncontrolled data flows to the cloud. Security agencies and experts stress the importance of adopting secure-by-design principles when integrating AI features into modern applications, advocating for rigorous threat modeling, least privilege, and continuous monitoring to mitigate the heightened risks associated with automated decision-making systems. As AI becomes a core component of business operations, organizations are urged to proactively address these evolving threats to safeguard their data and critical infrastructure.
Jun 29, 2026
Security researchers and industry experts are raising alarms about the growing use of artificial intelligence (AI) in both offensive and defensive cybersecurity operations. Attackers are leveraging AI to bypass advanced security controls, as demonstrated by a researcher who used AI to defeat an "AI-powered" web application firewall, and by the emergence of new malware that exploits AI model files and browser vulnerabilities to evade detection and exfiltrate credentials. Meanwhile, defenders are grappling with the proliferation of unsanctioned AI tools in the workplace, the challenge of auditing AI decision-making, and the surge in AI-powered bug hunting, which has led to a dramatic increase in vulnerability discoveries and bug bounty payouts. The risks are compounded by the lack of clear AI usage policies, the potential for data leaks through generative AI tools, and the difficulty in monitoring or controlling how sensitive information is processed and stored by these systems. Industry reports highlight that a significant portion of employees use unauthorized AI applications, often exposing sensitive data without IT oversight, and that prompt injection and model manipulation are now common vulnerability types. The security community is also debating the extent to which ransomware and other attacks are truly "AI-driven," with some reports criticized for overstating the role of AI in current threat activity. As organizations rush to adopt AI for efficiency and innovation, experts urge the implementation of robust governance, continuous monitoring, and red-teaming to anticipate and mitigate the evolving risks posed by both sanctioned and shadow AI systems. The rapid evolution of AI in cybersecurity is forcing a reevaluation of traditional defense models, emphasizing the need for transparency, operational oversight, and adaptive security strategies.
Jun 29, 2026
The rapid integration of artificial intelligence (AI) technologies into business operations has introduced new risks and challenges for cybersecurity teams. High-profile incidents, such as the accidental leakage of confidential information at Samsung via ChatGPT and the manipulation of AI chatbots at companies like Air Canada and Chevrolet dealerships, have highlighted the potential for both accidental and malicious exploitation of generative AI tools. As organizations increasingly rely on AI for efficiency and innovation, experts emphasize the need to adapt incident response plans to address AI-specific risks, referencing resources like MIT's AI Risk Repository to categorize and understand these emerging threats. While AI is automating repetitive security tasks and augmenting defensive capabilities, it is also being leveraged by attackers to enhance existing tradecraft, such as phishing and code generation. Recent research and industry analysis indicate that most so-called "AI malware" remains at an early stage of maturity, serving as a force multiplier rather than introducing fundamentally new attack techniques. Security professionals are advised to focus on monitoring the abuse of legitimate AI services and to avoid overreacting to sensationalized claims of fully autonomous AI-driven attacks, as the real-world impact of such threats remains limited at present.
Jun 29, 2026