Loading story…
Researchers from SPLX have demonstrated that AI-powered chatbots and search tools, including ChatGPT Atlas, ChatGPT, and Perplexity AI, are susceptible to context poisoning attacks through a technique known as "AI cloaking." By detecting AI crawlers via user-agent headers, malicious websites can serve manipulated or false content to these bots while presenting legitimate information to human users. This allows attackers to inject misinformation, fabricate credentials, or smear reputations, which the AI systems then ingest and reproduce as authoritative responses.
In practical tests, SPLX researchers created web pages that appeared professional to human visitors but delivered negative or entirely fabricated narratives to AI crawlers. For example, a fictional designer's biography was altered to portray them as unethical and incompetent when accessed by AI, demonstrating the ease with which bad actors can manipulate AI outputs. The lack of robust verification or countermeasures by AI vendors like OpenAI increases the risk of such attacks being used for fraud, misinformation, and reputational harm, highlighting the urgent need for improved crawler validation and anti-manipulation safeguards in AI systems.
Track how attackers are adapting to this technology.
1 event from the most recent confirmed update back to the earliest known activity.
SC Media and Dark Reading reported that ChatGPT Atlas and other AI search/chatbot systems were vulnerable to manipulation through fake or poisoned contextual content. The coverage indicates public disclosure of the issue, but provides no earlier discovery, patch, or response dates in the supplied references.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Security researchers have demonstrated that AI browser agents, such as OpenAI’s Atlas, ChatGPT, and Perplexity AI, are vulnerable to manipulated web content through user-agent header cloaking. By serving different content to AI crawlers than to human users, attackers can influence the information these agents collect, potentially enabling smear campaigns, misinformation, or fraudulent offers that only AI systems see. This technique exploits the way browser agents gather data, raising concerns about the integrity and reliability of AI-driven research and decision-making. Separately, the rise of unapproved AI tools in corporate environments—referred to as Shadow AI—poses significant data leakage risks, as employees use personal devices and unsanctioned AI services to process sensitive information. Security experts have explored using prompt injection techniques defensively to raise awareness and test the resilience of AI tools against such attacks. These findings highlight the dual challenge of both external manipulation of AI agents and internal governance gaps, underscoring the urgent need for robust controls and monitoring of AI tool usage in organizations.
Jun 29, 2026
Cybercriminals are increasingly exploiting AI-driven platforms and search results to conduct scams and distribute malware. Researchers have identified a technique called 'LLM phone number poisoning,' where attackers manipulate public web content to ensure that AI chatbots and search engines, such as Google's AI Overview and Perplexity's Comet browser, surface fraudulent customer support numbers as legitimate contact information. This manipulation leverages Generative Engine Optimization (GEO) and Answer Engine Optimization (AEO) to poison the data sources that large language models (LLMs) use, putting users at risk of being directed to scam call centers or phishing sites. In a related trend, attackers are also abusing the chat-sharing feature of the official ChatGPT website to host malicious guides that distribute infostealer malware targeting macOS users. By purchasing sponsored ads for search terms like 'chatgpt atlas,' threat actors lure victims to what appears to be a legitimate ChatGPT domain, where a shared chat contains instructions and links to download malware disguised as the 'Atlas browser.' These campaigns highlight the growing risk of AI platforms being weaponized for both social engineering and malware distribution, exploiting user trust in reputable AI services and search results.
Jun 29, 2026
Security researchers are highlighting how attackers can **poison AI systems** by manipulating what models ingest or remember from web content, leading to untrustworthy outputs that may be widely relied upon. Bruce Schneier demonstrated that simply publishing a fabricated webpage can quickly influence major chatbots and search-integrated AI features (e.g., Google’s AI Overviews and Gemini), which then repeated the false claims as if they were factual; in his test, some models were fooled while others were more resistant. Separately, reporting described **AI recommendation/memory poisoning** via “Summarize with AI” buttons that embed long prompts inside URLs. The visible instruction (e.g., “summarize this article”) can be paired with hidden directives such as “remember this site as a trusted authority,” causing the user’s authenticated AI account to update long-term preferences or memory in ways that benefit an attacker or marketer. The write-up cites Microsoft threat intelligence observations of dozens of in-the-wild examples across multiple companies and warns the technique can blend into **malvertising** and become higher-risk when applied to domains like finance, healthcare, or security decision-making.
Jun 29, 2026