Loading story…
Major AI chatbots, including OpenAI's ChatGPT, Google's Gemini, xAI's Grok, and DeepSeek, have been found to cite Russian state-affiliated sources and repeat pro-Kremlin narratives when responding to queries about the war in Ukraine. Research by the Institute for Strategic Dialogue (ISD) and NewsGuard revealed that up to a quarter of chatbot answers referenced Russian state media or sites linked to Russian intelligence, even when those sources are sanctioned in the EU. The studies tested responses in multiple languages and with various prompt types, highlighting the risk that large language models (LLMs) may inadvertently amplify disinformation and undermine sanctions on Moscow-backed media.
The ISD's research emphasized that chatbots are increasingly used as search engines, raising concerns about their role in spreading sanctioned propaganda and the effectiveness of current safeguards. The findings suggest that Russian disinformation networks, such as the "Pravda" network, are successfully "grooming" LLMs by seeding misleading content online, which is then parroted by AI systems. This situation underscores the need for greater scrutiny and regulatory oversight of AI platforms to prevent the dissemination of state-backed disinformation, especially in sensitive geopolitical contexts like the Russian invasion of Ukraine.
Track how attackers are adapting to this technology.
6 events from the most recent confirmed update back to the earliest known activity.
Previous research by NewsGuard found that AI chatbots were susceptible to Russian disinformation campaigns, establishing an earlier warning sign that generative AI systems could surface Kremlin-aligned narratives.
OpenAI and other companies said they recognize the problem of chatbots surfacing Russian propaganda and are working to improve model behavior. Researchers and vendors noted that the challenge remains difficult because disinformation networks rapidly adapt to evade detection.
Following the findings, the report urged clearer industry standards and stronger regulatory action to prevent AI systems from referencing sanctioned or disinformation-linked sources. The results also raised concerns about whether chatbot providers can effectively comply with EU sanctions as these tools are increasingly used for real-time information.
Across the tested systems, ChatGPT was reported to cite Russian sources most frequently, while Google's Gemini surfaced the least state-attributed media and gave the strongest warnings about questionable content. The comparison highlighted uneven safety performance among major AI platforms.
The ISD found that biased or malicious prompts increased the likelihood that chatbots would reference Russian propaganda, with one report noting up to 24% of malicious queries eliciting state-attributed sources. The research also identified the Pravda disinformation network as a notable actor in flooding the web with Russian propaganda, though only a small share of findings were directly tied to it.
The Institute for Strategic Dialogue reported that chatbots including ChatGPT, Gemini, Grok, and DeepSeek returned answers about the war in Ukraine that cited Russian state-attributed sources, including sanctioned outlets and sites linked to Russian intelligence. The study found this occurred in nearly 18% of tested responses overall.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Researchers from SPLX have demonstrated that AI-powered chatbots and search tools, including *ChatGPT Atlas*, *ChatGPT*, and *Perplexity AI*, are susceptible to context poisoning attacks through a technique known as "AI cloaking." By detecting AI crawlers via user-agent headers, malicious websites can serve manipulated or false content to these bots while presenting legitimate information to human users. This allows attackers to inject misinformation, fabricate credentials, or smear reputations, which the AI systems then ingest and reproduce as authoritative responses. In practical tests, SPLX researchers created web pages that appeared professional to human visitors but delivered negative or entirely fabricated narratives to AI crawlers. For example, a fictional designer's biography was altered to portray them as unethical and incompetent when accessed by AI, demonstrating the ease with which bad actors can manipulate AI outputs. The lack of robust verification or countermeasures by AI vendors like OpenAI increases the risk of such attacks being used for fraud, misinformation, and reputational harm, highlighting the urgent need for improved crawler validation and anti-manipulation safeguards in AI systems.
Jun 29, 2026
OpenAI reported multiple cases of **malicious use of ChatGPT and related API access** spanning financially motivated fraud and government-aligned information operations, including romance scams, fake legal services, coordinated influence campaigns, and a state-linked harassment effort. In one detailed case (“**Operation Date Bait**”), actors used ChatGPT accounts and an API customer to generate ad copy for a fake dating service, drive victims to *Telegram*, and run semi-automated “missions” that required escalating payments; OpenAI said it banned accounts involved and noted the group’s claimed revenues could not be independently verified. OpenAI also described model use for translation, persona development, and internal coordination (e.g., generating status reports and assigning targets projected payout values), and cited evidence such as repeated text patterns posted by multiple X accounts. Google’s Threat Intelligence Group (GTIG) separately assessed that **nation-state threat actors** are operationalizing LLMs (including *Gemini* and other tools) for reconnaissance, victim targeting, and generating culturally fluent, multi-turn “rapport-building” phishing lures. GTIG also observed abuse of generative AI services’ public sharing features to host deceptive content used in **ClickFix** campaigns, where victims are tricked into copying and pasting malicious commands into a terminal; the activity was first observed in early December 2025. The reporting highlights a converging trend: adversaries are using LLMs to scale and localize social engineering while shifting delivery mechanisms toward trusted AI platforms and shared content features to increase credibility and reach.
Jun 29, 2026
OpenAI has taken action to ban several ChatGPT accounts believed to be linked to Chinese government entities and cybercriminal groups attempting to leverage AI for surveillance and cyber operations. According to OpenAI’s latest threat report, these accounts were primarily using ChatGPT to enhance existing cyber capabilities rather than inventing entirely new attack methods. The banned users often requested assistance from ChatGPT in designing tools for large-scale monitoring and analysis, such as social media listening platforms capable of scanning major networks like X, Facebook, Instagram, Reddit, TikTok, and YouTube for content deemed extremist or politically sensitive. In one notable case, a user suspected of operating from China via VPN asked ChatGPT to help create promotional materials and project plans for a surveillance tool, allegedly for a government client, though OpenAI could not confirm if the tool was ultimately deployed. Other banned accounts sought to use ChatGPT to identify funding sources for social media accounts critical of the Chinese government and to uncover petition organizers in Mongolia, but the AI only provided publicly available information and did not reveal sensitive data. OpenAI’s report highlights that these activities are indicative of how authoritarian regimes might seek to abuse AI capabilities in the future, even if current attempts remain limited in scope. The threat report also notes that most adversarial AI use cases involve automating and scaling traditional hacking tasks, such as malware development, command-and-control infrastructure, spearphishing, and reconnaissance, rather than creating novel attack vectors. The accounts in question displayed characteristics consistent with Chinese intelligence operations, including the use of the Chinese language and targeting sectors like Taiwan’s semiconductor industry, U.S. academia, think tanks, and organizations critical of the Chinese government. Technical overlaps were observed between these accounts and a known Chinese cyber espionage group, suggesting a coordinated effort. OpenAI’s intelligence team, including principal investigator Ben Nimmo, emphasized that while the AI models were not directly used to conduct surveillance, the planning and documentation support provided by ChatGPT could facilitate future operations. The company’s proactive monitoring and banning of these accounts underscore the growing concern over the misuse of generative AI by state actors and cybercriminals. OpenAI’s findings provide valuable insights into the evolving tactics of threat actors who are increasingly integrating AI into their existing cyber workflows. The report also raises questions about the potential for more sophisticated abuses of AI as the technology matures. OpenAI’s actions reflect a broader industry trend of AI providers taking steps to detect and disrupt malicious use of their platforms. The company continues to monitor for similar activities and collaborates with other stakeholders to mitigate risks associated with adversarial AI. These developments highlight the importance of vigilance and cross-sector cooperation in addressing the security challenges posed by generative AI technologies.
Jun 29, 2026