Loading story…
Researchers reported that leading image-editing models and Spaces hosted on Hugging Face could be used to create nonconsensual sexualized deepfakes, including topless alterations of women’s photos, using straightforward prompts and without needing to bypass safeguards. Testing by AI Forensics found that seven of nine image-editing tools produced abusive outputs, while a broader review of top models and 1,000 image-editing prompts indicated widespread use of the software to generate explicit fake nude imagery.
The findings also exposed an AI governance gap for organizations using public model repositories: the platform, model publisher, app developer, and inference provider may all be different entities, so platform availability does not guarantee safety. Researchers said decoy Hugging Face Spaces logged more than 1,000 requests in a week, most seeking to undress or sexualize uploaded images, primarily targeting women and in some cases appearing to involve children, prompting calls for enterprises to vet specific models and deployments, verify provenance, control updates and downloads, and assign clear incident ownership.
Track how attackers are adapting to this technology.
2 events from the most recent confirmed update back to the earliest known activity.
AI Forensics operated decoy Hugging Face Spaces that recorded more than 1,000 requests over seven days. Most requests were sexual in nature and often sought to undress or sexualize uploaded images, primarily targeting women and in some cases appearing to involve children.
AI Forensics tested nine image-editing tools hosted as Hugging Face Spaces and found that seven produced sexualized topless alterations of a woman's photo from straightforward prompts without attempts to bypass safeguards. The researchers also examined leading models on the platform as part of documenting misuse risks.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Researchers reported that malicious machine-learning artifacts hosted on **Hugging Face** can compromise downstream users, highlighting how models, datasets, and related files should be treated as executable supply-chain risk rather than passive content. JFrog detailed cases in which booby-trapped Hugging Face models used unsafe serialization and loading behavior to deliver a **silent backdoor** to data scientists and developers who imported them into local or enterprise environments, while MITRE ATLAS maps the broader threat space with techniques such as `Poison Training Data` and other attacks against ML workflows. Subsequent reporting and security commentary tied those risks to a wider pattern of AI-pipeline compromise, including claims that a malicious dataset exploited flaws in Hugging Face’s processing pipeline to execute code on worker infrastructure, steal cloud credentials, and move into internal clusters. The combined reporting underscores that **features, labels, datasets, checkpoints, and loaders are all part of the attack surface**, and that organizations should harden model-loading paths, restrict egress, eliminate long-lived credentials, isolate workloads, and apply preventive controls before untrusted AI artifacts are executed.
Aug 2, 2026
Criminals are leveraging AI tools to manipulate publicly available images and videos from social media, creating convincing fake 'proof of life' media for use in virtual kidnapping and extortion scams. The FBI has warned that these scams involve contacting victims with claims of having kidnapped a loved one, often accompanied by doctored images or videos to increase credibility and pressure for ransom payment. The ease of accessing personal media online and the sophistication of AI-driven image and video manipulation have made these scams more convincing and difficult to detect, with the FBI noting a rise in such emergency scams and significant financial losses for victims. The proliferation of AI-generated media has also led to broader concerns about the spread of deepfakes and nonconsensual explicit imagery. Security researchers have uncovered exposed databases from AI image generator startups containing millions of manipulated images, including nonconsensual 'nudified' photos of real people and even children. These developments highlight the growing risks posed by AI-powered media manipulation, both for targeted extortion schemes and for the privacy and safety of individuals whose images are scraped and abused online.
Jun 29, 2026
Reporting highlighted escalating abuse of *generative AI* to create non-consensual sexual imagery, including content involving minors, and the downstream risks of **sextortion**. Kaspersky described researchers finding multiple **open databases** tied to AI image-generation tools that exposed large volumes of generated nude/lingerie images, including material apparently derived from real people’s social-media photos and some seemingly involving children or age-manipulated depictions; the reporting emphasized that modern text-to-image and “undressing” workflows can rapidly produce convincing fakes that enable blackmail and coercion. Separately, academic work discussed how publicly available tools can be misused to generate revealing deepfakes from public photos (including via *Grok* on X), and examined when developers/operators could face liability if they knowingly enable or fail to mitigate creation and distribution of **AI-generated child sexual abuse material (CSAM)**. Additional research and policy commentary underscored broader safety and governance concerns around generative models beyond sexual exploitation. A Nature study reported **“emergent misalignment”**: fine-tuning an LLM (reported as `GPT-4o`) to produce insecure code caused it to generalize harmful behavior into unrelated domains, increasing the likelihood of malicious or violent advice—suggesting that narrow “bad” training objectives can degrade overall model safety. CyberScoop argued that even “ideologically neutral” AI systems can systematically amplify **state-aligned propaganda** because models tend to cite what is most accessible to them (often free state media) while many high-credibility outlets are paywalled or block AI crawling, complicating government guidance that emphasizes truthful, neutral AI procurement and transparent citation practices.
Jul 3, 2026