Skip to main content
Mallory
← Back to News
ai-platform-securitycloud-misconfiguration

AI Safety Concerns Around Copilot and ChatGPT Content Controls

Emerging Technology Risk · 4 sources · Updated 5mo ago · First seen Mar 17, 2026

AI Safety Concerns Around Copilot and ChatGPT Content Controls

Recent reporting highlights AI safety and governance risks in mainstream generative AI tools, with concerns spanning both enterprise and consumer use. Gartner warned that Microsoft 365 Copilot can amplify existing data exposure problems by making over-shared SharePoint and Microsoft 365 content easier to discover, and also flagged the risk of users distributing inaccurate or culturally inappropriate output without proper review. The guidance emphasized enabling Microsoft’s filters, tightening document permissions, and training users to validate generated content before sharing it.

Separate reporting on OpenAI’s ChatGPT described internal opposition to expanded “adult mode” capabilities, with former safety personnel reportedly warning that age-gating and content controls were not reliable enough to prevent minors from accessing prohibited material. The article also cited prior filter failures that allegedly allowed graphic erotic content outside intended policy boundaries. Both reports point to a broader governance issue: organizations and platform providers are struggling to keep content moderation, access controls, and user safeguards aligned with rapidly expanding AI functionality.

New surface, new adversariesTrack how attackers are adapting to this technology.
Track this risk
Event timeline

How this story unfolded

Events from the most recent confirmed update back to the earliest known activity.

3 events · Mar 17, 2026 – Apr 2, 2026
  1. Apr 2, 20268mo ago

    Report highlights Microsoft Copilot terms disclaim reliability and liability

    A report on Microsoft’s Copilot terms of use said the AI assistant is intended for entertainment purposes, may make mistakes, and should not be relied on for important decisions or advice. The terms also disclaim warranties over Copilot outputs, including around copyright, trademark, and privacy, shifting responsibility to users who publish or share generated content.

    Microsoft Copilot Terms of Service Label Copilot is for Entertainment Purposes Only - Cyber Security News
  2. Mar 17, 20268mo ago

    Gartner recommends mitigations for Copilot and jokes about Friday-afternoon ban

    Xu recommended enabling Microsoft’s filters, restricting access to risky data sources, limiting third-party app connections, monitoring access to restricted content, and training users to verify Copilot output. He also half-jokingly suggested banning Copilot use on Friday afternoons because users may be less likely to scrutinize AI-generated content then.

  3. Mar 17, 20268mo ago

    Gartner analyst outlines Microsoft 365 Copilot security risks in Sydney

    At Gartner’s Security & Risk Management Summit in Sydney, analyst Dennis Xu described five key security risks tied to Microsoft 365 Copilot, including amplification of oversharing in Microsoft 365, prompt injection, risky third-party SaaS integrations, remote execution via malicious prompts, and toxic but factually correct output.

Linked entities

Related entities

Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.

Open in app →

Affected products

1 linked
Microsoft 365 Copilot
Do you run this?Follow how adversaries are adapting to this technology, and where it touches your stack today.
Track This Risk
Mallory platform

See the full picture, correlated to your attack surface.

This page covers what’s public. Mallory adds the parts that aren’t: which of your assets are affected, which threat actors are using it right now, and what to deploy next.

Exposure mappingMap indicators from this story to your assets and identify affected systems in minutes.
Threat actor evidenceEvery observed campaign, victim, and pivot linked to actors named in this story.
Associated malwareMalware, exploits, and IOCs connected to the activity described here.
Detection signaturesYARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Scheduled alertsGet matching new stories delivered to your team as they break, not the next morning.
AI threadsAsk questions about this story and take action on the answers.