Loading story…
Organizations are increasingly leveraging generative AI and large language models (LLMs) to accelerate software development and enhance security operations, but this adoption introduces new security risks. Experts emphasize that while AI can boost productivity and augment security teams, it can also generate code with embedded vulnerabilities, especially if models are trained on flawed or insecure codebases. To mitigate these risks, security leaders recommend a combination of human oversight, upskilling developers in secure coding practices, and implementing robust verification processes for AI-generated outputs.
Industry guidance, including a new whitepaper from AWS and SANS Institute, highlights the importance of responsible AI practices, automated reasoning to verify LLM outputs, and scaling security best practices across the AI lifecycle. Key recommendations include architecting secure generative AI solutions, balancing automation with human review, and adapting compliance strategies to address evolving regulatory and threat landscapes. As AI adoption accelerates, organizations must proactively address both the opportunities and challenges posed by AI in cybersecurity to maintain a strong security posture.
Track how attackers are adapting to this technology.
3 events from the most recent confirmed update back to the earliest known activity.
SC World published a podcast segment with James Manico on secure coding in the age of AI, emphasizing application security considerations for AI-assisted development.
Dark Reading published guidance on five critical security checkpoints requiring human oversight when using AI-developed code, reflecting growing focus on secure AI-assisted software development.
AWS released the whitepaper "AI for Security and Security for AI: Navigating Opportunities and Challenges," outlining opportunities and challenges at the intersection of AI and security.
Follow how adversaries are adapting to this technology, and where it touches your stack today.
3 references tracked. Mallory keeps watching after this page renders.
scworld.com
Open sourcedarkreading.com
Open sourceaws.amazon.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
The widespread adoption of generative AI (GenAI) tools in software development has significantly increased productivity, enabling developers to document, write, and optimize code at unprecedented speeds. According to a 2023 McKinsey study, organizations have rapidly integrated AI into their development workflows, with 83% using AI for code creation and 57% relying on AI-powered coding tools as a standard practice. However, this surge in AI-assisted coding has introduced new security risks, as traditional security models focused on perimeter or infrastructure controls do not adequately protect the data and code generated by these tools. Studies have revealed that nearly half of code snippets produced by popular AI models contain vulnerabilities, underscoring the prevalence of insecure code generation. High-profile incidents, such as Samsung's 2023 ban on ChatGPT following a sensitive code leak, highlight the real-world consequences of insufficient safeguards when using GenAI in development environments. The responsibility for securing data and code remains with developers, even as cloud providers secure the underlying infrastructure. The rapid pace of AI-generated code has outstripped the ability of traditional secure coding training to keep up, shifting the focus from training human programmers to ensuring that AI systems themselves are capable of secure coding. Industry experts note that AI is currently less effective at producing secure code than human programmers, with multiple studies and reports from sources like Schneier on Security, Veracode, and SC Media confirming this trend. The volume of vulnerabilities continues to rise, with over 47,000 publicly known vulnerabilities expected in a single year and at least 130 new vulnerabilities reported daily. This ongoing wave of vulnerabilities leads to constant exploitation and patching, further emphasizing the need for secure coding practices at the AI level. While AI has delivered substantial productivity gains—developers report 30% to 40% increases—these benefits are undermined by the security shortcomings of AI-generated code. The industry is now at a crossroads, where the imperative is to teach AI systems to code securely, rather than relying solely on human oversight or post-development security reviews. Integrating security into the AI coding process and providing developers with tools that embed data protection are seen as essential steps to address these emerging challenges. The shift towards AI-driven development necessitates a reevaluation of security strategies, focusing on proactive measures that align with the realities of modern software engineering. Without such changes, organizations risk exposing themselves to significant security threats stemming from the very tools designed to enhance their productivity.
Jun 29, 2026
The rapid adoption of AI-powered code generation tools such as GitHub Copilot, ChatGPT, and Amazon CodeWhisperer has fundamentally changed the software development landscape, introducing new security challenges for enterprise application security teams. Unlike traditional human-written code, AI-generated code often lacks clear provenance, making it difficult to verify its origin or ensure compliance with organizational security policies. This shift has led to the emergence of 'shadow code'—machine-generated code that may bypass standard security reviews and evade detection by traditional static and dynamic analysis tools, increasing the risk of invisible vulnerabilities in production systems. Generative AI models can introduce unique threats, including the creation of 'hallucinated' packages—references to non-existent or malicious libraries that may be inadvertently included in enterprise applications. Additionally, the language-based nature of large language models (LLMs) opens new attack surfaces, such as prompt injection and jailbreaking, where malicious inputs can manipulate model behavior or bypass safety constraints. As organizations accelerate the integration of AI into development workflows, application security programs must adapt to address these novel risks and build trust in the security of AI-powered software.
Jun 29, 2026
The rapid adoption of AI technologies, including large language models (LLMs) and AI coding assistants, is fundamentally transforming enterprise operations and software development. As organizations integrate AI into their systems, new security challenges emerge that differ from traditional application vulnerabilities. These include threats such as prompt injection, data poisoning, and the manipulation of semantic meaning, which can bypass conventional firewalls and security controls. Threat modeling for AI systems must account for these novel attack vectors, as adversaries exploit the way models interpret language and context rather than just code or configuration weaknesses. Simultaneously, the use of AI coding assistants is dramatically increasing developer productivity, with AI-assisted developers producing code at a much faster rate. However, this acceleration comes at a cost: the code generated with AI assistance contains significantly more security vulnerabilities, including architectural flaws that are harder to detect and remediate. Larger, multi-touch pull requests slow down code review processes and increase the likelihood of security issues slipping through due to human error or rushed reviews. The combination of increased coding velocity and the unique risks posed by AI systems underscores the urgent need for updated security practices and robust human oversight in both AI deployment and software development workflows.
Jun 29, 2026