Loading story…
CrowdStrike and Meta have introduced CyberSOCEval, an open-source benchmark suite to evaluate AI—particularly large language models—in Security Operations Centers. The suite provides practical, adversary-driven tests for malware analysis, incident response, and threat analysis comprehension. This initiative aims to bring transparency and operational readiness to AI adoption in cybersecurity.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
1 event from the most recent confirmed update back to the earliest known activity.
Initial story creation
4 references tracked. Mallory keeps watching after this page renders.
securitysenses.com
Open sourcesecuritysenses.com
Open sourcethecyberthrone.in
Open sourcesecuritysenses.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Security operations centers (SOCs) are increasingly leveraging artificial intelligence (AI) to enhance the efficiency and accuracy of their analysts during alert investigations. A benchmark study by Dropzone involving 148 security professionals demonstrated that those using AI assistance completed investigations more quickly and with greater accuracy compared to those working manually. The AI-supported analysts maintained a consistent pace and thoroughness, even as the complexity of their tasks increased, while manual analysts showed signs of fatigue and a decline in report detail over time. The study found that AI tools helped analysts produce more comprehensive and better-documented investigations, reducing the risk of errors associated with fatigue and high alert volumes. Analysts reported positive experiences with AI, describing the tools as efficient, helpful, and time-saving, and noted that AI made their workflow smoother and less repetitive. In the realm of penetration testing, AI is being adopted as a co-pilot to amplify human creativity and focus, rather than replace ethical hackers. AI assistants, such as the Cobalt AI Assistant, are enabling pentesters to automate tedious reconnaissance tasks, such as OSINT gathering, port scanning, and subdomain enumeration, compressing hours of work into minutes. This automation allows pentesters to concentrate on developing innovative exploits and attack chains, enhancing both the speed and quality of their assessments. AI-driven reconnaissance provides pentesters with prioritized maps of potential vulnerabilities, freeing them to focus on high-value targets and strategic thinking. The integration of AI in both SOC operations and penetration testing is not just about increasing speed, but also about improving the quality and consistency of security investigations and assessments. These advancements are helping security professionals stay alert to key indicators, maintain thorough documentation, and reduce the risk of oversight. The adoption of AI tools is being met with enthusiasm by practitioners, who appreciate the reduction in repetitive tasks and the ability to focus on more complex and creative aspects of their work. As AI continues to evolve, its role in supporting human analysts and testers is expected to grow, further transforming the cybersecurity landscape. The evidence from recent studies and industry experiences underscores the tangible benefits of AI in real-world security operations and testing environments. Organizations are encouraged to consider how AI can be integrated into their existing workflows to maximize both efficiency and effectiveness. The shift towards AI-assisted security practices marks a significant step forward in the ongoing effort to defend against increasingly sophisticated cyber threats.
Jun 29, 2026
A growing ecosystem of open-source and AI-powered tools is transforming how security teams approach penetration testing, OSINT investigations, and overall security operations. Recent roundups and guides highlight over 40 free and open-source solutions that address a wide range of security needs, from vulnerability scanning and network analysis to automated patching and digital forensics. Notable tools include *Aegis Authenticator* for 2FA management, *Arkime* for large-scale packet capture, *Artemis* for modular vulnerability scanning, *Autoswagger* for API authorization testing, and *Buttercup*, an AI-driven platform for automated vulnerability detection and remediation. Additionally, specialized AI tools such as *HexStrike AI MCP Agents*, *HackerAI*, and *OSINT-GPT* are enabling more effective and autonomous penetration testing and OSINT research workflows. For digital forensics, open-source frameworks like *Autopsy*, *Kuiper*, and *DFF by Arxsys* provide analysts with robust platforms for investigation and evidence analysis. The adoption of these tools is driven by the need for flexibility, transparency, and cost-effectiveness, allowing security teams to enhance their capabilities without the burden of licensing fees. The integration of AI into these solutions further accelerates threat detection and response, positioning open-source and AI-powered tools as essential components in modern cybersecurity strategies.
Jun 29, 2026
Elastic has outlined an evidence-based framework to benchmark large language models used in agentic security operations centers. The methodology keeps the Elastic AI Agent, prompts, tools, and datasets fixed while routing runs to different models and anonymizing model identities for blind scoring. Evaluations measure whether models select the correct skills, make appropriate tool calls with valid parameters, execute tasks in the right order, and produce findings grounded in collected evidence—not merely whether their final prose appears convincing. The Agent Builder suite uses a synthetic Windows intrusion involving Chrysalis DLL side-loading, with the **EICAR** test-file hash serving as a deterministic malicious VirusTotal result and live case-management and Slack-response tools. Additional suites test Attack Discovery's ability to correlate roughly 95 alerts into eight intrusions and Automatic Migration's conversion of 52 Splunk detection rules, including macro-dependent rules, into Elastic rules. The scoring rubric evaluates both answer quality and correct tool or skill use, caps unsupported answers when required evidence-gathering calls are absent, and reports reliability independently from quality.
Sep 9, 2026