SentinelLabs Benchmark Finds Most Frontier AI Models Fail Long-Horizon Malware Analysis

SentinelLabs introduced a multi-stage benchmark to test whether frontier AI models can autonomously sustain a full malware reverse-engineering investigation, using the 2005 Fast16 Windows sabotage toolkit as the case study. Fast16 was designed to interfere with LS-DYNA engineering software and has been compared to Stuxnet because of its apparent link to efforts affecting Iran’s nuclear weapons development. The benchmark recreated an eight-stage investigation with staged evidence release, IDA Pro-based artifact generation, and verification tied to specific addresses and recovered components, including Fast16’s Lua framework and its 101-rule patching engine.

In the reported results, GPT-5.6 Sol was the only publicly available model tested that completed all eight stages, while GPT-5.5, GLM-5.2, and Opus 4.7/4.8 demonstrated useful point-in-time analysis but failed to maintain project-wide coherence. SentinelLabs said the decisive capability was project-scale recovery: retracting invalidated conclusions, tracing dependencies, and repairing the broader investigative narrative as new evidence emerged. The researchers concluded that current leading models can materially assist malware analysts and, in the strongest cases, operate as supervised investigative agents, but human reverse engineers remain necessary because the models still make semantic, technical, and quality-control errors and can declare completion prematurely.