A technical write-up demonstrated a compact decoder-only transformer for malware analysis that learns the byte-level structure of legitimate ELF binaries and uses perplexity to flag anomalous regions associated with packing or encryption. The approach trains a roughly 600K-parameter model in PyTorch on binaries from /usr/bin, applying transformer concepts introduced in Attention Is All You Need to binary analysis rather than natural language tasks.
The author reported that the model trained in about 25 seconds on an NVIDIA L20, reduced validation perplexity to roughly 14, and achieved a ROC-AUC of 0.994 when separating normal binaries from simulated packed samples across 80 held-out files. The article said the method outperformed simple Shannon entropy because it captures byte order and code structure, while cautioning that legitimate compressed resources can still trigger false positives and that the signal should be combined with section context and YARA rules for practical detection.

Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
2 events from the most recent confirmed update back to the earliest known activity.
A blog post described an educational malware-analysis method using a small byte-level GPT model in PyTorch to detect packed or encrypted binaries via perplexity. The post reported a roughly 600K-parameter decoder-only transformer trained on legitimate ELF binaries and evaluated against simulated packed variants.
The foundational transformer paper 'Attention Is All You Need' was published on arXiv, introducing the architecture referenced by later GPT-style binary-analysis work.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
4 references tracked. Mallory keeps watching after this page renders.
malware.news
Open sourcecocomelonc.github.io
Open sourcepytorch.org
Open sourcearxiv.org
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.