Practical Guidance on Using LLMs in Security Work and Testing LLM Applications

NVISO published a technical introduction on automating LLM red teaming to find security weaknesses in LLM-based applications, focusing on AI-specific risks such as prompt injection, data leakage, jailbreaking, and other behaviors that can bypass guardrails. The post describes why manual testing is difficult due to LLMs’ probabilistic behavior and demonstrates using the promptfoo CLI to scale testing against a deliberately vulnerable ChainLit application, positioning automated test harnesses as a way to systematically probe LLM apps for exploitable failure modes.

Separately, a practitioner write-up describes how security analysts and engineers are using general-purpose LLM tools (Claude, Cursor, ChatGPT) to accelerate day-to-day security work through better prompting patterns rather than “keyword searching.” It provides practical prompting techniques (e.g., “role-stacking” and supplying richer context like requirements docs or code repositories) and includes an example of using an LLM to help design a small Flask application for collecting OSINT (DNS, WHOIS/RDAP, HTML) for URL investigations—guidance that is adjacent to, but not the same as, automated red-teaming of LLM applications.