
Hermes Emerges as Modular Harness for Practical AI Security Testing
The article details the development of Hermes, a lightweight modular harness designed to turn raw language models into reliable AI security agents. Unlike heavy frameworks such as OpenClaw, Hermes uses dynamic skill loading, strict runbooks stored in Git, and a file-system source of truth to eliminate hallucinations and context bloat. It continuously scans dozens of sources including NVD, CISA KEV, arXiv and research blogs, then applies the custom TIPS scoring metric to rank threats before attempting reproduction. A dedicated Paper-to-PoC skill converts academic attack descriptions into executable code that is validated inside an isolated Ollama and Docker sandbox. Eight mandatory validation gates must pass before any finding is accepted, ensuring reproducibility and freshness of data. The system already supports tools such as Garak, PyRIT, promptfoo, fickling, modelscan and presidio, and evolves autonomously by committing successful attack scenarios back to its repository.
HabrAug 23, 2026Translated from Russian

