Gemini 2.5 Flash

LangGraph Architecture Combines Hybrid RAG with YARA and Sigma Engines for Streaming Log Analysis

A new architectural pattern integrates LangGraph with a hybrid RAG system and deterministic signature engines to process large volumes of unstructured cybersecurity logs efficiently. The pipeline uses Vector for chunking logs into 250-line segments with 20-line overlap, Kafka for streaming, and an 8-node asynchronous DAG that runs AI and rule-based branches in parallel. Agent 1 groups events and generates hypotheses, while a hybrid RAG module performs query reformulation, vector plus BM25 search with 0.6/0.4 weighting, and LLM re-ranking against a translated MITRE ATT&CK knowledge base stored in ChromaDB. Parallel YARA and Sigma engines scan parsed logs using custom text-based rule implementations, with automatic YARA rule generation triggered when coverage gaps are detected. Final aggregation occurs in Agent 3, which validates findings, deduplicates confirmed incidents, and routes unconfirmed events for manual review while storing reports in PostgreSQL. Tested on a 43-minute synthetic dataset containing 38 MITRE techniques, the system achieved 85.7% precision and 78.9% recall at 3.5 lines per second using Gemini 2.5 Flash.