llama.cpp

Local LLMs in AppSec: What Open Tools and Models Can Handle in Application Security

The article examines how local large language models can be integrated into application security workflows without relying on cloud subscriptions. It traces the evolution from llama.cpp in 2023 through agentic frameworks like PentAGI and Strix in 2025-2026. Teams can now run models on local hardware, connect them to code analyzers such as Semgrep, and delegate specific tasks like vulnerability path reconstruction or multi-step application testing. The piece highlights practical scenarios including static code review, dynamic testing with browser and terminal agents, and security evaluation of embedded AI assistants using tools like Garak and PyRIT. It stresses the importance of task decomposition, context management, and verification of model outputs to avoid misleading results. Hardware performance, model selection, and custom agent configurations are presented as key factors for successful adoption.

Former OpenAI CTO Mira Murati Launches Thinking Machines' Inkling: Open-Weights Multimodal AI Model with 975 Billion Parameters and Self-Training Demo

Thinking Machines, founded by former OpenAI technical director Mira Murati, has released Inkling, its first open-weights multimodal AI model that supports text, images, and audio in a unified architecture. The model uses a mixture-of-experts design with 975 billion total parameters but activates only 41 billion at once, supports a 1-million-token context window, and was trained on 45 trillion tokens spanning text, images, audio, and video. A smaller Inkling Small variant with 12 billion active parameters was also introduced for faster and cheaper inference. Key innovations include adjustable reasoning depth that lets developers control compute usage per query and a self-training experiment where the model autonomously fine-tuned itself via the Tinker platform to avoid using one letter of the English alphabet. Weights are now available on Hugging Face with support for Transformers, vLLM, SGLang, and llama.cpp, positioning Inkling as a flexible foundation for further customization rather than a direct competitor to closed frontier models.