deepset Unveils Self-Improving Sovereign AI Agents With NVIDIA

BERLIN, October 21, 2026 — deepset, the German company behind the open-source Haystack framework, made a two-part push for sovereign AI agents at NVIDIA GTC Berlin: an open reference architecture for self-improving agents that runs on infrastructure organizations control, and new agent-optimization and custom-model services aimed at cutting the cost of running high-quality agents in government, defense, and industry. The announcements, published October 21, tie together the two pressures reshaping enterprise agent deployments: the need for agents that keep getting better in production, and the need to do it without losing control of data, models, or spend.

Two agents, one feedback loop

The centerpiece of the first announcement is a reference architecture for sovereign AI agents developed with NVIDIA open models and software, designed for locally controlled and air-gapped environments, according to deepset's October 21 announcement. Instead of the usual deploy-and-forget pattern, two Haystack agents form a continuous feedback loop. The first performs the domain task — in the GTC Berlin demonstration, a cybersecurity agent analyzes confidential source code and vulnerability information inside a locally controlled environment, using open models rather than sending sensitive code to external APIs. The second agent studies traces and results from those runs and proposes changes to the agent harness. Every proposed change is evaluated against predefined objectives and the current version before it can be promoted, with approval, audit, and rollback controls keeping people in charge of what reaches production.

The architecture rests on four building blocks, according to the announcement: open models for reasoning, a modular orchestration layer connecting models to an organization's data and tools, an optimization environment where improvements can be evaluated, and security controls governing what agents are allowed to do. Haystack supplies the open-source orchestration layer and agent harness, while NVIDIA contributes Nemotron open models served through NIM microservices, NeMo Gym as the optimization environment, and NemoClaw open blueprints that run the agent inside NVIDIA OpenShell, a secure runtime enforcing policy-based controls over files, networks, credentials, and tools. Open models can run on locally controlled NVIDIA-accelerated infrastructure, including DGX Spark, and a companion repository on deepset's GitHub page provides example code so developers can explore and adapt the architecture themselves.

deepset's chief executive and co-founder said powerful open models create the foundation for sovereign AI, but realizing their full potential requires engineering the complete system around them — combining NVIDIA's open models and optimization technologies with Haystack's orchestration so organizations can build specialized agents that evolve with their environment while operating within governed infrastructure under human oversight. NVIDIA's vice president of strategic partnerships added that organizations need agents that work effectively while retaining control over their data, infrastructure, and operations, according to the announcement. The two companies will present a session titled "Evolve Within Bounds: Secure, Self-Improving Agents for On-Prem AI" at GTC Berlin on October 22 at 10:00 a.m. CEST, with deepset at booth 3071.

Cutting the cost per task

The second announcement addresses the economics of running sovereign AI agents at scale. The Haystack Enterprise Platform now shows AI spend by model, agent, service, and team — both in centralized reporting and alongside production traces — so teams can find expensive or unreliable steps, according to the release. deepset's position is that AI spend cannot be controlled by looking at models alone; organizations must optimize model choice and the agent harness together, testing changes against cost and quality requirements while keeping control of models, data, and infrastructure.

Through custom model services, deepset engineers can assess and tailor NVIDIA Nemotron models to customer tasks, and the platform uses NVIDIA Triton Inference Server for shared, GPU-accelerated embedding and reranking. Shared permissions, policies, and usage controls govern how agents access models, tools, and data, so teams can improve AI economics within their security and regulatory requirements without tying applications to a single model provider or deployment environment.

The announcement builds on deepset's existing deployments: the European Commission's self-hosted AI@EC platform for governed AI applications, Germany's Federal Ministry of Research, Technology and Space, which uses an AI assistant for project-funding guidance, and Bosch, whose DRIVE Companion supports SAP migration teams. deepset's chief executive said the goal is to complete each task at the lowest cost that meets quality and security requirements — sometimes a smaller, specialized model, sometimes an improved harness around a larger one — with both measured against results and the system kept under sovereign control.

Sovereignty week for European public-sector AI

The GTC Berlin announcements land in the middle of a busy run of sovereign-AI moves for deepset. On October 15, the company made the Haystack Enterprise Platform available on STACKIT, the sovereign European cloud, letting public-sector organizations run AI agents in production from pilot through regular operations. Earlier partnerships include Celonis combining process intelligence with deepset technology into a sovereign AI platform for military, cybersecurity, and policing applications, and NTT DATA's sovereign AI transformation model whose first deployment is AI@EC, the European Commission's sovereign platform.

For governments and regulated organizations, sovereignty requires more than running a model locally, according to the announcement. Organizations also need control over what data and context an agent can access, which models and tools it uses, and the infrastructure on which the complete system operates. The reference architecture is designed to preserve that control across the stack, and Haystack's modular design lets organizations adapt individual parts of the system as requirements change, reducing dependence on a single model provider or closed application stack.

The message for agent operators is that the industry's agent story is maturing past capability demos: agents that improve themselves within bounds, spend that can be traced to the step level, and infrastructure that stays inside the organization's perimeter. deepset is betting that the next wave of enterprise agent adoption will be won on governance and economics rather than raw model performance.