LLM Gateways & Routers
Multi-provider gateways / routers / aggregators (one API key for many models). Compare them on what matters for European deployments: can the actual inference be kept in the EU, can the gateway run locally / self-hosted, and is it open source. Be skeptical of “GDPR-compliant” badges — most providers' “EU” means EU-resident logs/metadata, not EU routing of the model inference. The EU routing column captures the real capability.
35 of 35
| Gateway | Type | EU routing | Self-host / local | Open source | HQ | Pricing |
|---|---|---|---|---|---|---|
| Cortecs ↗ Sovereignty-first EU router; 150+ EU-hosted endpoints. | Router | EU-native 'Europe's LLM router' — routes only to models hosted within the EU; guarantees data never leaves Europe, incl. for proprietary models (via EU-resident deployment). Genuine EU inference routing. | Partial (dedicated EU instances) | No | Austria (Vienna) | Usage + flat 5% service fee (no base-price markup) |
| EUrouter ↗ Closed-source but small/new; EU residency is the core product. GitHub | Router | EU-native api.eurouter.ai — routes actual inference through EU/EEA providers by default; optional country pinning (DE/FR), eu_owned flag, ZDR. The most EU-native router. | No | No (helpers MIT) | Netherlands | Free 15% / Plus €39 9% / Pro €99 3% markup |
| Orq.ai ↗ Full GenAI platform; SOC2/GDPR/EU-AI-Act; 400+ models / 28+ providers. | Router (GenAI platform) | EU-native EU-hosted (Amsterdam); AI Router has a Location filter — set to Europe to restrict the pool to EU-hosted models. Genuine EU inference routing when set. | Yes (enterprise VPC/on-prem) | No | Netherlands | Free dev (50k spans); paid + enterprise |
| Eden AI ↗ Most mature EU-HQ option; multimodal (500+ models incl. OCR/speech/vision). GitHub | Router (multimodal) | EU option French company; dedicated EU endpoint + DPA + ZDR keeps data in Europe. Firmer on data residency than a hard EU-only-inference guarantee for all models. | No | No | France | Passthrough + 5.5% platform fee |
| IBM watsonx.ai ↗ Full enterprise studio; real EU regions + on-prem. 2026-07-22 re-check blocked (ibm.com 403) — values from 2026-06. GitHub | Enterprise platform | EU option Genuine EU inference regions: Frankfurt (eu-de) and London (eu-gb) endpoints. Built-in OpenAI-compatible model gateway to Granite + third-party providers (preview). | Yes (OpenShift/CP4D) | No (Granite models Apache-2.0) | US | Usage-based (Resource Units) + on-prem licensing |
| Kilo Code ↗ Coding agent + gateway; EU via provider selection. GitHub | Router (coding agent) | EU option No first-party EU endpoint; EU via official Inceptron partnership (kilo.ai/eu, EU infra) or third-party EUrouter. Single global gateway otherwise. | No (extension only) | Partial (extension MIT) | Unknown | Passthrough 0% markup; Kilo Pass from $19/mo |
| OpenRouter ↗ Largest model market; strongest true-EU-inference guarantee but enterprise-gated. | Router | EU option eu.openrouter.ai (enterprise) keeps prompts+completions entirely in the EU, EU-eligible providers only. Standard tier has no EU residency. All tiers get ZDR + provider routing controls. | No | No | US | Passthrough + 5.5% credit fee; BYOK |
| Pydantic AI Gateway ↗ Now lives inside Pydantic Logfire; OSS standalone repo archived. GitHub | Router | EU option gateway-eu.pydantic.dev EU proxy edge + Logfire EU region for logs/telemetry. EU edge routing yes; upstream model still runs where the provider hosts it. | Partial | AGPL-3.0 (repo archived) | UK | BYOK free; 5% markup (Personal/Team) / 3% (Growth) on managed inference |
| Requesty ↗ Dedicated EU subdomain endpoint; 600+ models. | Router | EU option router.eu.requesty.ai (Frankfurt, AWS eu-central-1): its own processing/logs stay in EU; inference stays in EU only if you pick EU model deployments. | No | No | UK | 5% markup; free 200 req/day |
| TrueFoundry ↗ Strong region-locked data-residency story; proprietary. GitHub | Enterprise gateway | EU option Regional gateway data planes deployable in EU; region-locked routing fails closed outside EU for both inference and logs (depends on configured EU backend). | Yes (K8s/Helm, enterprise) | No | US/India | Developer $0 / Pro $499 / Pro Plus $2999 / Enterprise |
| Arch / Plano (Katanemo) ↗ AI-native Envoy edge proxy; semantic routing + guardrails (renamed Arch→Plano). GitHub | Self-host gateway | Self-host in EU Self-host the Envoy-based proxy in EU; no managed EU endpoint. | Yes (Docker/CLI) | Apache-2.0 | US | OSS free |
| Bifrost (Maxim) ↗ Fast Go gateway; ~µs overhead; 1000+ models. GitHub | Self-host gateway | Self-host in EU Self-host in an EU region/VPC for residency; no Maxim-operated EU inference endpoint. | Yes (npx/Docker/Go/Helm) | Apache-2.0 | US | OSS free; Enterprise custom |
| Envoy AI Gateway ↗ CNCF project (Tetrate+Bloomberg); production-ready v1.0; 16 providers. GitHub | Self-host gateway | Self-host in EU Kubernetes-native; run your cluster in the EU. No managed EU endpoint. | Yes (K8s/Helm) | Apache-2.0 | US / CNCF | OSS free; Tetrate support |
| GPTRouter ↗ Appears abandoned (no commits since Apr 2024). GitHub | Self-host gateway | Self-host in EU Self-host (Docker) in EU; ~6 providers. | Yes (Docker) | MIT (inactive since 2024) | US/India | OSS free |
| GPUStack ↗ GPU-cluster + model-serving orchestrator (not a multi-provider router). GitHub | Self-host serving | Self-host in EU You run it on your own GPUs/servers (EU if you choose). Serves your own models, not a third-party router. | Yes (Docker/pip/Helm) | Apache-2.0 | China (Seal) | OSS free |
| Helicone ↗ Observability-first; lightweight Rust router; standalone ai-gateway repo idle since 2025-11 (obs core active). GitHub | Self-host gateway | Self-host in EU Managed offers US/EU data region for logs/storage only (not inference egress). Self-host the Rust gateway in EU for full control. | Yes (npx/Docker) | GPL-3.0 (gateway); Apache-2.0 (obs core) | US | OSS free; SaaS Pro $79 / Team $799 |
| Kong AI Gateway ↗ AI plugins on Kong; semantic routing/caching/guardrails. GitHub | Self-host gateway | Self-host in EU Runs entirely in your own infra; residency = where you deploy + which provider region you point at. No Kong-hosted inference. | Yes (Docker/Helm) | Apache-2.0 (some AI plugins enterprise) | US | OSS free; Konnect/Enterprise paid |
| LangDB ↗ Fast Rust gateway; OSS repo transferred to vllora/vllora and repositioned as 'Debug your AI agents' (ELv2). langdb.ai pricing unverifiable 2026-07-22 (403) — pricing from 2026-06. GitHub | Self-host gateway | Self-host in EU No EU endpoint advertised; self-host the Rust gateway on your own EU infra. | Yes (cargo/brew/source) | ELv2 (source-available) | Singapore | OSS free + SaaS/enterprise |
| LiteLLM ↗ Most-adopted OSS proxy; 100+ providers; region/load-balancing/guardrails. GitHub | Self-host gateway | Self-host in EU Self-host anywhere; region_name='eu' routing pins inference to EU-tagged provider deployments. No managed EU SaaS endpoint. | Yes (pip/Docker/Helm) | MIT (core) | US | OSS free; Enterprise paid |
| llmgateway.io (theopenco) ↗ Newer OSS alternative to Eden AI / Vercel / Cloudflare gateways. GitHub | Self-host gateway | Self-host in EU OSS unified-API gateway; deploy on your own EU infra. Hosted option exists. | Yes | AGPL-3.0 | Unknown | BYOK free; hosted credits +5% flat fee; self-host free (AGPL-3.0); Enterprise custom |
| New API / One API family ↗ one-api lineage (one-api→new-api/one-hub→done-hub); key-mgmt gateways, not semantic routers; homepage now newapi.ai. GitHub | Self-host gateway | Self-host in EU Self-host (Docker/binary) anywhere incl. EU; key-management + redistribution. Inference egress per configured channel. | Yes (Docker/binary) | AGPL-3.0 (New API) / MIT (One API) | China | OSS free (built-in resale/billing) |
| OptiLLM ↗ Test-time accuracy optimizer (CoT/MoA/MCTS), not a billing gateway. GitHub | Library (optimizing proxy) | Self-host in EU Self-hosted OpenAI-compatible proxy; residency depends on your deploy + backend. | Yes (pip/Docker) | Apache-2.0 | Singapore | OSS free |
| Portkey ↗ Runs locally; 1,600+ models / 250+ providers; 50+ guardrails; Gateway 2.0 merging enterprise gateway into OSS. GitHub | Self-host gateway | Self-host in EU OSS gateway self-hostable in your EU VPC; managed regional residency (US/EU/India) is enterprise (control plane stays global). Route to EU providers. | Yes (npx/Docker/K8s) | MIT | US (acq. Palo Alto Networks 2026) | OSS free; SaaS Developer free (10k logs/mo) / Production $49/mo (100k logs) / Enterprise |
| RouteLLM ↗ Strong/weak two-model cost router; ~85% cost cut research; repo inactive since Aug 2024. GitHub | Library (router) | Self-host in EU Python library; runs wherever you host it, dispatches via LiteLLM to your chosen (EU) provider. | Yes (pip) | Apache-2.0 | US (LMSYS) | OSS free |
| Braintrust ↗ Eval/observability-first; auto-logs every call as a trace. GitHub | Self-host gateway | Logs only EU data-plane residency for eval/observability logs on Enterprise; not inference routing. OSS proxy self-deployable to edge/Lambda. | Yes (OSS proxy) | MIT | US | Passthrough BYOK; SaaS for evals (Pro $249) |
| Cloudflare AI Gateway ↗ Easiest to adopt; weakest EU-residency story. | Router (proxy) | Logs only Global edge proxy; incompatible with Cloudflare's own Regional Services. Only log metadata can be EU-pinned; inference depends on upstream provider. | No | No | US | Free core + 5% unified billing |
| AIMLAPI ↗ 1000+ models reseller; EU relevance is corporate only. GitHub | Aggregator | EU entity only EU-domiciled (Estonia) but makes no EU-routing/residency claim; single global endpoint. | No | No | Estonia | Pay-as-you-go (prepaid, $20 min) |
| AnannasAI ↗ Small aggregator (~500 models); fallback + observability + BYOK. Site unreachable (HTTP 522) at 2026-07-22 re-check — values from 2026-06. GitHub | Aggregator | No EU No EU residency claim. | No | No (Pipecat integration BSD-2) | Unknown | 5% markup (0% BYOK first 1M req) |
| APIpie ↗ 700+ LLMs + integrated web search & chat memory; failover. GitHub | Aggregator | No EU No EU residency claim. | No | No | Unknown | Usage-based |
| Glama ↗ Gateway tied to an MCP server registry/inspector. GitHub | Router + MCP registry | No EU 'Globally distributed', no EU residency claim. OSS 'lightport' core self-hostable in EU. | Partial (lightport OSS) | Partial (lightport) | US | Subscription ($9/$26/$80) + passthrough |
| Keywords AI (Respan) ↗ Gateway + observability + prompt mgmt + evals (rebranded Keywords AI→Respan). | Router + observability | No EU Compliance certs (GDPR/SOC2/ISO) but no EU region endpoint for inference; self-host is enterprise-only. | Enterprise only | No | US | Free / Team $199 / Enterprise |
| Martian ↗ Pioneer quality/cost per-prompt 'model router'. 2026-07-22: homepage repositioned as AI-interpretability research lab; pricing page 404 and route.withmartian.com unreachable — pricing/self-host values stale (from 2026-06). GitHub | Router (semantic) | No EU No managed EU residency; enterprise VPC deployment could run in an EU cloud region. | Enterprise only | No | US | Free 2.5k req; usage + enterprise |
| nano-gpt ↗ Privacy/crypto pay-per-prompt aggregator; huge multimodal catalog; 5% surcharge for provider pinning and for BYOK. | Aggregator (privacy) | No EU No EU residency; privacy via TEE 'Private Mode' + crypto/no-account, not residency. | No | No (community clients OSS) | Unknown | Passthrough (0% markup) + $8/mo OSS-models sub; crypto |
| Unify ↗ Former benchmark-driven router — no longer a gateway product. GitHub | Discontinued router | No EU Router product shut down; pivoted to an AI-agent ('droids') SaaS. | No | No (SDKs MIT/Apache) | UK | Agent SaaS (Team $75/mo) |
| Vercel AI Gateway ↗ Zero markup, strong fallbacks/BYOK; no EU residency. GitHub | Router | No EU Single global endpoint ai-gateway.vercel.sh; no EU region routing (open feature request). ZDR = retention, not residency. | No (SDK OSS) | No (AI SDK Apache-2.0) | US | Passthrough, 0% markup (incl. BYOK); free tier (model subset, rate-limited); paid add-ons (team ZDR / provider allowlist $0.10/1k req, custom reporting) |
LLM gateways / routers / aggregators (one API for many models). 'EU' = can the request actually be kept in the EU: EU-native (inference routed in-EU by design) > EU option (EU endpoint/region available, often enterprise; inference EU only if you pick EU models) > Self-host in EU (OSS — run it on your own EU infra) > Logs-only (only logs/metadata are EU-resident) > No. Most are US-HQ (US CLOUD Act applies regardless of server location). Researched 2026-06 from each product's own docs/pricing; re-verified 2026-07-22 against the same first-party sources (fields that could not be re-confirmed keep the 2026-06 values and are flagged in notes).
Collected 2026-07-22. EU rank: EU-native (inference in-EU by design) › EU option (EU endpoint/region, often enterprise) › Self-host in EU (OSS, run it yourself) › Logs only › No.