AI and agents

An AI gateway for the enterprise: controlling LLM cost, routing, quotas and guardrails in one place

An AI gateway is the single entry point between your applications and agents and every language model you use. DataNature's AI gateway routes each request to an approved model endpoint, applies budgets, quotas and rate limits per team or agent, caches repeated answers, enforces PII and safety guardrails, keeps provider credentials out of applications, and logs tokens, cost and latency for every call, all inside your jurisdiction.

Published . Last updated

On this page

Once more than one team uses language models, three problems appear at once: spend nobody can attribute, API keys copied into application code, and no consistent way to stop sensitive data reaching the wrong model. An AI gateway solves all three by putting one governed layer between every caller and every model. This guide explains what DataNature's gateway does and how to configure it.

What is an AI gateway?

An AI gateway is a service that every model call goes through. Applications and agents call one endpoint; the gateway decides which model serves the request, checks it against policy, applies limits, and records what happened. It is to language models what an API gateway is to microservices.

What does the DataNature AI gateway control?

CapabilityWhat it doesWhy it matters
Model endpointsRegisters approved endpoints: self-hosted models on your GPUs, Dlytica-managed models in-country, or providers you approveOnly approved models can be called
RoutingSends each request to a model by use case, twin, sensitivity or size of taskRegulated twins stay on in-country models; simple tasks use cheaper models
FallbacksRetries on a secondary endpoint when the primary is slow or unavailableAgents keep working during a model outage
Budgets and quotasSets spend or token limits per team, twin or application, with alertsNo surprise bills; cost owned by the team that incurs it
Rate limitsCaps requests per minute per callerProtects shared GPUs and provider limits
CachingReturns stored answers for repeated identical requestsLower cost and latency for common questions
Central credentialsHolds provider keys in the platform's secret store, never in application codeKeys can be rotated without redeploying apps
GuardrailsPII detection and redaction, blocked topics, prompt-injection checks, output filtersSensitive data does not leave, and unsafe outputs are stopped
Usage loggingRecords caller, twin, model, tokens in and out, cost, latency and outcome for every callChargeback, capacity planning and audit evidence

Until an admin configures an endpoint, the gateway shows "Not configured" and no model can be called, which is a safe default.

How do you control LLM cost without slowing teams down?

Cost control works best as design, not as policing:

  1. Route by task. Classification, extraction and short answers rarely need the largest model. Send them to a smaller one and reserve large models for multi-step reasoning.
  2. Answer numbers from the semantic layer. When an agent asks a governed cube for revenue by month, the model formats an answer instead of reasoning through raw tables, which cuts tokens and errors.
  3. Retrieve before you generate. Grounding with private retrieval lets a mid-sized model answer accurately, so you need fewer large-model calls.
  4. Cache common requests, such as policy questions asked by many staff.
  5. Set budgets per team and twin with alerts at thresholds, and review usage logs monthly.
  6. Prefer self-hosted models for steady, high-volume workloads, where fixed GPU cost beats per-token pricing.

How do guardrails work at the gateway?

Guardrails run on the way in and on the way out:

  • Input: detect and redact personal data before it reaches a model outside your trust boundary; reject requests matching blocked topics; flag text that looks like a prompt-injection attempt.
  • Output: redact sensitive values in answers; block disallowed content; require citations for grounded answers where configured.
  • Agents: Voir's tool-level guards (SQL allowlists, document partitions) complement gateway guardrails, so even a manipulated model cannot reach data outside its scope.

How does the gateway support sovereignty?

The gateway runs inside your deployment, so routing decisions, prompts, logs and keys stay in your jurisdiction. Routing makes data flows explicit: you can show a regulator exactly which twins may call which endpoints, and prove from logs that regulated twins only ever used in-country models.

How do you set it up?

  1. Register endpoints with their location, owner and allowed data sensitivity.
  2. Define routes per use case and twin, with a fallback for each.
  3. Set budgets, quotas and rate limits per team and twin.
  4. Turn on PII guardrails and choose blocked topics.
  5. Point Voir and AI360 at the gateway; remove direct provider keys from applications.
  6. Review the usage dashboard weekly in the first month, then monthly.

Frequently asked questions

Can we use both self-hosted models and providers such as Cohere?

Yes. Register both as endpoints and route by sensitivity and task. Many teams keep regulated work on in-country models and use a provider for non-sensitive tasks.

Does the gateway see our prompts?

The gateway runs inside your deployment, so prompts are processed and logged within your environment, not by Dlytica.

How is cost attributed?

Every call is logged with the calling team, twin and application, the model used and tokens consumed, so cost can be charged back accurately.

Sources

How does DataNature Atlas keep this secure and compliant?

Agents are governed like users: scoped, masked, logged and unable to act without approval. DataNature Atlas is the platform's governance module:

  • Knows your data: a catalog with automated PII detection and classification across Bronze, Silver and Gold.
  • Controls access: role-based grants plus attribute-based policies, row filters and column masks (null, redact, partial, hash, year only).
  • Governs AI: Voir agents are scoped principals; masks, row filters and PII guardrails apply to every tool call and answer.
  • Proves it: lineage from source to metric to decision, and an audit log of who or what accessed which data.
  • Stays local: Atlas runs inside your deployment, so policies, lineage and logs never leave your jurisdiction.

These controls support GDPR, PIPEDA, Quebec Law 25, CCPA, HIPAA and sector rules such as OSFI E-23. Read DataNature Atlas in depth or see security and compliance.

See it on your own data.

Book a demo

Demos and quotes are handled by Dlytica, the company behind DataNature.