AI and agents
An AI gateway for the enterprise: controlling LLM cost, routing, quotas and guardrails in one place
An AI gateway is the single entry point between your applications and agents and every language model you use. DataNature's AI gateway routes each request to an approved model endpoint, applies budgets, quotas and rate limits per team or agent, caches repeated answers, enforces PII and safety guardrails, keeps provider credentials out of applications, and logs tokens, cost and latency for every call, all inside your jurisdiction.
Published . Last updated
On this page
Once more than one team uses language models, three problems appear at once: spend nobody can attribute, API keys copied into application code, and no consistent way to stop sensitive data reaching the wrong model. An AI gateway solves all three by putting one governed layer between every caller and every model. This guide explains what DataNature's gateway does and how to configure it.
What is an AI gateway?
An AI gateway is a service that every model call goes through. Applications and agents call one endpoint; the gateway decides which model serves the request, checks it against policy, applies limits, and records what happened. It is to language models what an API gateway is to microservices.
What does the DataNature AI gateway control?
| Capability | What it does | Why it matters |
|---|---|---|
| Model endpoints | Registers approved endpoints: self-hosted models on your GPUs, Dlytica-managed models in-country, or providers you approve | Only approved models can be called |
| Routing | Sends each request to a model by use case, twin, sensitivity or size of task | Regulated twins stay on in-country models; simple tasks use cheaper models |
| Fallbacks | Retries on a secondary endpoint when the primary is slow or unavailable | Agents keep working during a model outage |
| Budgets and quotas | Sets spend or token limits per team, twin or application, with alerts | No surprise bills; cost owned by the team that incurs it |
| Rate limits | Caps requests per minute per caller | Protects shared GPUs and provider limits |
| Caching | Returns stored answers for repeated identical requests | Lower cost and latency for common questions |
| Central credentials | Holds provider keys in the platform's secret store, never in application code | Keys can be rotated without redeploying apps |
| Guardrails | PII detection and redaction, blocked topics, prompt-injection checks, output filters | Sensitive data does not leave, and unsafe outputs are stopped |
| Usage logging | Records caller, twin, model, tokens in and out, cost, latency and outcome for every call | Chargeback, capacity planning and audit evidence |
Until an admin configures an endpoint, the gateway shows "Not configured" and no model can be called, which is a safe default.
How do you control LLM cost without slowing teams down?
Cost control works best as design, not as policing:
- Route by task. Classification, extraction and short answers rarely need the largest model. Send them to a smaller one and reserve large models for multi-step reasoning.
- Answer numbers from the semantic layer. When an agent asks a governed cube for revenue by month, the model formats an answer instead of reasoning through raw tables, which cuts tokens and errors.
- Retrieve before you generate. Grounding with private retrieval lets a mid-sized model answer accurately, so you need fewer large-model calls.
- Cache common requests, such as policy questions asked by many staff.
- Set budgets per team and twin with alerts at thresholds, and review usage logs monthly.
- Prefer self-hosted models for steady, high-volume workloads, where fixed GPU cost beats per-token pricing.
How do guardrails work at the gateway?
Guardrails run on the way in and on the way out:
- Input: detect and redact personal data before it reaches a model outside your trust boundary; reject requests matching blocked topics; flag text that looks like a prompt-injection attempt.
- Output: redact sensitive values in answers; block disallowed content; require citations for grounded answers where configured.
- Agents: Voir's tool-level guards (SQL allowlists, document partitions) complement gateway guardrails, so even a manipulated model cannot reach data outside its scope.
How does the gateway support sovereignty?
The gateway runs inside your deployment, so routing decisions, prompts, logs and keys stay in your jurisdiction. Routing makes data flows explicit: you can show a regulator exactly which twins may call which endpoints, and prove from logs that regulated twins only ever used in-country models.
How do you set it up?
- Register endpoints with their location, owner and allowed data sensitivity.
- Define routes per use case and twin, with a fallback for each.
- Set budgets, quotas and rate limits per team and twin.
- Turn on PII guardrails and choose blocked topics.
- Point Voir and AI360 at the gateway; remove direct provider keys from applications.
- Review the usage dashboard weekly in the first month, then monthly.
Frequently asked questions
Can we use both self-hosted models and providers such as Cohere?
Yes. Register both as endpoints and route by sensitivity and task. Many teams keep regulated work on in-country models and use a provider for non-sensitive tasks.
Does the gateway see our prompts?
The gateway runs inside your deployment, so prompts are processed and logged within your environment, not by Dlytica.
How is cost attributed?
Every call is logged with the calling team, twin and application, the model used and tokens consumed, so cost can be charged back accurately.
Sources
- DataNature documentation: BI, Voir AI and ML tools (AI gateway, PII guardrails)
- DataNature solution: sovereign LLM gateway
- DataNature: self-hosted LLM cost break-even
How does DataNature Atlas keep this secure and compliant?
Agents are governed like users: scoped, masked, logged and unable to act without approval. DataNature Atlas is the platform's governance module:
- Knows your data: a catalog with automated PII detection and classification across Bronze, Silver and Gold.
- Controls access: role-based grants plus attribute-based policies, row filters and column masks (null, redact, partial, hash, year only).
- Governs AI: Voir agents are scoped principals; masks, row filters and PII guardrails apply to every tool call and answer.
- Proves it: lineage from source to metric to decision, and an audit log of who or what accessed which data.
- Stays local: Atlas runs inside your deployment, so policies, lineage and logs never leave your jurisdiction.
These controls support GDPR, PIPEDA, Quebec Law 25, CCPA, HIPAA and sector rules such as OSFI E-23. Read DataNature Atlas in depth or see security and compliance.
Platform components in this article
Learn more about each DataNature product and module mentioned here.
Products
See it on your own data.
Book a demoDemos and quotes are handled by Dlytica, the company behind DataNature.