# Lakehouse architecture

> How to design, run and pay for a lakehouse: Iceberg and Delta, medallion layers, catalogs, migration, cost control and platform comparisons. Practical guides for data engineers and architects choosing or building a platform.

Source: https://datanature.ai/blog/category/lakehouse-architecture
Publisher: Dlytica Inc. (DataNature is a Dlytica company)
Updated: 2026-11-03

- [Website and app event tracking for a sovereign CDP: SDKs, identity stitching and real-time profiles](https://datanature.ai/blog/website-app-event-tracking-sdk-cdp-identity-stitching.md): AI360 captures behavioural signals from websites, mobile apps and servers with first-party event SDKs and an HTTP API, stitches anonymous and known activity into one customer profile, transforms events in flight, and lands them in the governed lakehouse within seconds through StreamHouse. Because collection runs on your own domain and platform, the clickstream stays in your jurisdiction instead of flowing to a foreign tag vendor.
- [StreamHouse: the real-time hot path on a sovereign lakehouse, with architecture and use cases](https://datanature.ai/blog/streamhouse-real-time-hot-path-lakehouse.md): StreamHouse is DataNature's real-time hot path. It captures inserts, updates and deletes from source database logs and event streams, runs them through managed Apache Kafka with a schema registry and in-flight processing, and writes them to Apache Iceberg with exactly-once, ordered, de-duplicated delivery. Fraud models, alerts and live dashboards act on data seconds old, in the same tables that hold history.
- [Oracle, SQL Server, MySQL and Postgres to Apache Iceberg: a practical ingestion guide](https://datanature.ai/blog/oracle-sql-server-to-iceberg-ingestion-guide.md): DataNature Connect extracts tables from Oracle, SQL Server, MySQL and PostgreSQL over JDBC and writes them as Apache Iceberg tables in a Bronze schema. Each table uses one of four load types: full, incremental on a watermark, full then incremental, or a one-off historical range. Good results depend on choosing the right bookmark column, partitioning only large tables, and reconciling row counts after backfills.
- [Telematics and IoT data for insurers: usage-based insurance, loss prevention and claims on a streaming lakehouse](https://datanature.ai/blog/insurance-telematics-iot-usage-based-insurance-streaming.md): Telematics and IoT devices produce continuous data about driving, buildings and assets. On DataNature, StreamHouse ingests device events in real time into Iceberg tables, features such as harsh braking, night driving, water leaks or temperature alerts are computed continuously, and AI360 Studio uses them for usage-based pricing, loss prevention alerts and faster claims. DataNature Atlas governs location and behaviour data, which is highly sensitive, and keeps it in-country.
- [Demand, capacity and cash forecasting on DataNature: build forecasts that teams trust](https://datanature.ai/blog/demand-forecasting-on-datanature-lakehouse.md): DataNature supports forecasting from data to decisions. Historical sales, usage, load or cash flows land in Iceberg tables; dbt builds clean time series with calendars, prices and events; Spark trains many forecasts in parallel with experiment tracking; the model registry versions them; scheduled jobs write forecasts and prediction intervals to Gold tables; and dashboards, AI360 Studio and Voir put them in front of planners. Forecast accuracy is tracked against actuals so models improve over time.
- [dbt on Spark and Trino with Apache Iceberg: building medallion pipelines that stay fast and correct](https://datanature.ai/blog/dbt-spark-trino-iceberg-medallion-pipelines.md): DataNature runs dbt against three targets: Spark 4 Connect for heavy transformations into Iceberg, Trino for lighter SQL and federated sources, and DN Warehouse for low-latency serving tables. Projects live in Git, follow a staging, silver, intermediate and marts layout, de-duplicate in Silver, use incremental models for large facts, and commit packages so scheduled jobs run offline.
- [DataNature architecture, technically: the open engines behind a sovereign AI lakehouse](https://datanature.ai/blog/datanature-architecture-technical-deep-dive.md): DataNature is a Kubernetes-native lakehouse built on open engines. Data lands in Apache Iceberg tables on object storage, Spark 4 and dbt refine it from Bronze to Gold, Trino and DN Warehouse serve SQL, StreamHouse runs the real-time hot path, and Atlas applies access policies, masking, lineage and audit across every engine. AI runs on the same governed data through experiment tracking, a model registry, a vector store and Voir agents.
- [Databricks vs Snowflake vs DataNature for sovereign and private AI: a technical comparison](https://datanature.ai/blog/databricks-vs-snowflake-vs-datanature-sovereign-private-ai.md): Databricks and Snowflake are strong managed platforms that run in public clouds with vendor-operated control planes. DataNature offers similar building blocks (open lakehouse tables, Spark and SQL engines, governance, a semantic layer, ML and agents) and can run the entire stack, control plane included, in-country, on-premises or air-gapped. Choose by where processing must happen, not by feature count.
- [Building a customer data platform on a sovereign lakehouse: the AI360 architecture, step by step](https://datanature.ai/blog/customer-data-platform-architecture-ai360-lakehouse.md): A lakehouse-native CDP builds the customer profile where the data already lives. In DataNature, Connect and StreamHouse land customer data in Bronze, dbt resolves identities in Silver, AI360 Studio builds Gold profiles, features, segments and scores, and reverse ETL, Lakebase and Voir activate them. DataNature Atlas enforces consent, purpose and masking throughout, all inside your jurisdiction.
- [Can Snowflake run on-premise or air-gapped? The answer, and what to use instead](https://datanature.ai/blog/can-snowflake-run-on-premise.md): No. Snowflake runs only in Snowflake-operated accounts on AWS, Azure and Google Cloud. There is no on-premises, bare-metal or air-gapped edition, and Virtual Private Snowflake is dedicated but still operated by Snowflake. Snowflake can query external tables on S3-compatible storage you run, but compute, metadata and results stay in Snowflake's cloud.
- [Can Databricks run on-premise? What is possible, what is not, and the sovereign alternative](https://datanature.ai/blog/can-databricks-run-on-premise.md): No. Databricks has no supported on-premises or air-gapped deployment. The control plane is always operated by Databricks in its cloud, and even classic compute runs in your public cloud account, not your data centre. Teams that need the lakehouse inside their own walls run open engines themselves, or use a platform such as DataNature that deploys the full stack on-premises.
- [Top 5 lakehouse platforms for 2026, ranked for sovereignty](https://datanature.ai/blog/top-5-lakehouse-platforms-2026.md): The top lakehouse platforms in 2026 are DataNature, Snowflake, Databricks, Cloudera and Dremio. All five now support open table formats such as Apache Iceberg. They differ most on where they can run and who operates them. Databricks and Snowflake lead on ecosystem and scale in public cloud. DataNature, Cloudera and Dremio can run in your own country, data centre or national cloud, which matters for regulated banks, telcos and governments.
- [Apache Iceberg vs Delta Lake: which open table format should you choose?](https://datanature.ai/blog/iceberg-vs-delta-lake.md): Apache Iceberg and Delta Lake are both open table formats that add ACID transactions, schema evolution and time travel to files in object storage. Iceberg has the broadest engine support across vendors. Delta Lake is the default in Databricks. DataNature supports both natively, so you can choose per table without being locked in.
- [Why data platform bills surprise teams, and how to keep lakehouse costs predictable](https://datanature.ai/blog/lakehouse-cost-surprises-and-how-to-avoid-them.md): Lakehouse bills surprise teams for the same few reasons: clusters left running, premium serverless rates, platform fees layered on top of cloud compute, unbounded ingestion and AI usage, and estimates that do not match real workloads. Predictable costs come from transparent unit pricing, auto-stop, per-team budgets and alerts, and right-sized compute.
- [Lock-in moved to the catalog: how to keep your data platform portable](https://datanature.ai/blog/avoid-data-platform-lock-in-catalog-layer.md): Open table formats such as Delta Lake and Apache Iceberg made data files portable, but lock-in moved up a layer: to the catalog, governance policies, pipelines and semantic models. To stay portable, use an open REST catalog protocol, keep policies and lineage exportable, write pipelines in open tools such as dbt and Spark, and keep a tested exit plan.
- [Medallion architecture explained: Bronze, Silver and Gold](https://datanature.ai/blog/medallion-architecture-explained.md): Medallion architecture organizes lakehouse data into Bronze (raw, immutable copies of sources), Silver (cleaned, standardized and validated) and Gold (certified, business-ready tables). It makes pipelines auditable and reprocessable, and on DataNature it runs on open Delta or Iceberg tables with lineage in Atlas.
- [Databricks alternatives for regulated, data-resident workloads](https://datanature.ai/blog/databricks-alternatives-for-data-residency.md): For workloads that must stay in-country or on-premises, the main Databricks alternatives are self-managed open-source stacks (Spark, Trino, Iceberg), cloud-native warehouses in local regions, and sovereign platforms such as DataNature that package the full lakehouse, governance and AI agents to run inside your jurisdiction.
- [Migrating regulated workloads from Databricks for data sovereignty](https://datanature.ai/blog/migrating-from-databricks-for-data-sovereignty.md): Migrating regulated workloads from Databricks to DataNature for sovereignty usually means moving Spark, SQL and dbt code with modest changes, keeping Delta tables as they are, re-creating jobs and permissions, and reconciling outputs with lineage before cutover. Many teams keep Databricks for public-cloud work and run both.
- [What is a sovereign AI lakehouse?](https://datanature.ai/blog/what-is-a-sovereign-ai-lakehouse.md): A sovereign AI lakehouse is a data lakehouse (open tables plus compute) that runs entirely within a chosen jurisdiction, including its control plane, metadata, logs and AI models. It lets organizations use modern analytics and AI on regulated data without sending that data, or anything derived from it, outside their borders.
