DataNature platform
Data Engineering: dbt and Spark, Bronze to Gold
DataNature Data Engineering is the transformation workbench. A dbt editor and distributed Apache Spark build medallion pipelines from Bronze to Silver to Gold, scaling out with the cluster, with a query editor, workflow orchestration, lineage sync and a Voir copilot that explains, optimizes and fixes code.
Updated
- dbt editor
- Version-controlled SQL models with tests and docs.
- Spark engine
- Distributed processing that scales out with the cluster.
- Query editor
- Explore and test with results and history.
- Voir copilot
- Explain, optimize and fix pipelines in place.
- Lineage sync
- dbt runs and metadata flow into Atlas.
On this page
Who uses the Data Engineering workbench?
Data engineers, analytics engineers and data scientists. Each works in the tool they prefer on the same governed tables.
How do medallion layers work?
Connect lands raw data in Bronze. dbt models clean it into Silver and model it into Gold. Atlas records the lineage at every step.
Does access control apply in notebooks?
Yes. Every query runs as the user who wrote it, with the same row and column rules as anywhere else.
Platform components in this module
Learn more about each DataNature product and module mentioned here.
Products
Learn more about Data Engineering
Articles and guides
- Lakehouse architecturedbt on Spark and Trino with Apache Iceberg: building medallion pipelines that stay fast and correct
- Lakehouse architectureDataNature architecture, technically: the open engines behind a sovereign AI lakehouse
- Lakehouse architectureMedallion architecture explained: Bronze, Silver and Gold
- AI and agentsHow to build predictive models on DataNature: from lakehouse data to production predictions
- Lakehouse architectureBuilding a customer data platform on a sovereign lakehouse: the AI360 architecture, step by step
- Lakehouse architectureMigrating regulated workloads from Databricks for data sovereignty
Success stories
- Nabil Bank, Nepal (named with permission)Nabil Bank brings its AI initiatives onto one sovereign lakehouse in Nepal
- A commercial bank in NepalA commercial bank in Nepal builds customer 360 and faster loan processing on DataNature
- A leading mobile and broadband operator in JordanA leading telecom operator in Jordan turns network and subscriber data into retention and revenue
Solution blueprints
- Medallion data warehouse on an open lakehouseBronze, Silver and Gold on Delta or Iceberg tables, with no separate warehouse
- Migration to a sovereign lakehouseMove from legacy warehouses, Hadoop or cloud platforms without a big bang
- Feature store and MLOpsFrom notebook to governed production model
- Conversational analytics on a semantic layerPlain-language questions, governed SQL, consistent numbers
See it on your own data.
Book a demoDemos and quotes are handled by Dlytica, the company behind DataNature.