Data Platforms
Databricks Lakehouse Architecture
How the lakehouse pattern combines lake economics with warehouse guarantees, and where the design effort actually goes.
6 min read
What the pattern actually changes
The lakehouse pattern places transactional table semantics over open file storage. That gives ACID writes, schema enforcement, time travel and efficient upserts against data that remains in the organisation's own object storage in an open format.
The architectural consequence is that a single copy of data can serve engineering, analytics and machine learning workloads, reducing the number of extracts and derived copies that traditionally accumulate around a warehouse.
Where design effort is required
The technology removes some problems and exposes others. Workspace and catalogue topology, environment separation, access model, cost attribution and cluster or warehouse sizing policy all need explicit design.
- Catalogue and namespace design across environments and domains
- Access model: groups, service principals and least-privilege grants
- Job orchestration boundaries and idempotent reprocessing
- Table layout: partitioning, file sizing and maintenance operations
- Cost controls and chargeback visibility per workload
When it is the wrong answer
A lakehouse is a poor fit where workloads are small, entirely relational and already well served by an existing warehouse. Architecture should be proportional to the problem; introducing a distributed platform for modest volumes adds operating cost without benefit.
Working through this in your own environment?
We help organisations make these decisions with the constraints they actually have.