What is a data lakehouse? Architecture and trade-offs

Your company just finished migrating to a new data platform. Storage is unified, costs are down, and the architecture diagrams look beautiful. Six months later, the analyst team is still drowning in ad-hoc requests, business users are still pinging Slack for numbers, and someone in Finance is quietly building a shadow spreadsheet because they can't get answers fast enough.

A lakehouse fixes major infrastructure problems. But you only see the full value when you also invest in how people find, define, and use the data above the storage layer.

What a data lakehouse actually is

A data lakehouse is a data management architecture that combines the scalable, low-cost storage of a data lake with the transactional reliability and query performance of a data warehouse. You get one place to store structured tables, semi-structured logs, and unstructured data, with ACID (atomicity, consistency, isolation, durability) transactions, schema evolution, and efficient updates built in.

Lakehouses separate compute and storage. Your data sits in cloud object storage (S3, GCS, ADLS), and query engines connect to it independently. You can run a Spark job for ML training and a SQL query for a board report against the same underlying tables without duplicating anything or fighting over resources.

Open table formats

Open table formats make this possible. Apache Iceberg, Delta Lake, and Apache Hudi sit on top of Parquet files in object storage and add transactional capabilities that plain files lack: concurrent writes without corruption, row-level updates and deletes (critical for GDPR compliance), schema changes that don't break downstream queries, and partition pruning that keeps queries fast at petabyte scale. If you're evaluating formats, Iceberg offers the broadest multi-engine compatibility and has become the default for vendor-neutral stacks. Delta Lake has the tightest Databricks integration and the most mature time-travel implementation. Hudi was built around incremental processing for near-real-time workloads.

In practice, the major platforms each implement lakehouses differently. Databricks built its Lakehouse Platform around Delta Lake and Unity Catalog. Snowflake has steadily blurred the warehouse-lake boundary with Iceberg table support and external data access. Google BigQuery offers BigLake for cross-cloud lakehouse workloads. The implementations vary, but they converge on the same idea: open table formats on top of cloud object storage, with governance and SQL performance layered in.

Most modern data stacks that follow this pattern organize data in a medallion architecture: bronze (raw ingestion), silver (cleaned and conformed), and gold (business-ready aggregates). If you've worked with dbt or any transformation framework, the layering is familiar. You always preserve raw data, transformations stay traceable, and the gold layer is what analysts and AI agents actually query.

How lakehouses change what's possible for analytics teams

Before lakehouses, structured business data sat in the warehouse while unstructured data lived in the lake, accessible mainly to engineers comfortable with Spark. Lakehouses collapse that divide. SQL analysts can query the same underlying tables that data scientists use for ML training, through whatever engine fits the workload (Trino for interactive analytics, Spark for heavy transformations, a warehouse engine for BI), without anyone duplicating data between systems.

Batch and streaming coexisting in one infrastructure is where lakehouses get more interesting. Instead of maintaining separate pipelines for historical analysis and real-time use cases, you run both within the same architecture. Your product team gets real-time feature adoption metrics while your finance team runs monthly revenue reconciliation against the same data store. That convergence used to demand heavy engineering investment. With a lakehouse, it's just how things work.

Data lakehouse vs. data lake vs. data warehouse

The choice between these architectures comes down to how strictly you need to govern data up front versus how much flexibility you need at ingest.

How the three architectures compare

In practice, most modern stacks mix these patterns. The "right" answer often comes down to which workloads need the strongest guarantees.

When to choose what

Stick with a warehouse when your workloads are primarily structured data with well-defined reporting needs and strong compliance requirements.

Use a lake when you need cost-effective storage for massive raw data volumes and your primary consumers are engineers and scientists comfortable with distributed computing.

Choose a lakehouse when you need both structured analytics and ML workloads, want to consolidate architecture complexity, or need to support diverse data types with proper governance. One practical caveat: moving to a lakehouse isn't just a storage swap. It involves rethinking access patterns, rewriting transformations, retraining teams, and often running old and new systems in parallel for months. Teams that underestimate migration effort end up with two data platforms and the worst of both worlds.

Why having a lakehouse doesn't automatically fix the analytics workflow

A lakehouse fixes infrastructure problems. It doesn't fix the organizational patterns that keep your team fielding the same requests week after week.

Tool sprawl persists despite unified storage. Your data lives in one place, but you're still writing SQL in one tool, building charts in another, and pasting screenshots into a doc to share with stakeholders. The work lives scattered across disconnected tools with no lineage or version control. In Hex's State of Data Teams 2026 report, 31% of data leaders cited trust as their top concern with AI on their data — nearly twice any other concern. That dissatisfaction has more to do with how people interact with data than where it lives. And the gap has only widened as more teams adopt lakehouses and discover that the consumption layer is the real bottleneck.

Organizational bottlenecks don't disappear with better infrastructure. Ad-hoc request queues don't shrink just because the data is more accessible in theory. Approval processes and gatekeeping patterns remain unchanged. Faster infrastructure often just surfaces more demand. Meanwhile, someone builds a solid analysis, shares it via Slack, and it disappears into the conversation history. Without a discovery layer, teams have no way to find and build on what others already figured out. The backlog grows, and the duplication compounds.

Metric definitions still drift. This is the subtlest and most damaging gap. Having all your data in one storage layer doesn't mean everyone agrees on what "revenue" or "active user" means. When five teams define "monthly active user" five different ways, the lakehouse dutifully serves all five answers — each technically correct against its own logic, none of them useful for making a consistent decision. Many teams spend more time keeping data available than doing strategic analysis, and that work shows up as maintenance instead of new insights.

Most teams end up with a mix of technical and organizational problems, and the lakehouse only covers one side of that equation.

What to look for in the analytics layer above your lakehouse

Whether your lakehouse investment pays off depends on what you build above it. Infrastructure vendors tend to treat this layer, everything between stored data and the people making decisions, as someone else's problem. But it's where the ROI lives.

A context layer that keeps AI answers grounded

Context is what separates an AI agent that confidently returns the wrong number from one that returns a useful answer. Every analytics team on a lakehouse gets here eventually. The question is just where to start.

The lightest move is also the fastest: endorse the tables your analysts already trust and add plain-language descriptions to your warehouse. That gives an AI agent enough signal to prefer the right tables, understand what columns mean, and avoid common misinterpretations, often further than teams expect before they've touched a semantic model.

From there, workspace rules and agent rules files let platform teams set standing instructions like "always filter to the current fiscal year," "this table is billing data, not usage data," without building anything formal. For metric definitions that need to be consistent everywhere, Hex's Modeling Agent lets teams author semantic models directly. One definition for churn rate, one for monthly active user, resolved the same way whether the query comes from a notebook, a dashboard, or an AI agent.

What makes this practical rather than aspirational is observability. Context Studio shows which questions are producing low-confidence answers, which metrics are being queried inconsistently, and where agents are routing around governed definitions to hit raw tables directly. Governance investment gets targeted where it actually matters, not built speculatively from scratch.

The context layer isn't a monolith you install before AI analytics can work. It's a spectrum you deepen based on where your answers actually break down.

Governance that works for humans and AI

Good governance speeds your team up by making access predictable, not by making every question a negotiation. You know it's broken when people routinely export to spreadsheets to get around access rules.

In practice, that means centralized policy enforcement that follows users across tools, visibility into how metrics are used across the organization, and audit trails that make lineage traceable without needing a data engineering degree to navigate. You want to see which metric definitions teams query most often, what questions produce low-confidence answers, and where people are still routing around governed definitions to hit raw tables directly.

This matters even more now that AI agents are becoming first-class consumers of your lakehouse data. Your governance layer needs to enforce the same access controls and business logic for AI agents as it does for human users. Standards like Model Context Protocol (MCP) are emerging to give AI agents access to governed business definitions rather than raw, uncontextualized table structures. If your analytics layer can't serve AI with the same governed context it serves your analysts, governance gaps widen with every new AI use case.

Self-service people actually use

Self-service works when it matches how different people ask questions, not when it asks everyone to behave like analysts.

Someone asking "what was our churn last quarter?" needs a different interface than someone building a cohort model in Python. Quick answers need natural language. Deep analysis needs SQL and fast iteration. Modeling and governance need full code environments. If your analytics layer forces all of those through the same surface, it'll serve none of them well.

In Hex, someone can ask a question in plain language through Threads, get an answer grounded in semantic models, and click through to see the full SQL in a notebook if they want to verify it. The data team sees every question that's been asked and can improve the context powering those answers over time. At PandaDoc, that workflow delivered click-through rate data 75% faster than building it manually — and analysts could still inspect every query behind the scenes.

Self-service has to ship with governance and inspectability, or it doesn't stick.

Where lakehouses are headed

Iceberg is converging toward a de facto standard. Snowflake, Databricks, Google, and AWS have all added or deepened Iceberg support over the past year, and the catalog wars (Polaris, Unity Catalog, Gravitino) are sorting out how metadata interoperability will work. For data teams, this means less lock-in risk and more freedom to swap engines without rewriting pipelines. And as the storage layer becomes increasingly commoditized, the competitive surface is moving up the stack.

AI is accelerating that shift. Major cloud and warehouse providers now position their data platforms as AI-native, and lakehouses increasingly serve as the foundation where AI analytics runs alongside SQL analytics. The semantic definitions your team already maintains for human analysts are becoming the interface for AI agents too, keeping metrics consistent and enforcing governance policies regardless of who (or what) is asking. Teams that have invested in a governed semantic layer have a head start over those still relying on raw table access.

Making the lakehouse work for your team

Treat the lakehouse as the foundation, then invest equal attention in the layer where people and processes live. Metric definitions won't converge on their own, and self-service won't happen just because queries run faster.

The teams that get the most from their lakehouse are the ones that pair it with governed semantic definitions, self-service that people actually use, and collaborative workflows where analysis builds on itself instead of disappearing. If you want to see what that looks like in practice, sign up or request a demo.

Frequently Asked Questions

How do open table formats like Iceberg reduce vendor lock-in?

Open table formats store your data in standard file formats (typically Parquet) on cloud object storage you control, with metadata layers that any compatible engine can read. That means you can run Spark, Trino, Snowflake, or Databricks against the same tables without copying data or rewriting pipelines. If you decide to switch query engines later, your data stays put and the table metadata remains readable. The practical result is that your choice of compute becomes separable from your choice of storage. That gives you more negotiating leverage with vendors and lowers the cost of switching if a platform stops meeting your needs.

How does a data lakehouse affect data governance and compliance?

Lakehouses generally improve governance over raw data lakes by adding ACID transactions, schema enforcement, and time-travel queries for auditing. That said, the unified storage model can create more complex governance scenarios because structured transactions and unstructured documents coexist in the same layer. Build governance into the architecture from day one rather than retrofitting it after your team has already built workarounds. Teams that treat governance as an afterthought tend to discover the gaps only once they're deep into production workloads.

What's the relationship between a data lakehouse and a semantic layer?

A semantic layer sits on top of your lakehouse and translates raw table structures into business-friendly definitions, so "revenue" means the same thing whether a human analyst queries it, a dashboard displays it, or an AI agent reasons about it. The lakehouse provides the governed, transactional storage foundation, while the semantic layer provides the shared business context. Together, they create a path to self-service that works in practice: people interact with familiar concepts while the technical complexity stays abstracted. As AI agents become more common consumers of lakehouse data, the semantic layer becomes the interface that keeps AI answers grounded in governed business logic rather than raw guesswork.