The Stack Has Shifted
Three years ago, the modern data stack meant dbt, Fivetran, Snowflake, and a BI tool of your choice. That combination still works. But the teams building data infrastructure for AI workloads in 2026 have a different set of constraints — and they’re making different choices.
The shift is driven by three forces: the need to serve both analytical and ML workloads from the same data, the economics of data movement, and the growing demand for lower-latency insights.
Lakehouses Are Now the Default
The debate between data warehouses and data lakes is largely over. The lakehouse architecture — open table formats like Apache Iceberg or Delta Lake on object storage, with a query engine layer on top — has become the default for teams building from scratch.
The appeal is practical: one copy of data, open formats that prevent vendor lock-in, and the ability to run both SQL analytics and ML training jobs on the same tables. The trade-off is operational complexity — lakehouses require more infrastructure discipline than managed warehouses.
Zero-ETL Is Changing the Economics
Cloud providers have aggressively pushed zero-ETL integrations that eliminate the extract-transform-load pipeline between operational databases and the analytical layer. Aurora to Redshift. DynamoDB to OpenSearch. Postgres to BigQuery.
For narrow use cases — primarily operational reporting on transactional data — zero-ETL delivers on its promise. The caution is scope creep: zero-ETL is not a general-purpose data integration strategy. Complex transformations, multi-source joins, and data quality enforcement still require a real pipeline.
Real-Time Streams Are No Longer Optional for AI
Batch pipelines were sufficient when analytics was the primary consumer of data infrastructure. AI changes the requirement. Recommendation systems, fraud detection, and personalisation models often need data that is seconds old, not hours old.
Kafka, Flink, and managed streaming services have become first-class components of the data stack for teams running AI in production. The integration of stream processing with lakehouse storage — via formats like Apache Hudi — is the area moving fastest right now.
What Hasn’t Changed
The fundamentals of good data engineering haven’t changed: model your data for its consumers, test your pipelines, document your assumptions, and don’t build complexity you don’t need yet. The teams that struggle with the modern data stack are almost always the ones who adopted new tools before mastering the fundamentals.
The 2026 stack is more powerful than what came before. It’s also more complex. The right architecture is the simplest one that meets your actual requirements — not the most sophisticated one that exists.