Implementing OLAP and OLTP integration based on a single storage without ETL, data replicas, or pipelines
With the proliferation of AI agents increasing the demand for real-time processing in data infrastructure, Databricks has introduced a new data processing architecture that integrates transaction and analytics workloads into a single storage layer.
Databricks unveiled the 'LTAP (Lake Transactional/Analytical Processing)' architecture on the 25th.
LTAP is an architecture that integrates Online Analytical Processing (OLAP) and Online Transaction Processing (OLTP) based on a single copy of data within a lake, and is implemented by combining the serverless Postgres-based services 'Lakebase' and 'Lakehouse'.
Lakebase has already been adopted by thousands of customers worldwide and is handling 12 million database executions per day, the company said.
Existing HTAP (Hybrid Transaction and Analytics Processing) attempted to integrate two workloads into a single engine, resulting in performance degradation and dependency on proprietary infrastructure.
The 'Zero ETL' method also... rather than eliminating the pipeline The company explained that they only went as far as concealing it.
Unlike this, LTAP adopted a method of integrating data at the storage layer itself.
Lakebase has a structure that directly stores data in the Unity Catalog in open formats such as Iceberg and Delta.
There are three main features of LTAP.
With integrated governance based on a single source of truth, all data is stored in open object storage without transformation, and a single identity, permission, and auditing model is applied.
Transaction workloads scale independently across the lakehouse and operate without performance degradation, while analytics workloads scale independently in PostgreSQL, which guarantees full ACID.
The company explained that it reduces operating costs and maintains data freshness by eliminating data movement connectors between ETL pipelines and systems at the architecture level.
Along with this announcement, Databricks also expanded the capabilities of Lakebase.
Cross-cloud and cross-region disaster recovery capabilities have been added, and Git-style branching and snapshot features have made it possible to test without affecting operational data.
The company also stated that through autonomous database operation capabilities, agents can independently perform system status monitoring, performance degradation detection, index suggestions, and disaster recovery.
Databricks co-founder and CEO Ali Ghodsi stated, “Agents write code and execute loops at speeds humans cannot keep up with,” adding, “The infrastructure that supported previous computing eras has now become a bottleneck, and LTAP stated, "P removes this."