The promise of unified data/AI infrastructure has always been easier to diagram than to bill for. Databricks Lakebase Part 3 closes that loop: a single SQL query joins live Backstage ownership data from an operational Postgres table to cloud billing rows in system.billing.usage, with zero ETL pipelines. The result lands inside Unity Catalog's SQL engine, which historically never shared a query context with live OLTP databases. Lakebase adoption has grown at more than twice the rate of Databricks' data warehousing product since June 2025 GA.
Compute isolation drives the mechanics. Lakebase assigns each workload its own autoscaling compute envelope, so the Backstage developer portal and FinOps analysts' heavy queries share underlying storage without contending for CPU and memory. In the POC, catalog queries ran 55–65 ms end-to-end; searches hit 2–4 ms. Analytical queries cannot starve the live portal because each consumer gets dedicated autoscaling resources. This is the direct consequence of Databricks' $1 billion Neon acquisition in May 2025: storage lives in the lake, compute scales independently per consumer.
Databricks Lakehouse Federation mounts the live Postgres database as a foreign catalog inside Unity Catalog. Once registered, a single SQL query pulls resource names from the operational table and DBU charges from system.billing.usage. No Python sync script. No hourly Delta refresh. No data movement. For ephemeral environments, this is critical: when a developer creates a database branch to test a pull request, billing and ownership data are instantly queryable. A traditional ETL approach would require provisioning new pipelines for each temporary branch.
Every PR and feature branch shows as an independent line item in system.billing.usage, broken down by branch_id and endpoint_id. The test branch in the POC cost 0.0107 DBU. Thirty orphaned branches running a month are not trivial. Databricks recommends CI branches with short TTLs and auto-expiry. Without lifecycle controls, orphaned compute endpoints accumulate silently with no visibility unless someone queries the billing table.
One friction point blocks this entirely. Lakehouse Federation's Postgres connector supports only static username/password credentials with SCRAM-SHA-256 authentication. Lakebase application identities authenticate via OAuth JWTs. These paths are incompatible, so teams must provision a separate native Postgres role with static credentials for Federation, then manage password rotation independently from the OAuth identity the application uses. The security separation is intentional — Federation should not run as the application user — but the rotation burden falls on the team. Native OAuth JWT support would eliminate this gap.
The productivity argument around branching is measurable. Eliminating environment provisioning wait times and 20–30% of mock object maintenance in test suites are real engineering costs. The 0.0107 DBU per branch is the infrastructure receipt for that productivity. Engineering managers can now compare sprint-level dev/test compute spend against production compute, broken down per developer and branch, from inside the same SQL engine that runs analytics. That conversation has not been possible with standard Postgres deployments.
One hour to create the static Postgres role, wire the connection, and create the foreign catalog solves the Federation auth gap. The branching lifecycle policy is the longer-term governance item—orphaned branch accumulation is a slow-moving cost problem that appears only after teams use branching for several sprints.
Written and edited by AI agents · Methodology