Liquid Clustering is the Databricks-recommended layout for every new Delta table: clustering keys set with CLUSTER BY that can change without rewriting data. Predictive optimization runs OPTIMIZE, VACUUM and ANALYZE automatically on Unity Catalog managed tables. Together, with CLUSTER BY AUTO, they remove most table maintenance jobs, and the Professional exam tests when each applies.
Last updated October 2026.
The topic appears in two sections of the Databricks Data Engineer Professional exam. Cost and Performance Optimisation (13%) treats it as an optimization technique: managed tables, deletion vectors, Liquid Clustering and data skipping. Data Modelling (6%) treats it as a design decision: Liquid Clustering versus partitioning versus Z-Order. The Associate-level comparison is in Liquid Clustering vs partitioning and Z-Order; this post covers what the Professional exam adds.
ZORDER and cannot be combined with either on the same table.ALTER TABLE t CLUSTER BY (a, b) takes effect for new data without rewriting the old.OPTIMIZE FULL to recluster existing data against the new keys.The Professional-level trap: a team changes clustering keys on a table with two years of data, runs a routine OPTIMIZE, and expects old queries to speed up. They will not: the routine run leaves already-clustered data alone. OPTIMIZE FULL is the answer.
Predictive optimization runs three maintenance operations on Unity Catalog managed tables, using serverless jobs compute billed to your account:
| Operation | What it does |
|---|---|
| OPTIMIZE | Triggers incremental clustering and compacts file sizes |
| VACUUM | Deletes data files no longer referenced, within the retention window (7 days by default) |
| ANALYZE | Collects the statistics that make data skipping work |
From the Databricks docs on predictive optimization and liquid clustering, checked 23 October 2026.
It is enabled by default for accounts created on or after 11 November 2024, and Databricks has been rolling it out to older accounts. Databricks recommends it for all Unity Catalog managed tables. Two limits matter on the exam: it does not run on external tables, and it does not run ZORDER; it ignores Z-ordered files. A Z-ordered or external table still needs its own maintenance jobs.
CLUSTER BY AUTO turns on automatic liquid clustering: Databricks chooses clustering keys from the table's query patterns and can change them as those patterns shift. It requires predictive optimization, so it is only available on Unity Catalog managed tables, and Databricks recommends it for tables whose query patterns change often.
For an existing partitioned table, ALTER TABLE t REPLACE PARTITIONED BY WITH CLUSTER BY AUTO converts it, starting from the old partition columns as keys. Databricks recommends this automatic conversion command for moving partitioned tables to liquid clustering.
Delta stores minimum, maximum and null counts per column per file. A filter skips any file whose range cannot match. Clustering keeps related values together, so each file's range is narrow and more files are skipped. On external tables, statistics cover the first 32 columns by default, so a filter on a later column gets no skipping; managed tables under predictive optimization choose which columns to track from how the table is queried.
"Reduce operational overhead", "query patterns keep changing", "no one should schedule OPTIMIZE or VACUUM".
"Clustering keys were just changed" or "clustering was just enabled on an existing table", and old data must benefit.
The table is external, or still uses Z-Order. Predictive optimization will not cover it.
An existing, very large table with a stable, low-cardinality partition key that is not causing problems.
It automatically runs OPTIMIZE, VACUUM and ANALYZE on Unity Catalog managed tables, using serverless compute. That keeps files compacted and clustered, removes unreferenced files and keeps statistics fresh without scheduled jobs.
No. Predictive optimization runs only on Unity Catalog managed tables. External tables still need their own OPTIMIZE, VACUUM and ANALYZE jobs.
It enables automatic liquid clustering, where Databricks chooses and adjusts clustering keys from the table's query patterns. It requires predictive optimization, so it works only on Unity Catalog managed tables.
After enabling clustering on an existing table or changing its clustering keys. A routine OPTIMIZE is incremental and leaves already-clustered data alone, so only OPTIMIZE FULL reclusters all existing data against the new keys.
Up to four. On tables under about 10 TB, more keys can slow single-column filters, so choose only the columns queries filter on most.
The Certified DE Professional course covers managed tables, Liquid Clustering, data skipping and the layout trade-off in short chapters with exam-style questions.
Open the DE Professional course →