Databricks Professional Exam: Performance Tuning and Optimization Explained

Where this domain sits in the exam

The Professional exam has 10 domains. Developing Code for Data Processing is the largest at 22%, and Cost and Performance Optimisation is second at 13%, ahead of every other domain except coding. The full domain table and how weighting shifts from the Associate exam is covered in the Professional exam topics breakdown. This post goes one level deeper into what the 13% actually tests.

The five things this domain actually tests

Predictive Optimization removes manual maintenance

Managed tables under Unity Catalog can run Predictive Optimization, enabled by default on new accounts. It automatically runs OPTIMIZE, VACUUM, and statistics collection, three jobs a team would otherwise have to schedule and remember by hand. Exam questions in this area test whether you can explain why a managed table stays fast and keeps small-file counts low with no scheduled maintenance job visible anywhere.

Deletion vectors avoid rewriting whole files

DELETE, UPDATE, and MERGE used to mean rewriting every affected Parquet file. A deletion vector instead marks which rows in a file are logically removed or replaced, leaving the file itself untouched until it is compacted later. The smaller write finishes faster, and two transactions touching different rows in the same file are far less likely to collide.

Liquid Clustering is the default layout, not an alternative

CLUSTER BY replaces partitioning and Z-Order for new tables, and the current syllabus treats Liquid Clustering as the recommended approach, not one option among three. A table cannot mix Liquid Clustering with partitioning or Z-Order. Clustering keys can be redefined at any time with ALTER TABLE ... CLUSTER BY, and a routine OPTIMIZE only reclusters data that has drifted out of the current layout. OPTIMIZE FULL forces a complete recluster, which is what a key change or a first-time enable actually needs. This same trade-off gets a dedicated chapter under Data Modelling, because the decision applies to schema design as much as to performance.

Data skipping depends on tight file statistics

Every file written to a Delta table automatically carries per-column statistics: min, max, null count, and record count. A filter is checked against those stats before a file is opened, so a file whose stored max falls below the filter's threshold gets skipped entirely, never scanned. Liquid Clustering keeps related values physically together, which narrows each file's min and max range and lets the same query skip more of the table. External tables cap statistics collection at the first 32 columns by default; managed tables under Predictive Optimization are not capped that way.

Change Data Feed has real limitations

CDF exposes row-level changes for downstream streaming consumption, but this domain's questions focus on where it breaks down: VACUUM can remove files that CDF still needs to reference for older versions, schema changes complicate what a consumer sees, and retaining both time travel history and change data on the same table has real storage and latency costs that need to be weighed, not assumed away.

Query Profile diagnoses cost, not just correctness

The same operator graph used elsewhere to debug a wrong result gets read differently here: a node's spill-to-disk warning is one of the least visible cost drains, since the operator still finishes, just far slower than it should. Large shuffle read or write bytes on one operator usually point to an expensive join or aggregation strategy. One task taking dramatically longer than its neighbors is a skew signal, not a general capacity problem. The fix is almost always to change the query or the table (broadcast a small join side, filter earlier in the plan, recluster on the skewed key) before reaching for more compute.

The pattern behind all five

Every one of these swaps an expensive, all-or-nothing operation for a smaller, targeted one: marking rows instead of rewriting files, reclustering what drifted instead of the whole table, skipping files instead of scanning them, fixing the one slow operator instead of scaling the whole cluster. That trade-off, a full rewrite versus a targeted one, is the actual thing this domain is testing, independent of which specific feature a question names.

FAQ