Foundation Model APIs serve Databricks-hosted models from a ready-made endpoint, billed either pay-per-token or as provisioned throughput for production and fine-tuned models. Mosaic AI Gateway sits in front of those endpoints to track and govern traffic with inference tables, usage tables and rate limiting. Since August 2026 the Databricks docs call the gateway Unity Gateway; the exam guide still says AI Gateway.
Last updated October 2026.
The Databricks Generative AI Engineer Associate exam guide (the edition live since 18 March 2026) uses Mosaic AI Gateway, short form AI Gateway, and Mosaic AI Vector Search. Databricks has renamed both in its docs since then:
Answer with the exam guide's names; questions are written from the guide. Knowing the new names helps when you read current docs while studying.
Rename dates from the Databricks platform release notes (June and August 2026), checked 23 October 2026.
Foundation Model APIs give you state-of-the-art models through Databricks-managed serving endpoints, with no deployment to maintain. Most are OpenAI-compatible, so the OpenAI client works with the endpoint name passed as the model.
| Pay-per-token | Provisioned | |
|---|---|---|
| What it is | Ready-made endpoints billed by tokens used | Dedicated capacity with performance guarantees |
| Recommended for | Getting started, trying models, low volume | Production, high throughput, extra security needs |
| Fine-tuned models | No | Yes |
| Commitment | None | On-demand, or reserved for 1 or 3 months |
From the Foundation Model APIs documentation, checked 23 October 2026.
Two details catch people. Provisioned throughput needs serverless compute. And pay-per-token now has a priority mode, opted into per request, which admits requests ahead of standard traffic for steadier latency at a higher per-token rate. On the exam, "steady high volume with predictable latency" and "serve our fine-tuned model" both point to provisioned throughput.
For batch work, such as scoring a whole table, the SQL path is ai_query() with AI Functions rather than calling an endpoint row by row.
Keep two things apart. Calling a Foundation Model API endpoint runs only inference: you send a prompt, Databricks returns a completion. Serving your agent means logging your chain or agent with MLflow and deploying it as its own endpoint, which runs your retrieval, tools and orchestration. agents.deploy() from the Mosaic AI Agent Framework does that in one call and also creates a Review App and inference tables.
The gateway is the governance and traffic layer in front of model serving endpoints, including endpoints for external providers and for agents. The exam tests three tracking surfaces, and the distinction between them is the usual question:
What was said. Requests and responses logged to a Unity Catalog Delta table, the raw material for monitoring, evaluation and debugging.
Who spent what. Token counts and request volume per endpoint and per caller, so cost can be attributed to a team, app or user.
Caps requests or tokens per endpoint or per user in real time. The only one of the three that acts rather than records.
Safety and PII filters on requests and responses, configured once at the gateway instead of in every application.
Read the verb. "Stop a looping integration now" is rate limiting. "Show finance each team's spend" is usage tables. "Review what the model answered last week" is inference tables.
An application calls a serving endpoint from its backend, with the app's own credentials. Any option that puts a Databricks token or provider API key in browser JavaScript is wrong, however convenient it looks.
Pay-per-token uses preconfigured endpoints billed by tokens consumed and is recommended for getting started. Provisioned throughput reserves dedicated capacity with performance guarantees, is recommended for production, and is required to serve fine-tuned or custom-weight models.
It governs and tracks traffic to model serving endpoints. For the exam, know its three tracking surfaces: inference tables log request and response content, usage tables track consumption per caller, and rate limiting caps traffic in real time.
In the Databricks docs, yes: Unity Gateway became generally available on 4 August 2026, and the endpoint-level feature is documented as AI Gateway (legacy). The exam guide still uses Mosaic AI Gateway, so answer exam questions with that name.
Usage tables. They record token counts and request volume per endpoint and per caller, so spend can be attributed to a team, application or user.
The Certified GenAI Engineer course covers Foundation Model APIs, endpoint access control and AI Gateway tracking in short chapters with exam-style questions.
Open the GenAI Engineer course →