AI readiness · Databricks
Databricks AI readiness assessment
Score your Databricks estate against the six factors that decide whether data is ready for an AI workload — read-only, metadata only, never a row of your data. Free to run.
Databricks has a distinction the other platforms do not: it is usually where the AI workload itself will run. Teams pick the lakehouse because the plan is models, agents, and retrieval. That makes the readiness question unusually concrete, because the pipeline that will embed your documents and the table full of unmasked emails are often in the same workspace.
Unity Catalog can answer it. Every catalog it governs exposes an INFORMATION_SCHEMA covering tables, columns, comments, constraints, tags, masks, row filters, and owners. The catch is the word governs: most estates still carry tables in the legacy hive_metastore, which sit outside Unity Catalog's auditing, lineage, and access control entirely.
The Databricks readiness tension: coverage — how much of the estate lives under governance at all.
What the assessment reads on Databricks
- Unity Catalog INFORMATION_SCHEMA — structure, comments, data types, declared constraints, and last-altered timestamps per catalog.
- Tags, column masks, and row filters — what governance has actually been applied, as opposed to what policy says should be.
- Table owners — whether anyone is accountable for a dataset an AI workload is about to consume.
- Lineage system tables — where a table came from — optional, and reported honestly as not assessable when unavailable.
- Governance coverage — which parts of the estate are inside Unity Catalog and which remain in hive_metastore.
Those signals roll up into the six factors — Clean, Contextual, Consumable, Current, Correlated, Compliant — scored against roughly 50 requirements for the specific AI workload you pick, whether that is retrieval, agents, model training, or feature serving. The output is a readiness score, the single biggest blocker, and a prioritised fix plan with the score gain each fix earns. See the methodology for how scoring works.
Access it needs
A workspace URL, a SQL warehouse, and a personal access token whose identity holds three grants on the catalogs in scope: USE CATALOG, USE SCHEMA, and BROWSE. BROWSE is metadata visibility with no data access — the identity never receives SELECT on your tables. The metadata queries are lightweight, so a small warehouse runs the scan and auto-suspends afterward.
-- Run as a metastore or catalog admin. BROWSE = metadata only, no data SELECT.
GRANT USE CATALOG ON CATALOG <catalog> TO `<token-identity>`;
GRANT USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<token-identity>`;
GRANT BROWSE ON CATALOG <catalog> TO `<token-identity>`;The exact SQL each connector runs is published and generated from source, so a security review is a verification rather than a trust fall. If your warehouse cannot accept connections from the internet, the CLI agent runs the same metadata-only assessment inside your network. Full setup steps live in the connection guide.
The honest limits on Databricks
- Lineage requires an account admin to enable the system.access schema. Without it that single requirement reports not assessable rather than guessing; everything else still scores.
- Tables still registered in hive_metastore sit outside Unity Catalog governance. The scan surfaces that as a coverage finding rather than silently skipping them.
A metadata scan cannot verify values anywhere: a comment can be stale, a constraint can be worked around in application code, and a fresh-looking table can be full of duplicates. What it gives you is the structural truth of the estate — what exists, what is documented, what is governed, what is fresh, and what your chosen workload requires that is missing. It prepares evidence. It does not certify compliance, and no tool that reads only metadata honestly can.
Frequently asked questions
What access does an AI readiness scan need on Databricks?
A personal access token and a SQL warehouse, with three grants on the catalogs in scope: USE CATALOG, USE SCHEMA, and BROWSE. These expose catalog metadata and nothing else; the token identity never receives SELECT on table data.
Do I need Databricks system tables enabled?
Only for one signal. The lineage completeness check reads system.access tables, which require an account admin to enable. If they are not enabled, that requirement reports not assessable rather than guessing, and the rest of the assessment scores normally.
What about tables still in hive_metastore?
They are surfaced as a coverage finding. Unity Catalog only describes what lives inside it, so an estate can score respectably on its governed half while the ungoverned half — often the older, less documented half — is where an AI corpus would actually get assembled from.
Get your readiness score.
Connect read-only credentials and see your score and biggest blocker in minutes. Metadata only. Starts free.
Run the free scanWant to see the output before connecting anything? Read a sample readiness report built entirely from metadata. For the deeper technical walkthrough, read the Databricks assessment post. Other platforms: see all six.