Install & Compatibility
Where this runs
tested against v0.0.9 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 20.2MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 1.7s · import 0.000s · 21MB
19MB installed
● package 19MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
DX
✓ from discoverx import DX
✗ from discoverx import DX
This quickstart demonstrates how to initialize DiscoverX, define a set of tables using a wildcard pattern, and then apply a SQL template (counting rows) concurrently across all matching tables in a Databricks environment.
from discoverx import DX
# Initialize DiscoverX. 'locale' can be set for region-specific rules.
dx = DX(locale="US")
# Define the tables to operate on using a wildcard pattern
# Example: all tables in 'my_catalog.my_schema'
from_tables = "my_catalog.my_schema.*"
# Example: Count rows in all selected tables and display the results
# The '{full_table_name}' placeholder is automatically replaced.
table_counts = dx.from_tables(from_tables).with_sql("SELECT COUNT(*) FROM {full_table_name}").apply()
# Display the resulting DataFrame
table_counts.display()
Debug
Known issues
breakingDiscoverX is a Databricks Labs project and is provided "AS-IS" without formal Service Level Agreements (SLAs). Issues should be filed as GitHub Issues and will be reviewed as time permits.fixBe aware of the experimental nature; do not rely on it for critical production workloads without internal support.
affects: All versions
deprecatedThe `scan` command has been deprecated. Users should migrate to `intro` or `scan (experimental)` for semantic classification and other scanning functionalities.fixReplace `.scan(...)` with `.intro()` for general overview or `.scan(experimental=True, ...)` for detailed scanning.
affects: 0.0.9 and earlier
gotchaWhen installing `dbl-discoverx` within a Databricks notebook using `%pip install`, it is often necessary to restart the Python kernel (`dbutils.library.restartPython()`) for the newly installed package to be properly loaded and available.fixAlways follow `%pip install dbl-discoverx` with `dbutils.library.restartPython()` in Databricks notebooks.
affects: All versions in Databricks notebooks
gotchaOn Databricks Runtime 15.4 LTS and above, regular expression handling in Photon is updated to match Apache Spark behavior. Previously accepted invalid regex patterns in `with_sql` commands might now cause queries to fail.fixEnsure all regex patterns used in `with_sql` or similar operations are strictly valid according to Apache Spark's regular expression syntax.
affects: Databricks Runtime 15.4 LTS and above
Upgrade
Version history
0.0.9latest on PyPI · released May 2, 2025
Audit
Dependencies
pysparkrequiredCore dependency for Databricks Lakehouse operations.
pandasrequiredRequired for data manipulation and DataFrame operations. PyPI specifies <2.0.0,>=1.0.0.
numpyrequiredRequired for numerical operations. PyPI specifies <1.24.0,>=1.16.0.