Databricks-labs-lsql is a Python library that provides lightweight, stateless SQL execution capabilities for Databricks, built on top of the Databricks SDK. It's designed for serverless or containerized short-lived applications where minimal dependencies and faster startup times are critical, fetching results in JSON format through presigned external links. The current version is 0.17.0, and it appears to have an active release cadence with regular updates.
pip install databricks-labs-lsqlVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to initialize the `StatementExecutionExt` client using the `databricks-sdk` `WorkspaceClient` and execute SQL queries. It shows how to fetch results by iterating over them and how to execute DDL/DML statements. Authentication relies on environment variables (`DATABRICKS_HOST`, `DATABRICKS_TOKEN`) for the `WorkspaceClient`.
Ensure all generated resource names adhere to the new validation rules (alphanumeric, hyphens, underscores). Review and update resource naming conventions in your application.
Choose the appropriate Databricks SQL client based on your application's requirements: `lsql` for stateless, REST-driven, short-lived tasks; `databricks-sql-connector` for stateful, high-throughput, traditional SQL workloads.
Review your application's reliance on `ucx` checks. If critical, consider implementing custom validation logic or monitoring for potential issues that the disabled check previously covered.
Upgrade to v0.13.0 or later and utilize the `escape_name` and `escape_full_name` functions from `databricks.labs.lsql.escapes` to properly escape SQL identifiers, especially when dealing with user-generated or dynamic names.
Ensure the library is installed using pip: `pip install databricks-labs-lsql`
Configure Databricks authentication by setting environment variables (e.g., `DATABRICKS_HOST`, `DATABRICKS_TOKEN`), using a Databricks CLI configuration profile, or explicitly passing a `Config` object to the `LSQLClient`.
Verify the spelling and correctness of the table or view name, including its catalog and schema if necessary (e.g., `catalog.schema.your_table_name`). Ensure the user or service principal has `SELECT` permissions on the table/view.
This often indicates a known bug or an interaction issue between `lsql` and the underlying PySpark environment. Check the `databricks-labs-lsql` GitHub issues or documentation for updates, workarounds, or ensure compatibility with the PySpark version in use. You might need to explicitly cast or convert the `pyspark.sql.Row` objects if the library doesn't handle it automatically.