dbnd-spark

library1.0.34.1pypypiunverified

DBND Spark provides integration between Databand's data orchestration framework and Apache Spark. It enables tracking, monitoring, and logging of Spark jobs, including data metrics, lineage, and execution context. The library wraps SparkSession to automatically capture logs and telemetry. Version 1.0.34.1 is the latest stable release, with monthly updates. 'dbnd-spark' is part of the 'dbnd' ecosystem but installed separately. Maintained by Databand (now IBMA).

data observability workflow

pip install dbnd-spark

github ↗package ↗homepage ↗docs ↗

INSTALL

IMPORT

SIG · DBND-SPARK

dbnd-spark

datapythonv1.0.34.1

Install

4.6s avg

Import

—

Disk

51MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.0.34.1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.000s · 50.6MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 4.6s · import 0.000s · 52MB

51MB installed

● package 51MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

DbndSparkConfig

✓ from dbnd_spark import DbndSparkConfig

✗ dbnd_spark.DbndSparkConfig

Direct import avoids full package path; however, using module prefix is also valid.

Define a Spark job as a DBND task, configure tracking URL (optional), and run via dbnd_run.

from dbnd import dbnd_config, task from dbnd_spark import DbndSparkConfig @task def my_spark_job(): from pyspark.sql import SparkSession spark = SparkSession.builder.appName("test").getOrCreate() df = spark.range(10) df.show() spark.stop() if __name__ == "__main__": dbnd_config.set(DbndSparkConfig.webapp_url=os.environ.get('DBND_WEBAPP_URL', '')) my_spark_job.dbnd_run()

Debug

Known issues

deprecatedThe 'dbnd-spark' package is being deprecated in favor of 'dbnd' unified package. New versions of dbnd include Spark support internally.

fix

Migrate to 'dbnd' package and use 'from dbnd_spark import ...' from within dbnd.

affects: >=1.0.34

gotchaSparkSession must be created inside a DBND task. Creating it at module level breaks tracking.

fix

Always create SparkSession inside a @task-decorated function.

affects: all

breakingChanged from camelCase to snake_case for configuration attributes in v1.0.20.

fix

Use underscore style: app_name instead of appName.

affects: >=1.0.20

Upgrade

Version history

1.0.34.1latest on PyPI · released Mar 5, 2026

Audit

Dependencies

dbndrequiredCore library required for orchestration and tracking runtime.

pysparkrequiredRequired to interact with Spark.

Agent activity

18 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/databand-ai/dbnd ↗

packagehttps://pypi.org/project/dbnd-spark/ ↗

homepagehttps://databand.ai ↗

docshttps://dbnd.readme.io/ ↗