Registry / data / pydeequ

pydeequ

JSON →
library1.6.0pypypi✓ verified 27d ago

PyDeequ is a Python API for Deequ, a library built on top of Apache Spark for defining 'unit tests for data', which measure data quality in large datasets. The current version is 1.5.0, released on April 1, 2025. PyDeequ follows a regular release cadence, with updates approximately every few months.

pip install pydeequ
INSTALL
IMPORT
SIG · PYDEEQU
P
pydeequ
datapythonv1.6.0
Install
8.0s avg
Import
Disk
165MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.6.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.000s · 164.9MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 8.0s · import 0.000s · 157MB
165MB installed
● package 165MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

AnalysisRunner
from pydeequ.analyzers import AnalysisRunner
from pydeequ.analyzers import AnalysisRunner

This script sets up a Spark session, creates a sample DataFrame, performs a completeness analysis on column 'a', and displays the results.

import os from pyspark.sql import SparkSession, Row import pydeequ # Set up Spark session spark = (SparkSession .builder .config('spark.jars.packages', pydeequ.deequ_maven_coord) .config('spark.jars.excludes', pydeequ.f2j_maven_coord) .getOrCreate()) # Sample data df = spark.sparkContext.parallelize([ Row(a='foo', b=1, c=5), Row(a='bar', b=2, c=6), Row(a='baz', b=3, c=None)]).toDF() # Perform analysis from pydeequ.analyzers import AnalysisRunner analysisResult = AnalysisRunner(spark) \ .onData(df) \ .addAnalyzer(pydeequ.analyzers.Completeness('a')) \ .run() # Show results analysisResult
Debug
Known issues
breakingPyDeequ 2.0 introduces Spark Connect support, moving away from the Py4J-based JVM bridge. This change may require code modifications for compatibility.
fix
Update your code to utilize Spark Connect for distributed data processing tasks.
affects: >=2.0.0b1
deprecatedThe 'hasPattern' function is deprecated in PyDeequ 1.5.0 and will be removed in future releases.
fix
Use alternative methods for pattern matching in data validation tasks.
affects: >=1.5.0
Upgrade
Version history
1.6.0latest on PyPI · released Jul 8, 2026
Audit
Dependencies
pysparkrequiredPyDeequ relies on PySpark for distributed data processing capabilities.
Agent activity
36 hits · last 30 days
node
32
OpenAI (training)
1
Resources
pydeequ — pip install pydeequ · libregistry