Registry / data / sdmetrics

sdmetrics

JSON →
library0.28.0pypypi✓ verified 85d ago

SDMetrics is an open-source Python library developed by DataCebo (part of the Synthetic Data Vault project) for evaluating the quality and efficacy of synthetic datasets. It provides a variety of metrics to compare synthetic data against real data across aspects like quality, privacy, and utility, and includes tools for generating comprehensive visual reports. The library is model-agnostic, allowing evaluation of synthetic data generated by any model. The current version is 0.28.0, with active and frequent releases.

pip install sdmetrics
INSTALL
IMPORT
SIG · SDMETRICS
S
sdmetrics
datapythonv0.28.0
Install
16.3s avg
Import
4784ms
Disk
432MB
Pass rate
5/ 10
Env Coverage5 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.28.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
build_error
glibc
py 3.103.910 runs
installs and imports cleanly · install 16.3s · import 4.784s · 415MB
432MB installed
● package 432MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

load_demo
from sdmetrics import load_demo
QualityReport
from sdmetrics.reports.single_table import QualityReport
CategoryCoverage
from sdmetrics.single_column import CategoryCoverage

This quickstart demonstrates how to load demo data, generate a single-table QualityReport, retrieve the overall score, and optionally visualize results. SDMetrics can also work with your own pandas DataFrames and metadata dictionaries.

import pandas as pd from sdmetrics import load_demo from sdmetrics.reports.single_table import QualityReport # Load demo data (real, synthetic, and metadata) real_data, synthetic_data, metadata = load_demo(modality='single_table') # Or create your own dataframes and metadata # real_data = pd.DataFrame({'column1': [1, 2, 3], 'column2': ['A', 'B', 'C']}) # synthetic_data = pd.DataFrame({'column1': [1, 2, 2], 'column2': ['A', 'C', 'B']}) # metadata = {'columns': {'column1': {'sdtype': 'numerical'}, 'column2': {'sdtype': 'categorical'}}, 'primary_key': None} # Create a QualityReport report = QualityReport() # Generate the report report.generate(real_data, synthetic_data, metadata) # Print the overall quality score print(f"Overall Quality Score: {report.get_score():.2f}%") # Get a visualization for a specific property (e.g., 'Column Shapes') # fig = report.get_visualization(property_name='Column Shapes') # fig.show() # Save the report # report.save(filepath='demo_data_quality_report.pkl') # To load later: loaded_report = QualityReport.load(filepath='demo_data_quality_report.pkl')
Debug
Known issues
breakingSDMetrics dropped support for Python 3.8 starting from version 0.24.0. Ensure your environment uses Python 3.9 or newer.
fix
Upgrade your Python environment to 3.9 or a newer supported version (<3.15).
affects: >=0.24.0
breakingSDMetrics pinned Pandas below version 3.0 in v0.26.0 to ensure compatibility. Direct usage with Pandas 3.x might lead to unexpected behavior or errors.
fix
Ensure your Pandas version is <3.0 when using SDMetrics. Check SDMetrics release notes for future Pandas 3.x compatibility updates.
affects: >=0.26.0
gotchaWhen using `CorrelationSimilarity` on noisy data with no clear trends, the metric might return a high score, indicating that the synthetic data successfully captures the non-existent 'trend'. This can be misleading if you expect to measure actual correlation preservation.
fix
Consider setting a `real_correlation_threshold` when computing `CorrelationSimilarity` to filter out column pairs without strong correlations in the real data. Values of 0.4 or higher are recommended.
affects: All
gotchaWhen generating reports, if some metric computations fail, SDMetrics might report them as `NaN` (Not a Number) scores rather than explicit errors, potentially hiding underlying issues with data or metric configuration.
fix
Carefully review report breakdowns and individual metric scores. If `NaN`s appear unexpectedly, investigate the input data for those specific columns or metric configurations for potential causes of failure.
affects: All
gotchaPassing an `SDV` metadata object directly to `sdmetrics.reports` (e.g., `QualityReport.generate`) will raise a `TypeError`. SDMetrics expects a plain dictionary for metadata.
fix
Convert `SDV` metadata objects to a dictionary using the `.to_dict()` method before passing them to SDMetrics reports. Example: `report.generate(real_data, synthetic_data, sdv_metadata_object.to_dict())`.
affects: All
Errors
Common errors & fixes
TypeError: Expected a dictionary but received a <class 'sdv.metadata.SingleTableMetadata'> instead.
An `sdv.metadata.SingleTableMetadata` (or similar SDV metadata object) was passed directly to an SDMetrics report method that expects a standard Python dictionary for metadata.
fix
Convert the SDV metadata object to a dictionary using its `.to_dict()` method: `report.generate(real_data, synthetic_data, sdv_metadata_object.to_dict())`.
ValueError: Inputs contain NaN, infinity or a value too large for dtype('float64').
This error often occurs when numerical data contains missing values (NaNs) or extreme values that a metric or underlying scikit-learn model cannot handle without prior processing.
fix
Pre-process your real and synthetic dataframes to handle missing values (e.g., imputation or removal) and outliers before passing them to SDMetrics. Check the metadata to ensure correct `sdtype` for columns.
KeyError: 'column_name not found'
The specified 'column_name' in a metric computation (e.g., `CategoryCoverage.compute`) or a report configuration does not exist in the provided real or synthetic dataframes.
fix
Verify that the column names in your dataframes exactly match those referenced in your SDMetrics calls and the `metadata` dictionary.
IncomputableMetricError: The metric cannot be computed with the given data.
This generic error can occur if the data does not meet the specific requirements of a metric (e.g., attempting a numerical correlation metric on categorical data, or insufficient data points).
fix
Review the documentation for the specific metric being used to understand its data requirements. Ensure column `sdtype` in the metadata accurately reflects the data types and that there's enough data for computation.
Upgrade
Version history
0.28.0latest on PyPI · released Mar 19, 2026
Audit
Dependencies
pythonrequiredRequires Python versions >=3.9, <3.15.
Agent activity
7 hits · last 30 days
node
6
OpenAI (training)
1
Resources
sdmetrics — pip install sdmetrics · libregistry