SDMetrics is an open-source Python library developed by DataCebo (part of the Synthetic Data Vault project) for evaluating the quality and efficacy of synthetic datasets. It provides a variety of metrics to compare synthetic data against real data across aspects like quality, privacy, and utility, and includes tools for generating comprehensive visual reports. The library is model-agnostic, allowing evaluation of synthetic data generated by any model. The current version is 0.28.0, with active and frequent releases.
pip install sdmetricsVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to load demo data, generate a single-table QualityReport, retrieve the overall score, and optionally visualize results. SDMetrics can also work with your own pandas DataFrames and metadata dictionaries.
Upgrade your Python environment to 3.9 or a newer supported version (<3.15).
Ensure your Pandas version is <3.0 when using SDMetrics. Check SDMetrics release notes for future Pandas 3.x compatibility updates.
Consider setting a `real_correlation_threshold` when computing `CorrelationSimilarity` to filter out column pairs without strong correlations in the real data. Values of 0.4 or higher are recommended.
Carefully review report breakdowns and individual metric scores. If `NaN`s appear unexpectedly, investigate the input data for those specific columns or metric configurations for potential causes of failure.
Convert `SDV` metadata objects to a dictionary using the `.to_dict()` method before passing them to SDMetrics reports. Example: `report.generate(real_data, synthetic_data, sdv_metadata_object.to_dict())`.
Convert the SDV metadata object to a dictionary using its `.to_dict()` method: `report.generate(real_data, synthetic_data, sdv_metadata_object.to_dict())`.
Pre-process your real and synthetic dataframes to handle missing values (e.g., imputation or removal) and outliers before passing them to SDMetrics. Check the metadata to ensure correct `sdtype` for columns.
Verify that the column names in your dataframes exactly match those referenced in your SDMetrics calls and the `metadata` dictionary.
Review the documentation for the specific metric being used to understand its data requirements. Ensure column `sdtype` in the metadata accurately reflects the data types and that there's enough data for computation.