Registry / data / empirical-calibration

empirical-calibration

JSON →
library0.12pypypiunverified

Empirical Calibration (EC) is a Python library (version 0.12) designed for correcting bias in data samples using generic weighting methods. It formulates the calibration problem as a convex optimization, solved efficiently in a dual form, and aims to reduce data biases in various statistical fields, such as survey sampling and causal studies with observational data. The library is actively maintained, with the latest release in May 2024 and ongoing development on GitHub.

pip install empirical-calibration
INSTALL
IMPORT
SIG · EMPIRICAL-CALIBRAT
E
empirical-calibration
datapythonv0.12
Install
Import
Disk
Pass rate
0/ 10
Env Coverage0 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.12 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
✕ build_error
2/4 runs
py 3.11
✕ build_error
2/4 runs
py 3.12
✕ build_error
2/4 runs
py 3.13
✕ build_error
2/4 runs
py 3.9
✕ build_error
2/4 runs
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

empirical_calibration
import empirical_calibration
import empirical-calibration

This quickstart demonstrates how to use `empirical_calibration` to compute sample weights. It simulates two sets of covariates: `covariates_sample` representing your biased data and `target_covariates` representing the desired distribution (e.g., from a population). The `maybe_exact_calibrate` function then calculates weights for the sample data such that its weighted covariate distribution matches the target distribution as closely as possible, using the specified optimization objective (here, `ENTROPY`).

import numpy as np import pandas as pd import empirical_calibration as ec # Create dummy covariate dataframes for demonstration # In a real scenario, these would come from your biased sample and target population covariates_sample = pd.DataFrame({ 'sex': np.random.choice([0, 1], size=100), 'age': np.random.randint(18, 65, size=100) }) target_covariates = pd.DataFrame({ 'sex': np.random.choice([0, 1], size=1000), 'age': np.random.randint(18, 65, size=1000) }) # Apply empirical calibration to compute weights # Using ENTROPY objective as a common choice try: weights, _ = ec.maybe_exact_calibrate( covariates=covariates_sample, target_covariates=target_covariates, objective=ec.Objective.ENTROPY ) print(f"Successfully computed weights. First 5 weights: {weights[:5]}") print(f"Sum of weights: {np.sum(weights):.2f}") except ec.ConvergenceError as e: print(f"Calibration did not converge: {e}") except Exception as e: print(f"An unexpected error occurred: {e}")
Debug
Known issues
gotchaThe `covariates` and `target_covariates` inputs should typically be pandas DataFrames or numpy arrays with consistent columns and order. Mismatched column names or different data types can lead to unexpected behavior or errors during internal preprocessing and optimization.
fix
Ensure both inputs have identical structure (column names, order, and data types) for the variables intended for calibration. Consider explicit type conversions or column reordering if loading from different sources.
affects: All versions
gotchaThe calibration optimization problem may not always converge, especially with highly disparate covariate distributions, sparse data, or certain objective choices. This results in a `ConvergenceError`.
fix
Consider adjusting the `objective` (e.g., `QUADRATIC` might be more robust for some problems), `epsilon` (tolerance for matching marginals), or `max_iter` parameters in `calibrate` or `maybe_exact_calibrate`. Pre-processing covariates (e.g., binning continuous variables, handling rare categories) can also improve convergence.
affects: All versions
gotcha`empirical-calibration` is a distinct Python library. There is also an R package named 'EmpiricalCalibration' (e.g., by OHDSI) which addresses similar statistical concepts but has a different API and implementation. Do not confuse the two when searching for documentation or examples.
fix
Always verify that documentation or examples found online pertain to the Python `empirical-calibration` library (often associated with `google/empirical_calibration` on GitHub) to avoid applying incorrect API calls or concepts.
affects: All versions
Upgrade
Version history
0.12latest on PyPI · released May 5, 2024
Audit
Dependencies
numpyrequiredFundamental numerical operations and array handling.
pandasrequiredData manipulation, especially for covariate dataframes.
scipyrequiredOptimization routines (e.g., `scipy.optimize`) are used for the convex optimization problem.
scikit-learnrequiredUsed for preprocessing tasks, such as `preprocessing.StandardScaler` for covariates.
patsyrequiredUsed internally for formula-based design matrix creation, though not directly exposed in the main API calls like `calibrate`.
Agent activity
8 hits · last 30 days
node
8
Resources