baycomp

library1.0.3pypypi✓ verified 80d ago

Bayesian estimation and comparison of classifier performance. Provides Bayesian equivalents of frequentist hypothesis tests (sign test, Wilcoxon signed-rank test) via posterior distributions over the difference of two classifiers. Version 1.0.3 supports three tests: sign test, signed-rank test, and correlated t-test. Not actively maintained; last release 2019.

ai-ml testing

pip install baycomp

github ↗homepage ↗

INSTALL

IMPORT

SIG · BAYCOMP

baycomp

ai-mlpythonv1.0.3

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

CorrelatedTTest

✓ from baycomp import CorrelatedTTest

✗ from baycomp import sign_test

SignTest

✓ from baycomp import SignTest

✗ from baycomp import sign_test

SignedRankTest

✓ from baycomp import SignedRankTest

✗ from baycomp import sign_test

Imports baycomp, creates sample accuracy arrays, runs Bayesian sign test with a region of practical equivalence (ROPE) of 0.01, and prints posterior probabilities.

import numpy as np from baycomp import sign_test # Example: two classifiers with 5-fold cross-validation accuracies x = np.array([0.9, 0.85, 0.88, 0.92, 0.87]) y = np.array([0.86, 0.82, 0.84, 0.89, 0.85]) # Bayesian sign test, returns probability that x > y, x < y, and tie region prob_x_better, prob_y_better, prob_tie, region = sign_test(x, y, rope=0.01) print(f"P(x better) = {prob_x_better:.3f}, P(y better) = {prob_y_better:.3f}, P(tie) = {prob_tie:.3f}")

Debug

Known issues

gotchaAll test functions return 4 values: prob_x_better, prob_y_better, prob_tie, region. New users often expect only p-values and miss the last two.

fix

Unpack all four values: prob_x, prob_y, prob_tie, region = test(...)

affects: >=1.0.0

gotchaThe 'rope' parameter is a scalar (absolute difference) for sign_test and signed_rank_test, but for t_test it must be a list/array of two values (lower, upper rope bounds).

fix

For t_test, provide rope as [lower, upper] (e.g., rope=[-0.01, 0.01]). For sign_test et al., provide a single positive number.

affects: 1.0.0 - 1.0.3

gotchaIf samples are 2D (multiple comparisons), functions expect a list of arrays, not a 2D numpy array. Passing a 2D array can silently produce wrong results.

fix

Convert your 2D array to a list of arrays: [arr for arr in matrix]

affects: >=1.0.0

deprecatedLibrary is unmaintained; no new releases since 2019. May have undiscovered bugs. Consider alternatives like BayesianSignTest in scikit-posthocs or manual PyMC3 implementation.

fix

If you need active support, switch to scikit-posthocs (bayesian_sign_test) or implement with PyMC/Stan.

affects: 1.0.3 (latest)

Upgrade

Version history

1.0.3latest on PyPI · released Jul 22, 2023

Audit

Dependencies

No dependency data recorded yet.

Agent activity

21 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/janezd/baycomp.git ↗

homepagehttps://github.com/janezd/baycomp.git ↗