Development Changelog

Development history and milestone releases for vfairness. We are building towards v0.1.0, following Keep a Changelog and Semantic Versioning conventions.

0.0.8
Next Release
Alpha
Status
~45.5k
Lines of Code
Pre-Release Software

vfairness is in active development. APIs may change before the stable v0.1.0 release. We welcome feedback and contributions via GitHub.

Roadmap to v0.1.0

The following features are planned for the stable 0.1.0 release:

Stable API

Finalized, documented API with backward compatibility guarantees.

Complete Test Coverage

Comprehensive unit and integration tests with >90% coverage.

Production Documentation

Full API reference, tutorials, and deployment guides.

PyPI Publication

Official package release on PyPI with all dependencies.

v0.0.8

Current March 2026

Release consolidating all v0.0.x development work into a stable alpha milestone. Will include 43 historical discrimination patterns across US, EU, EU AI Act & Swiss jurisdictions, finalized APIs for the full pipeline, expanded test coverage, and polished documentation across all modules.

Added (May 2026) 6
DisparateImpactRemover (pre-processing transformer)
Feldman et al. (2015) geometric repair: aligns each numeric feature's distribution across protected groups toward a shared per-quantile-median target, removing disparate impact while preserving within-group ranking. Tunable repair_level (0 = no change, 1 = full repair) with a tie-robust empirical-CDF mapping. Registered under dispatch key disparate_impact_removal and wired into the platform's pre-processing step (with a configurable repair-strength control).
DisparateImpactRemover disparate_impact_removal
LabelMassager (pre-processing transformer)
Kamiran & Calders (2012) “massaging”: a logistic ranker scores each instance by proximity to the decision boundary, then the minimal set of borderline labels is flipped (promote deprived-group negatives, demote favored-group positives) to equalize group positive rates. Row-preserving; changes only the target. Exposes the additive get_massaged_labels() hook and a max_flip_fraction safety cap. Wired into the platform's pre-processing step.
LabelMassager label_massaging
Resampler (pre-processing transformer)
Kamiran & Calders (2012) sampling: balances protected-group (and group × label) representation by random over-sampling with replacement up to the largest cell, or under-sampling down to the smallest. Exposes the additive get_resampled_data() hook with configurable strategy and balance_by. The platform rebalances the training split only, leaving the held-out test split intact for honest evaluation.
Resampler resampling
Fairness Regularization (in-processing, platform wiring)
The Navigator's “Fairness Regularization” tile now runs a torch training loop that minimizes BCE + λ · GroupFairnessRegularizer. The fairness metric (demographic parity, equalized odds, equal opportunity, or FPR parity) is selected from the chosen fairness definition, and the penalty strength gives continuous control over the accuracy-fairness trade-off. Registered under dispatch key fairness_regularization; falls back to reweighting if torch is unavailable.
GroupFairnessRegularizer fairness_regularization
FairRepresentationTransformer (pre-processing transformer)
Adversarial fair-representation learning (Zemel et al. 2013 LFR; Louizos et al. 2016 VFAE; Madras et al. 2018 LAFTR): an autoencoder encodes the numeric features into a latent code trained with reconstruction loss plus a gradient-reversal adversary that cannot recover the protected group, yielding a group-invariant representation. transform returns the latent columns (rep_0..). Tunable representation_dim and lambda_fairness; multiple attributes handled via their intersection. Registered as fair_representation and wired into the pre-processing step + Pareto sweep. Requires torch.
FairRepresentationTransformer fair_representation
Intervention hardening: intersectional, secondary constraints, guardrails
Across the intervention suite: (1) DisparateImpactRemover, LabelMassager, Resampler, FairRepresentationTransformer and the fairness-regularization training loop now operate on the intersection of all protected attributes rather than only the first (e.g. they repair / rebalance / penalize race x gender jointly); (2) the regularizer accepts secondary fairness constraints (penalized jointly); (3) a degenerate-collapse guard detects and softens penalties that drive the model to a single class; (4) a training-row cap keeps the torch sweep responsive on large datasets; (5) pre-processing techniques (Disparate Impact Removal, Label Massaging, Resampling, Fair Representation) gained Pareto strength sweeps; and (6) results surface proxy-survival notes and small-sample / confidence warnings so a near-unchanged metric is explained rather than mistaken for a no-op.
get_massaged_labels get_resampled_data secondary_constraints
Planned 9
Historical discrimination patterns
43 patterns (up from 12) with multi-jurisdiction coverage across US, EU, and Swiss contexts.
Attribute × domain historical-pattern cross-walk
New attribute_historical_pattern(attribute, domain) helper resolves 24 documented (attribute class × regulated domain) precedents -- race / gender / age / national-origin / religion / disability / geographic proxy across hiring, lending, healthcare, justice, insurance and education. Pulse attaches the citation-backed envelope onto bias findings so the “Historical pattern” channel keys off a structured flag instead of regex-matching evidence text.
attribute_historical_pattern
EU AI Act patterns
11 patterns: 4 prohibited practices (Art. 5) and 7 high-risk systems (Annex III).
European discrimination patterns
12 patterns covering employment, housing, welfare, policing, and education.
Swiss-specific patterns
8 patterns including Ausweis-based discrimination, naturalisation bias, and cantonal disparities.
EU AI Act compliance detection
Automated risk classification with penalty exposure alerts for prohibited and high-risk AI systems.
CRITICAL HIGH MEDIUM
Penalty exposure alerts
Prohibited practices (€35M / 7% turnover) and high-risk systems (€15M / 3% turnover).
Expanded synthetic demo dataset
19 columns including EU, AI Act, and Swiss-specific features.
Stabilised API surface
Consistent interface across all 6 sub-packages.
Comprehensive documentation
Curated academic references for each jurisdiction.
Added (March 2026) 6
Statistical validation module _statistics.py
Comprehensive uncertainty quantification: bootstrap CIs (percentile, BCa, basic), stratified bootstrap for fairness metrics, Bayesian credible intervals (proportion, difference, mean), multiple testing corrections (Bonferroni, Benjamini-Hochberg FDR), effect sizes (Cohen's d, risk ratio, odds ratio), and auto method selection based on sample size. Zero scipy dependency.
bootstrap_ci stratified_bootstrap_ci bayesian_proportion_ci bayesian_difference_ci compute_metric_with_ci
Proportion tests and power analysis
Two-proportion z-test, Fisher's exact test (exact for n≤200, chi-squared fallback), Cohen's h effect size for binary outcomes, minimum detectable effect computation, and human-readable power warnings for small subgroups.
proportion_z_test fisher_exact_test cohens_h minimum_detectable_effect power_warning
Intersectional disparity SVG template
New ranked-bar chart SVG template (intersectional_disparity.svg) with ground truth overlay, severity-colored bars, prediction delta indicators, most/least advantaged comparison cards, and insight cards. 44th SVG template in the rendering module.
intersectional_disparity_to_svg()
Dual-lens intersectional analysis
identify_privileged_groups() and intersectional_disparity_analysis() now compute ground truth rates, false positive rates, and prediction deltas per subgroup. New outcome_polarity parameter supports both favorable (loan approval) and unfavorable (recidivism) outcome directions.
ground_truth_rate false_positive_rate prediction_delta outcome_polarity
Confidence intervals in structured findings
Intersectional structured findings now include Bayesian credible intervals for small subgroups, enriching compliance reporting with uncertainty quantification.
Improved SVG chart rendering
Enhanced Pareto frontier visuals, improved disparity heatmap layout, and refined calibration adapters for cleaner publication-ready output.
Fixed (March 2026) 2
Validation SVG: raw dict/list strings no longer leak into rendered text
data_validation_to_svg() now safely handles nested dicts and lists, truncates overly long string representations, and only includes scalar values in metric cards. Previously, raw Python repr output (e.g., {'count': 5}) could appear in the SVG text.
Polarity-aware ratio in intersectional insights
Fixed the insight generation to correctly compute advantage ratios when outcome_polarity='positive_unfavorable'. Previously, the ratio calculation was inverted for unfavorable outcomes, making insights misleading.

v0.0.7

Stable February 2026 Show details

Documentation quality overhaul and major feature release. Complete site redesign with gradient theme system, plus new Reporting & Dashboards, Experimentation & A/B Testing, and Workflow Integration modules. 43 SVG templates across 15 adapter modules, 114 gallery illustrations.
Added 18
Reporting module vfairness.operations.reporting
Privacy-preserving queries, Plotly progressive-disclosure views, multi-format NLG output, and interactive Dash/HTML dashboards.
MetricsStore FairnessDashboard ReportGenerator InteractiveDashboard
Experimentation module vfairness.operations.experimentation
A/B testing with intersectional analysis, SPRT early stopping, Pareto frontier and causal decomposition.
FairnessExperiment FairnessPowerAnalyzer ExperimentAnalysis
Monitoring expansion
Wavelet + KS multi-scale drift detection, adaptive thresholds, and alert prioritisation.
FairnessDriftDetector AdaptiveThresholdManager FairnessAlertPrioritizer
Monitoring SVG templates
4 new templates for production monitoring visualisation.
monitoring_dashboard drift_report alert_timeline temporal_analysis
SVG template redesign
All 32 core templates redesigned with consistent card-based layout system and explanation overlays.
SVG gallery page update
Reporting & Experimentation sections, new filter categories, and expanded code examples.
Workflow integration vfairness.operations.cicd
Three-level intersectional gate evaluation, PR-ready markdown reports, and under-powered group detection.
HierarchicalGateConfig FairnessReportCard SmallSampleWarning
W&B integration log_fairness_to_wandb()
Weights & Biases experiment tracking, mirroring the existing MLflow integration.
Auto-logging decorator @auto_log_fairness
Automatic fairness logging during model training with MLflow and W&B backend support.
Pre-commit hooks
Early validation of fairness documentation before commits.
vfairness-check-config vfairness-check-model-card
Formal pytest plugin
Registered via pytest11 entry point with marker, fixture, and terminal summary.
@pytest.mark.fairness fairness_gate
CI/CD configurations
GitHub Actions workflow, GitLab CI config, and PR template with fairness checklist.
fairness-checks.yml
Workflow SVG templates
3 new templates for workflow visualisation.
workflow_overview hierarchical_gate report_card
Getting Started guide expansion
Reporting, Experimentation, and Workflow Integration quick-start sections.
Notebooks
Interactive tutorials for new modules.
reporting_units_3.ipynb experimentation_unit_4.ipynb vfairness_8_workflow_integration.ipynb
8 new SVG templates (35 → 43)
General correlation matrix with mixed methods, causal decomposition, robustness testing, ranking fairness, data validation, auto discovery, regression fairness, and multi-tier reporting dashboard.
correlation_matrix causal_decomposition robustness_testing ranking_fairness data_validation auto_discovery regression_fairness reporting_dashboard
6 new rendering adapter modules
Dedicated adapter files for robustness, ranking, validation, discovery, regression, and reporting SVG templates.
adapters_robustness.py adapters_ranking.py adapters_validation.py adapters_discovery.py adapters_regression.py adapters_reporting.py
Full notebook coverage for all 43 SVG templates
Every SVG adapter function is now exercised in at least one notebook. SVG rendering demo notebook expanded from 35 to 43 templates with new Advanced Modules category.
Changed 7
Public API expansion
200+ exports across all pipeline stages with backward-compatible sys.modules aliasing.
Rendering adapters
All adapters updated for consistent explanation parameter and template context.
Project dependencies pyproject.toml
New module dependencies with Plotly and Dash as optional extras.
SVG gallery
Expanded to 43 templates with 114 gallery illustrations including advanced modules for robustness, ranking, validation, discovery, regression, and reporting.
Documentation overhaul
New Workflow Integration nav link, expanded Business Guide with CI/CD gates and MLOps tracking sections, API Reference with 6 new component pages.
Hero stats updated
Getting Started page statistics refreshed: 60K+ lines of code, 1,211 functions, 217 classes, 43 SVG templates, 114 illustrations.
Business Guide glossary repositioned
Glossary section moved to its correct location before Appendices for better document flow.
Fixed 3
Column label readability in correlation SVG templates
Rotated column labels in correlation_heatmap and correlation_matrix templates were painted over by grid cells. Fixed text-anchor from end to start, increased font size to 12px, and added 100px panel headroom.
Cohen's d lollipop chart alignment in regression fairness SVG
Effect size dots were displaced ∼200px left of the center axis due to mismatched coordinate origins. Recentered dot_x computation around the axis position and added smart label placement for negative values.
SVG template count references
Outdated template counts (30+, 35) across Getting Started, API Reference, and main index pages updated to 43.
Feature Release

Major feature and documentation release. New modules require optional dependencies: pip install vfairness[reporting], pip install vfairness[experimentation], or pip install vfairness[mlops].

v0.0.5-dev

Alpha February 2026 Show details

Post-processing expansion and SVG rendering engine. Added threshold optimisation, prediction reweighting, operations/monitoring modules for production deployment, and 28 purpose-built SVG templates for server-side report generation.
Added 10
Threshold optimisation
Multi-objective threshold tuning for group-aware decisions.
ThresholdOptimizer GroupThresholdOptimizer
Prediction reweighting
Prediction adjustment techniques for fairer outcomes.
PredictionReweighter RejectionOptionClassifier CalibratedEqualizer
Operations module
Production deployment tools for bias validation, gating, and monitoring.
DataBiasValidator ModelFairnessGate BiasMonitor
Drift detection & constraint specification
Alert system for fairness metric drift and declarative constraint types.
DriftAlert FairnessConstraintType
Testing utilities
pytest integration for fairness assertions.
FairnessTestSuite @fairness_test
Rendering sub-package
Core SVG engine, adapters, and 28 purpose-built templates for server-side report generation.
vfairness.rendering
Fairness report SVG adapters
Adapters for classification, regression, and ranking results.
Calibration SVG adapters
Reliability diagrams and disparity plots.
Feature engineering SVG adapters
Correlation matrices and proxy chain visualisation.
Training analysis SVG adapters
Loss curves and constraint satisfaction tracking.
Changed 4
Post-processing restructuring
Reorganised into calibration, threshold_optimization, and reweighting sub-modules.
CI/CD module relocation
Moved from evaluation.integrations to dedicated operations.cicd module.
CalibrationAnalyzer enhancement
Pareto frontier and impossibility theorem diagnostics.
Rendering / visualisation separation
Plotly/Matplotlib for interactive use, SVG engine for static reports.

v0.0.3-dev

Alpha January – February 2026 Show details

Full pipeline expansion. Post-processing calibration, feature engineering, in-processing training interventions, and enhanced visualisation. Expanded vfairness from detection-only to a full-pipeline fairness toolkit.
Added 14
FairExplAIner module
Human-readable explanations for all fairness metrics.
FairExplAIner
Calibration module
Group-aware probability calibration with 5 methods, intersectional calibration, and Pareto trade-off analysis.
GroupCalibrator IntersectionalCalibrator CalibrationAnalyzer
Feature engineering sub-package
Fairness-aware data transformation, 5 transformers, and proxy chain detection via mutual information.
vfairness.preprocessing.feature_engineering FeatureEngineeringAnalyzer
In-processing module
Complete training-time intervention toolkit with 12 loss functions, 5 constraint types, and 5 regularizers.
vfairness.in_processing ExponentiatedGradient FairClassifier
Trainable group calibrators
5 calibration methods for per-group adjustment during training.
TemperatureScaling PlattScaling TrainableGroupCalibrator
Enhanced visualisation suite
Interactive Plotly charts, publication-ready matplotlib outputs, reliability diagrams, and disparity dashboards.
Bayesian confidence intervals
Conjugate priors for small sample sizes (n < 30).
Auto-discovery of protected attributes
Automatic detection from dataset column names and metadata.
Geographic bias detection
Location-based disparity analysis.
Quick analysis function
Convenience function for rapid fairness analysis.
classification_fairness_report()
Multi-class classification support
Fairness metrics extended to multi-class settings.
Training analyser
Monitoring fairness metrics during model training.
FairnessTrainingAnalyzer
Documentation site
Comprehensive API reference, tutorials, and 6 Jupyter demo notebooks.
scikit-learn wrappers
Drop-in compatible classifiers and regressors.
FairClassifier FairRegressor
Changed 5
Package reorganisation
6 top-level sub-packages: preprocessing, in_processing, post_processing, evaluation, operations, rendering.
Library scope expansion
From detection/measurement to full-pipeline fairness: detect, mitigate, calibrate, monitor.
Bootstrap confidence intervals
Bias-corrected and accelerated (BCa) method for improved accuracy.
MLflow integration
Automatic artifact logging and metric tracking.
FairnessAnalyzer refactoring
Better memory efficiency with large datasets.
Performance 3
Bootstrap computation
40% faster through vectorized operations.
Intersectional analysis
60% reduced memory footprint.
Permutation tests
Parallel processing support.
Fixed 3
Demographic parity edge case
Fixed calculation when one group has zero positive predictions.
Calibration numerical stability
Resolved issues with very small probabilities.
pytest assertion messages
Clearer failure diagnostics.
Scope Change

vfairness is no longer detection-only. With in-processing and post-processing modules, it now covers the full ML fairness pipeline.

v0.0.1-alpha

Alpha December 2025 Show details

Initial development release. Core fairness metrics, statistical validation, bias detection module, and MLOps integration foundations.
Added 16
Core fairness analyser
Classification fairness analysis with demographic parity, equal opportunity, and equalized odds.
FairnessAnalyzer
Calibration & disparate impact metrics
Calibration difference, predictive parity, and 80% rule calculation.
Intersectional analysis
Multi-attribute fairness evaluation with automatic subgroup discovery.
Regression fairness metrics
Error rate disparity and prediction gap analysis.
Statistical validation
Bootstrap confidence intervals, permutation testing, and multiple testing corrections (Bonferroni, Benjamini-Hochberg).
Effect size calculations
Cohen's d, risk ratio, and odds ratio.
Bias detection module
Comprehensive data-level bias analysis with historical pattern detection and representation bias.
BiasDetector
Proxy variable identification
Correlation and mutual information-based detection.
MLflow integration
Native fairness logging for experiment tracking.
log_fairness_to_mlflow()
pytest plugin
Fairness assertions in CI/CD pipelines.
Ranking fairness metrics
NDKL, skew, and exposure disparity.
Training callbacks
Framework-specific monitoring hooks for Keras, PyTorch, and scikit-learn.
Basic visualisation
Matplotlib bar charts and heatmaps.
Report export
JSON, dictionary, and structured output formats.
Sample size warnings
Recommendations for adequate statistical power.
Unified API
Consistent interface across classification, regression, and ranking analysers.
Fixed 3
NaN handling
Fixed incorrect handling of NaN values in sensitive attribute columns.
Memory leak fix
Resolved leak in iterative confidence interval computation.
Effect size calculations
Corrected calculations for imbalanced groups.
Initial Development Release

First public development release. Not recommended for production use. Install from source or GitHub.

Future Plans (Post v0.1.0)

Planned features for releases after the stable v0.1.0:

v0.1.x Series

  • Causal fairness metrics
  • Model-agnostic explanations
  • Fairness-aware model selection

v0.2.x Series

  • LLM fairness evaluation
  • Continuous monitoring dashboard
  • Enterprise compliance reports
Contributing

Found a bug or have a feature request? Please open an issue on GitHub. We welcome contributions from the community.

← Academic References Home →