Development Changelog

Development history and milestone releases for vfairness. Version 0.1.0, cut on 2026-08-23, is the first public beta; its v0.1.0 tag triggers the TestPyPI-then-PyPI publish, which is the pending release step. The API is not frozen until the stable 1.0.0. We follow Keep a Changelog and Semantic Versioning conventions.

0.1.0
Current Release
Beta
Status
~97k
Lines of Code
Beta Software

vfairness 0.1.0 is the first public beta. The release is cut and pip install vfairness is the intended install command, with the first PyPI publish as the pending release step. The API is not frozen until the stable 1.0.0 release. We welcome feedback and contributions via GitHub.

Roadmap to 1.0.0

The following features are planned for the stable 1.0.0 release:

Stable API

Finalized, documented API with backward compatibility guarantees.

Complete Test Coverage

Comprehensive unit and integration tests with >90% coverage.

Production Documentation

Full API reference, tutorials, and deployment guides.

PyPI Publication

Cut with the 0.1.0 beta; the v0.1.0 tag triggers the TestPyPI-then-PyPI publish (the pending release step). 1.0.0 will be the first stable release under the backwards-compatibility contract.

Unreleased

Unreleased August 27, 2026

The [Unreleased] block of CHANGELOG.md: two library-polish improvements that landed after the 0.1.0 cut.

Added 1
Typed report contract (FairnessReport)
The report returned by get_report, classification_fairness_report and regression_fairness_report is now documented and type-checked by a set of TypedDict definitions (FairnessReport plus AssessmentReport, DataInfo, ExplanationsReport, MetricStatusEntry and InsufficientEvidenceGroup), re-exported from the top-level vfairness package and from vfairness.evaluation and added to the frozen public surface. These are static-only annotations: the report stays the same plain, JSON-serialisable dict, so every existing consumer, subscript access and json.dumps is unchanged; a type checker now catches a mistyped key such as report["assessement"].
FairnessReport AssessmentReport DataInfo ExplanationsReport MetricStatusEntry InsufficientEvidenceGroup
Changed 1
Fairness radar chart plots fairness outward
The spider chart previously plotted raw disparity on a tight adaptive axis, so fair metrics collapsed into an unreadable blob near the centre. Each metric now maps to a per-metric fairness score on a fixed 0-to-1 axis: 1.0 (fully fair) at the rim, 0.0 (unfair) at the centre, honouring metric direction (disparate impact and the other *_ratio metrics are higher-is-better). A fair model therefore draws a large round shape and a failing metric caves inward, and every threshold maps to a single clean reference ring (a dot outside the ring passes, inside it fails). Raw values stay on the dot labels; the auto-generated radar explanation was rewritten to match. The interactive Plotly radar plot_metrics_radar was brought into line with the same fairness axis, so the SVG and interactive radars now read the same way, and the docs-site gallery visual was regenerated.
radar_chart_to_svg plot_metrics_radar

v0.1.0

Beta August 23, 2026

The first public beta: a plain SemVer 0.x release carrying the Development Status :: 4 - Beta classifier, installable with a normal pip install vfairness. This section is the release cut; the v0.1.0 tag triggers the TestPyPI-then-PyPI publish. The API is not frozen until 1.0.0. This section highlights the [0.1.0] block of CHANGELOG.md.

Added 6
Output branding switch
New public set_branding() and branding_enabled() remove the validant.ai mark from generated SVGs and reports, honouring an explicit call, then the VFAIRNESS_BRANDING environment variable, then a branded default. A single chart can opt out with render_svg(name, {"branding": False}). An unbranded chart contains no occurrence of the brand string at all, and the switch is unconditional by design: nothing verifies a licence or calls home.
set_branding branding_enabled
SSRF egress guard (vfairness.net)
A new cross-cutting infrastructure sub-package. Wired into the LLM api_proxy first; coverage of the remaining outbound call sites is follow-up work. validate_endpoint resolves and vets the target, guarded_post is a drop-in for requests.post, PinnedIPAdapter pins the vetted IP against DNS rebinding, and a refusal raises SSRFError. Wired into the LLM api_proxy, so assessment traffic cannot be pointed at loopback, private, link-local or cloud-metadata targets. A local model server needs the explicit allow_loopback=True opt-in, which unlocks loopback only.
vfairness.net validate_endpoint guarded_post PinnedIPAdapter SSRFError
Groundedness axis (vfairness.validity)
A public sub-package for the validity / groundedness axis: the VG_* metric identifiers (faithfulness, groundedness, hallucination rate, citation accuracy, context precision and recall, answer correctness), a GroundednessScorer, a fail-closed judge ladder (GroundednessJudge / LlmGroundednessJudge) and aggregate_validity. Deliberately not registered in the capability manifest yet, so nothing can be sealed on it before its gold set lands.
vfairness.validity GroundednessScorer aggregate_validity
Statistical-robustness layer for the sealed spec-v2 gates
A gate now passes on affirmative confidence-interval evidence (one-sided equivalence / TOST), never on a point estimate. New *_with_ci variants cover disparate impact, FPR parity, negative predictive value, conditional demographic disparity, pricing disparity, the integrated calibration index and multicalibration. Custom-shape statistics get an interval from a new bootstrap_over_index helper that resamples row indices stratified by group. The interval width feeds the three-state verdict.
bootstrap_over_index disparate_impact_ratio_with_ci multicalibration_with_ci
Spec-v2 metric panel: independence, sufficiency, calibration, decision utility
New spine primaries and diagnostics: disparate_impact_ratio (the four-fifths selection-rate ratio), conditional_demographic_disparity (the CJEU objective-justification mirror), negative_predictive_value_difference, auroc_parity (a validity gate against levelling-down), grid-free integrated_calibration_index and multicalibration, the TRIPOD recalibration diagnostics calibration_in_the_large and calibration_slope, plus pricing_disparity, net_benefit_parity and conditional_adverse_impact. The error-rate metrics the catalogue referenced but the engine lacked (fpr_parity_difference, fnr_parity_difference, accuracy_parity_difference, worst_group_accuracy) also landed, closing the overclaims found in the knowledge-graph spine audit.
Blocking lint and type gates, mutation testing, governance
ruff check src tests and mypy src are both blocking on every library change, with the toolchain declared in the dev extra so CI, pre-commit and local runs agree. The type burndown reached zero errors and surfaced three real bugs (a dead subgroup-robustness audit in Pulse, a crash on list-valued sample_weight in FairRegressor.fit, and a no-op correlation proxy detector). Also added: mutation testing on the band logic, new property-based / determinism / deserialization-safety / reference-parity suites, an sdist allowlist, beta-programme materials, and the supply-chain governance files.
Changed 2
Licence moved from MIT to Apache-2.0
The library ships under Apache-2.0 (copyright Glinz & Company GmbH), with a NOTICE file alongside the LICENSE.
Python floor raised to 3.11
requires-python is now >=3.11, following Scientific Python SPEC 0.
Fixed 3
An undefined group rate is insufficient evidence, not 0.0 parity
predictive_parity_difference, fpr_parity_difference, fnr_parity_difference and equalized_odds_difference collapsed a NaN per-group rate (a group with no positive predictions, no negatives, or no positives) into a 0.0 “perfect parity” result, which read as PASS with a deceptively tight [0.0, 0.0] bootstrap CI. They now return NaN so the verdict routes to insufficient_evidence, mirroring the earlier NPV fix.
False PASS on unmeasurable spec-v2 gates
Four of the new sealed metrics collapsed an unmeasurable result to the numeric “perfectly fair” 0.0 instead of NaN, bypassing both the report's not-assessable guard and the bootstrap NaN filter, so a no-evidence case could seal as fair. multicalibration, negative_predictive_value_difference, auroc_parity and conditional_demographic_disparity now return NaN when there is no evidence, and the last of those renormalises its stratum weights instead of biasing toward “fair” in proportion to the un-assessable population.
Verdicts derive from the interval, not the point estimate
MetricResult gains a three-state verdict (fair / unfair / insufficient_evidence), and is_fair becomes an alias for verdict == 'fair'. A wide small-sample interval is no longer reported as a pass or a fail. predictive_parity_difference_with_ci closes the last sufficiency metric that shipped as a bare point estimate.
Security 2
No implicit deserialization of untrusted models
The XAI sidecar refuses to deserialize a model payload unless the caller opts in explicitly (trust_input / VFAIRNESS_TRUST_MODEL_INPUT), closing a remote-code-execution vector. An AST guard test also stops the metric core from swallowing computation errors silently.
Defence in depth on agent- and artifact-supplied paths
The MCP load_dataframe tool rejects remote data_path URLs, so an agent-supplied path cannot become network egress; the Pulse artifact download pins the scheme to https and caps the response size; and the deserialization-safety scan now resolves aliased imports and covers marshal, pandas.read_pickle, jsonpickle and shelve.

v0.0.9

Alpha August 2026

A deep correctness-and-honesty hardening milestone. Five audit waves swept every subsystem for silently-wrong results, false fairness certificates and band inconsistencies; the rendering layer gained self-explaining, accessible SVGs and a single unified design language (Blanco); and the test/CI infrastructure was strengthened with golden-file value pins, render-smoke coverage across every template, and coverage measurement in CI. This was the last 0.0.x milestone before the 0.1.0 public beta.

Added 5
Self-explaining, accessible SVGs
Every rendered SVG now carries its own explanation automatically, wired at the render_svg choke point (so every adapter and .to_svg() inherits it): an auto-generated on-canvas explanation (concept → how to read it → the finding on this chart → recommended action) plus an accessible, machine-readable layer (role="img", <title>, a one-line <desc>, and a <metadata> JSON block). Severity is derived from the same data the badge uses, so the explanation can never contradict the badge. Curated content covers every template.
vfairness.rendering.explain CHART_META
Blanco — the one SVG design language
render_svg applies Blanco (“The Silent Gallery”) by default: a single post-render transform giving sharp corners, editorial type (Plus Jakarta Sans / JetBrains Mono), desaturated graphite chrome, and colour reserved for the four semantic tones. It never touches the templates or adapters themselves.
apply_skin list_skins SKINS
Golden-file regression tests
Exact-value pins for the core classification, regression and ranking metrics and the FairnessAnalyzer report, so a numeric regression fails loudly instead of drifting silently.
tests/test_golden_metrics.py
Render-smoke coverage for every template
Drives render_svg for every registered SVG template and asserts well-formed XML with no unrendered Jinja tokens.
tests/test_rendering.py
Task result envelope & coverage-in-CI
New TaskResult dataclass formalises the task-handler result envelope (schema_version, task_type, success, data/error, optional warnings) with a backward-compatible to_dict(). Coverage is now measured in CI (--cov=vfairness) behind a ratcheting --cov-fail-under gate.
vfairness.result.TaskResult
Changed 3
SVG gallery is all-Blanco
The gallery renders every example in Blanco; the Original/Blanco skin switch and the parallel mirror were removed, along with the coloured category-icon boxes and the unused design-proposal SVGs.
CI runs the full test suite
The -m fairness marker matched zero tests (so CI silently ran nothing); the filter was removed from both GitHub and GitLab pipelines, and both now install the extras the suite exercises.
Task handlers use the result envelope
The CI/CD and Pulse handlers build their result via TaskResult.to_dict(). The success/data/error semantics are unchanged; the envelope gains additive schema_version and task_type keys.
Fixed — deep-audit Waves 1–5 7
Undefined metrics are not failures
A metric that returns NaN (undefined, e.g. an error-rate metric when a group has no positive labels, or R² when a group has constant y_true) is now surfaced in assessment.not_assessable_metrics and excluded from the fairness score, instead of being mis-scored as a FAIL. The classification report gains a not_assessable_metrics field.
Evaluation metrics & statistical honesty
Effect sizes now measure the gap the metric actually reports; equal opportunity returns NaN with a warning (not a silent “fair” 0.0) when a group has no positives; dropped small groups are named; fisher_exact_test uses the real exact test (no silent chi-square degrade); Bayesian credible intervals delegate to scipy (the homegrown Beta PPF had collapsed “95%” intervals to ~55% coverage); Wilson score intervals replace NaN bounds for large groups; multiple-testing correction runs real two-proportion z-tests instead of inverted pseudo p-values.
No more false fairness certificates
Degenerate data (0 samples or a single group) is marked NOT ASSESSABLE instead of certifying fairness_score 1.0; critical findings floor the overall risk band (1+ → MEDIUM, 3+ → HIGH) so three critical issues can no longer average out to a green badge; constant-probability ECE/MCE report the true error instead of a “perfect” 0.0.
Band consistency & explainer coherence
A single canonical vfairness._bands module drives every severity/threshold decision, so the badge, radar chart, detailed report and explanation text can no longer give contradictory verdicts for the same input; the divergent 0.2/0.4/0.6/0.8 risk scale was eliminated everywhere; NaN scores render a neutral N/A badge rather than a green MINIMAL.
In-, post- and pre-processing correctness
Threshold optimization and three of five reweighters crashed on every fit — both fixed; BetaCalibrator is now real Kull et al. 2017 beta calibration; GridSearch predicts with the classifier it reports; FairRegressor raises instead of silently fitting an unconstrained model for unsupported objectives; the adversarial debiasing loss trains its adversary in the correct direction; the outcome column is no longer scanned as its own proxy.
Rendering, XAI, Pulse & ops robustness
Hostile inputs (empty modules, None nests, stringified numbers, Infinity) render or raise a clear error naming the template instead of leaking ZeroDivision/Overflow; falsy-zero bugs (score 0, zero-tolerance thresholds) read at the correct severity; deep-model XAI routing selects Integrated Gradients (Phase-2 DeepExplainer had made every deep job fail); SHAP attributions, base value and prediction now come from one class; causal identification and refuter-crash disclosure fixed; monitoring/drift windows and health-score trends corrected.
Packaging, frozen API surface & test hardening
requirements.txt now includes the hard runtime deps requests and scikit-learn (a minimal install previously could not even import vfairness); the mcp extra is gated to Python ≥ 3.10 so the project locks again; frozen-surface exports (calibration_difference, r2_parity_difference, residual_bias) restored; ~800 new regression tests across tests/test_audit_wave1–5 pin every fix. Documentation examples corrected against the real APIs.

v0.0.8

Alpha March 2026

Release consolidating all v0.0.x development work into a stable alpha milestone. Includes 43 historical discrimination patterns across US, EU, EU AI Act & Swiss jurisdictions, finalized APIs for the full pipeline, expanded test coverage, and polished documentation across all modules.

Added (May 2026) 6
DisparateImpactRemover (pre-processing transformer)
Feldman et al. (2015) geometric repair: aligns each numeric feature's distribution across protected groups toward a shared per-quantile-median target, removing disparate impact while preserving within-group ranking. Tunable repair_level (0 = no change, 1 = full repair) with a tie-robust empirical-CDF mapping. Registered under dispatch key disparate_impact_removal and wired into the platform's pre-processing step (with a configurable repair-strength control).
DisparateImpactRemover disparate_impact_removal
LabelMassager (pre-processing transformer)
Kamiran & Calders (2012) “massaging”: a logistic ranker scores each instance by proximity to the decision boundary, then the minimal set of borderline labels is flipped (promote deprived-group negatives, demote favored-group positives) to equalize group positive rates. Row-preserving; changes only the target. Exposes the additive get_massaged_labels() hook and a max_flip_fraction safety cap. Wired into the platform's pre-processing step.
LabelMassager label_massaging
Resampler (pre-processing transformer)
Kamiran & Calders (2012) sampling: balances protected-group (and group × label) representation by random over-sampling with replacement up to the largest cell, or under-sampling down to the smallest. Exposes the additive get_resampled_data() hook with configurable strategy and balance_by. The platform rebalances the training split only, leaving the held-out test split intact for honest evaluation.
Resampler resampling
Fairness Regularization (in-processing, platform wiring)
The Navigator's “Fairness Regularization” tile now runs a torch training loop that minimizes BCE + λ · GroupFairnessRegularizer. The fairness metric (demographic parity, equalized odds, equal opportunity, or FPR parity) is selected from the chosen fairness definition, and the penalty strength gives continuous control over the accuracy-fairness trade-off. Registered under dispatch key fairness_regularization; falls back to reweighting if torch is unavailable.
GroupFairnessRegularizer fairness_regularization
FairRepresentationTransformer (pre-processing transformer)
Adversarial fair-representation learning (Zemel et al. 2013 LFR; Louizos et al. 2016 VFAE; Madras et al. 2018 LAFTR): an autoencoder encodes the numeric features into a latent code trained with reconstruction loss plus a gradient-reversal adversary that cannot recover the protected group, yielding a group-invariant representation. transform returns the latent columns (rep_0..). Tunable representation_dim and lambda_fairness; multiple attributes handled via their intersection. Registered as fair_representation and wired into the pre-processing step + Pareto sweep. Requires torch.
FairRepresentationTransformer fair_representation
Intervention hardening: intersectional, secondary constraints, guardrails
Across the intervention suite: (1) DisparateImpactRemover, LabelMassager, Resampler, FairRepresentationTransformer and the fairness-regularization training loop now operate on the intersection of all protected attributes rather than only the first (e.g. they repair / rebalance / penalize race x gender jointly); (2) the regularizer accepts secondary fairness constraints (penalized jointly); (3) a degenerate-collapse guard detects and softens penalties that drive the model to a single class; (4) a training-row cap keeps the torch sweep responsive on large datasets; (5) pre-processing techniques (Disparate Impact Removal, Label Massaging, Resampling, Fair Representation) gained Pareto strength sweeps; and (6) results surface proxy-survival notes and small-sample / confidence warnings so a near-unchanged metric is explained rather than mistaken for a no-op.
get_massaged_labels get_resampled_data secondary_constraints
Planned (all shipped in this release) 9
Historical discrimination patterns
43 patterns (up from 12) with multi-jurisdiction coverage across US, EU, and Swiss contexts.
Attribute × domain historical-pattern cross-walk
New attribute_historical_pattern(attribute, domain) helper resolves 24 documented (attribute class × regulated domain) precedents -- race / gender / age / national-origin / religion / disability / geographic proxy across hiring, lending, healthcare, justice, insurance and education. Pulse attaches the citation-backed envelope onto bias findings so the “Historical pattern” channel keys off a structured flag instead of regex-matching evidence text.
attribute_historical_pattern
EU AI Act patterns
11 patterns: 4 prohibited practices (Art. 5) and 7 high-risk systems (Annex III).
European discrimination patterns
12 patterns covering employment, housing, welfare, policing, and education.
Swiss-specific patterns
8 patterns including Ausweis-based discrimination, naturalisation bias, and cantonal disparities.
EU AI Act compliance detection
Automated risk classification with penalty exposure alerts for prohibited and high-risk AI systems.
CRITICAL HIGH MEDIUM
Penalty exposure alerts
Prohibited practices (€35M / 7% turnover) and high-risk systems (€15M / 3% turnover).
Expanded synthetic demo dataset
19 columns including EU, AI Act, and Swiss-specific features.
Stabilised API surface
Consistent interface across all 6 sub-packages (as of this release; the taxonomy is 15 top-level sub-packages as of 0.1.0).
Comprehensive documentation
Curated academic references for each jurisdiction.
Added (March 2026) 6
Statistical validation module _statistics.py
Comprehensive uncertainty quantification: bootstrap CIs (percentile, BCa, basic), stratified bootstrap for fairness metrics, Bayesian credible intervals (proportion, difference, mean), multiple testing corrections (Bonferroni, Benjamini-Hochberg FDR), effect sizes (Cohen's d, risk ratio, odds ratio), and auto method selection based on sample size. Zero scipy dependency. (Superseded in v0.0.9: the homegrown scipy-free Beta PPF was found to collapse "95%" credible intervals to about 55% coverage, so it now delegates to scipy, which is a hard runtime dependency.)
bootstrap_ci stratified_bootstrap_ci bayesian_proportion_ci bayesian_difference_ci compute_metric_with_ci
Proportion tests and power analysis
Two-proportion z-test, Fisher's exact test (exact for n≤200, chi-squared fallback), Cohen's h effect size for binary outcomes, minimum detectable effect computation, and human-readable power warnings for small subgroups.
proportion_z_test fisher_exact_test cohens_h minimum_detectable_effect power_warning
Intersectional disparity SVG template
New ranked-bar chart SVG template (intersectional_disparity.svg) with ground truth overlay, severity-colored bars, prediction delta indicators, most/least advantaged comparison cards, and insight cards. 44th SVG template in the rendering module.
intersectional_disparity_to_svg()
Dual-lens intersectional analysis
identify_privileged_groups() and intersectional_disparity_analysis() now compute ground truth rates, false positive rates, and prediction deltas per subgroup. New outcome_polarity parameter supports both favorable (loan approval) and unfavorable (recidivism) outcome directions.
ground_truth_rate false_positive_rate prediction_delta outcome_polarity
Confidence intervals in structured findings
Intersectional structured findings now include Bayesian credible intervals for small subgroups, enriching compliance reporting with uncertainty quantification.
Improved SVG chart rendering
Enhanced Pareto frontier visuals, improved disparity heatmap layout, and refined calibration adapters for cleaner publication-ready output.
Fixed (March 2026) 2
Validation SVG: raw dict/list strings no longer leak into rendered text
data_validation_to_svg() now safely handles nested dicts and lists, truncates overly long string representations, and only includes scalar values in metric cards. Previously, raw Python repr output (e.g., {'count': 5}) could appear in the SVG text.
Polarity-aware ratio in intersectional insights
Fixed the insight generation to correctly compute advantage ratios when outcome_polarity='positive_unfavorable'. Previously, the ratio calculation was inverted for unfavorable outcomes, making insights misleading.

v0.0.7

Alpha February 2026 Show details

Documentation quality overhaul and major feature release. Complete site redesign with gradient theme system, plus new Reporting & Dashboards, Experimentation & A/B Testing, and Workflow Integration modules. 43 SVG templates across 15 adapter modules, 114 gallery illustrations.
Added 18
Reporting module vfairness.operations.reporting
Privacy-preserving queries, Plotly progressive-disclosure views, multi-format NLG output, and interactive Dash/HTML dashboards.
MetricsStore FairnessDashboard ReportGenerator InteractiveDashboard
Experimentation module vfairness.operations.experimentation
A/B testing with intersectional analysis, SPRT early stopping, Pareto frontier and causal decomposition.
FairnessExperiment FairnessPowerAnalyzer ExperimentAnalysis
Monitoring expansion
Wavelet + KS multi-scale drift detection, adaptive thresholds, and alert prioritisation.
FairnessDriftDetector AdaptiveThresholdManager FairnessAlertPrioritizer
Monitoring SVG templates
4 new templates for production monitoring visualisation.
monitoring_dashboard drift_report alert_timeline temporal_analysis
SVG template redesign
All 32 core templates redesigned with consistent card-based layout system and explanation overlays.
SVG gallery page update
Reporting & Experimentation sections, new filter categories, and expanded code examples.
Workflow integration vfairness.operations.cicd
Three-level intersectional gate evaluation, PR-ready markdown reports, and under-powered group detection.
HierarchicalGateConfig FairnessReportCard SmallSampleWarning
W&B integration log_fairness_to_wandb()
Weights & Biases experiment tracking, mirroring the existing MLflow integration.
Auto-logging decorator @auto_log_fairness
Automatic fairness logging during model training with MLflow and W&B backend support.
Pre-commit hooks
Early validation of fairness documentation before commits.
vfairness-check-config vfairness-check-model-card
Formal pytest plugin
Registered via pytest11 entry point with marker, fixture, and terminal summary.
@pytest.mark.fairness fairness_gate
CI/CD configurations
GitHub Actions workflow, GitLab CI config, and PR template with fairness checklist.
fairness-checks.yml
Workflow SVG templates
3 new templates for workflow visualisation.
workflow_overview hierarchical_gate report_card
Getting Started guide expansion
Reporting, Experimentation, and Workflow Integration quick-start sections.
Notebooks
Interactive tutorials for new modules.
reporting_units_3.ipynb experimentation_unit_4.ipynb vfairness_8_workflow_integration.ipynb
8 new SVG templates (35 → 43)
General correlation matrix with mixed methods, causal decomposition, robustness testing, ranking fairness, data validation, auto discovery, regression fairness, and multi-tier reporting dashboard.
correlation_matrix causal_decomposition robustness_testing ranking_fairness data_validation auto_discovery regression_fairness reporting_dashboard
6 new rendering adapter modules
Dedicated adapter files for robustness, ranking, validation, discovery, regression, and reporting SVG templates.
adapters_robustness.py adapters_ranking.py adapters_validation.py adapters_discovery.py adapters_regression.py adapters_reporting.py
Full notebook coverage for all 43 SVG templates
Every SVG adapter function is now exercised in at least one notebook. SVG rendering demo notebook expanded from 35 to 43 templates with new Advanced Modules category.
Changed 7
Public API expansion
200+ exports across all pipeline stages with backward-compatible sys.modules aliasing.
Rendering adapters
All adapters updated for consistent explanation parameter and template context.
Project dependencies pyproject.toml
New module dependencies with Plotly and Dash as optional extras.
SVG gallery
Expanded to 43 templates with 114 gallery illustrations including advanced modules for robustness, ranking, validation, discovery, regression, and reporting.
Documentation overhaul
New Workflow Integration nav link, expanded Business Guide with CI/CD gates and MLOps tracking sections, API Reference with 6 new component pages.
Hero stats updated
Getting Started page statistics refreshed: 60K+ lines of code, 1,211 functions, 217 classes, 43 SVG templates, 114 illustrations. (Counts as of v0.0.7, measured as raw definitions; the current stats count registered public capabilities instead: 85 functions, 91 classes, 44 SVG templates as of 0.1.0.)
Business Guide glossary repositioned
Glossary section moved to its correct location before Appendices for better document flow.
Fixed 3
Column label readability in correlation SVG templates
Rotated column labels in correlation_heatmap and correlation_matrix templates were painted over by grid cells. Fixed text-anchor from end to start, increased font size to 12px, and added 100px panel headroom.
Cohen's d lollipop chart alignment in regression fairness SVG
Effect size dots were displaced ∼200px left of the center axis due to mismatched coordinate origins. Recentered dot_x computation around the axis position and added smart label placement for negative values.
SVG template count references
Outdated template counts (30+, 35) across Getting Started, API Reference, and main index pages updated to 43.
Feature Release

Major feature and documentation release. New modules require optional dependencies: pip install vfairness[reporting], pip install vfairness[experimentation], or pip install vfairness[mlops].

v0.0.5-dev

Alpha February 2026 Show details

Post-processing expansion and SVG rendering engine. Added threshold optimisation, prediction reweighting, operations/monitoring modules for production deployment, and 28 purpose-built SVG templates for server-side report generation.
Added 10
Threshold optimisation
Multi-objective threshold tuning for group-aware decisions.
ThresholdOptimizer GroupThresholdOptimizer
Prediction reweighting
Prediction adjustment techniques for fairer outcomes.
PredictionReweighter RejectionOptionClassifier CalibratedEqualizer
Operations module
Production deployment tools for bias validation, gating, and monitoring.
DataBiasValidator ModelFairnessGate BiasMonitor
Drift detection & constraint specification
Alert system for fairness metric drift and declarative constraint types.
DriftAlert FairnessConstraintType
Testing utilities
pytest integration for fairness assertions.
FairnessTestSuite @fairness_test
Rendering sub-package
Core SVG engine, adapters, and 28 purpose-built templates for server-side report generation.
vfairness.rendering
Fairness report SVG adapters
Adapters for classification, regression, and ranking results.
Calibration SVG adapters
Reliability diagrams and disparity plots.
Feature engineering SVG adapters
Correlation matrices and proxy chain visualisation.
Training analysis SVG adapters
Loss curves and constraint satisfaction tracking.
Changed 4
Post-processing restructuring
Reorganised into calibration, threshold_optimization, and reweighting sub-modules.
CI/CD module relocation
Moved from evaluation.integrations to dedicated operations.cicd module.
CalibrationAnalyzer enhancement
Pareto frontier and impossibility theorem diagnostics.
Rendering / visualisation separation
Plotly/Matplotlib for interactive use, SVG engine for static reports.

v0.0.3-dev

Alpha January – February 2026 Show details

Full pipeline expansion. Post-processing calibration, feature engineering, in-processing training interventions, and enhanced visualisation. Expanded vfairness from detection-only to a full-pipeline fairness toolkit.
Added 14
FairExplAIner module
Human-readable explanations for all fairness metrics.
FairExplAIner
Calibration module
Group-aware probability calibration with 5 methods, intersectional calibration, and Pareto trade-off analysis.
GroupCalibrator IntersectionalCalibrator CalibrationAnalyzer
Feature engineering sub-package
Fairness-aware data transformation, 5 transformers, and proxy chain detection via mutual information.
vfairness.preprocessing.feature_engineering FeatureEngineeringAnalyzer
In-processing module
Complete training-time intervention toolkit with 12 loss functions, 5 constraint types, and 5 regularizers.
vfairness.in_processing ExponentiatedGradient FairClassifier
Trainable group calibrators
5 calibration methods for per-group adjustment during training.
TemperatureScaling PlattScaling TrainableGroupCalibrator
Enhanced visualisation suite
Interactive Plotly charts, publication-ready matplotlib outputs, reliability diagrams, and disparity dashboards.
Bayesian confidence intervals
Conjugate priors for small sample sizes (n < 30).
Auto-discovery of protected attributes
Automatic detection from dataset column names and metadata.
Geographic bias detection
Location-based disparity analysis.
Quick analysis function
Convenience function for rapid fairness analysis.
classification_fairness_report()
Multi-class classification support
Fairness metrics extended to multi-class settings.
Training analyser
Monitoring fairness metrics during model training.
FairnessTrainingAnalyzer
Documentation site
Comprehensive API reference, tutorials, and 6 Jupyter demo notebooks.
scikit-learn wrappers
Drop-in compatible classifiers and regressors.
FairClassifier FairRegressor
Changed 5
Package reorganisation
6 top-level sub-packages: preprocessing, in_processing, post_processing, evaluation, operations, rendering.
Library scope expansion
From detection/measurement to full-pipeline fairness: detect, mitigate, calibrate, monitor.
Bootstrap confidence intervals
Bias-corrected and accelerated (BCa) method for improved accuracy.
MLflow integration
Automatic artifact logging and metric tracking.
FairnessAnalyzer refactoring
Better memory efficiency with large datasets.
Performance 3
Bootstrap computation
40% faster through vectorized operations.
Intersectional analysis
60% reduced memory footprint.
Permutation tests
Parallel processing support.
Fixed 3
Demographic parity edge case
Fixed calculation when one group has zero positive predictions.
Calibration numerical stability
Resolved issues with very small probabilities.
pytest assertion messages
Clearer failure diagnostics.
Scope Change

vfairness is no longer detection-only. With in-processing and post-processing modules, it now covers the full ML fairness pipeline.

v0.0.1-alpha

Alpha December 2025 Show details

Initial development release. Core fairness metrics, statistical validation, bias detection module, and MLOps integration foundations.
Added 16
Core fairness analyser
Classification fairness analysis with demographic parity, equal opportunity, and equalized odds.
FairnessAnalyzer
Calibration & disparate impact metrics
Calibration difference, predictive parity, and 80% rule calculation.
Intersectional analysis
Multi-attribute fairness evaluation with automatic subgroup discovery.
Regression fairness metrics
Error rate disparity and prediction gap analysis.
Statistical validation
Bootstrap confidence intervals, permutation testing, and multiple testing corrections (Bonferroni, Benjamini-Hochberg).
Effect size calculations
Cohen's d, risk ratio, and odds ratio.
Bias detection module
Comprehensive data-level bias analysis with historical pattern detection and representation bias.
BiasDetector
Proxy variable identification
Correlation and mutual information-based detection.
MLflow integration
Native fairness logging for experiment tracking.
log_fairness_to_mlflow()
pytest plugin
Fairness assertions in CI/CD pipelines.
Ranking fairness metrics
NDKL, skew, and exposure disparity.
Training callbacks
Framework-specific monitoring hooks for Keras, PyTorch, and scikit-learn.
Basic visualisation
Matplotlib bar charts and heatmaps.
Report export
JSON, dictionary, and structured output formats.
Sample size warnings
Recommendations for adequate statistical power.
Unified API
Consistent interface across classification, regression, and ranking analysers.
Fixed 3
NaN handling
Fixed incorrect handling of NaN values in sensitive attribute columns.
Memory leak fix
Resolved leak in iterative confidence interval computation.
Effect size calculations
Corrected calculations for imbalanced groups.
Initial Development Release

First public development release. Not recommended for production use. Install from source or GitHub.

What's Next

Some of the most interesting parts of vfairness are already in the code and maturing toward a full documentation page and a frozen API; others are planned. Here is where the work is heading.

In progress (already in the code)

  • Explainability & XAI: the SHAP family, the Lundberg fairness decomposition and DiCE counterfactuals (shipping in the code today; full documentation page landing this release)
  • Validity / groundedness axis: the VG-* metric family with a fail-closed LLM judge (interim, live)
  • Vision representation fairness: Skew / NDKL / bias-amplification math shipped, plus an optional FairFace demographic sidecar
  • Causal fairness graphs and fairness-aware model selection

Planned

  • A per-symbol API reference for every top-level export
  • An owned groundedness detector to succeed the interim LLM judge
  • Continuous monitoring dashboard
  • Enterprise compliance reports
Contributing

Found a bug or have a feature request? Please open an issue on GitHub. We welcome contributions from the community.

← Academic References Home →