Learn · Changelog

Development Changelog

Development history for vfairness. Version 0.1.0 is the first public beta, published 2026-10-02 from the v0.1.0 tag. The gate that decided whether it could be pushed passed: scripts/release_gate.py reports BETA READY, with all eight beta conditions met. The API is not frozen until the stable 1.0.0. We follow Keep a Changelog and Semantic Versioning conventions.

0.1.0
Current release, on PyPI since 2 Oct 2026
Beta
Status
~141k
Lines of Code
Beta Software

vfairness 0.1.0 is the first public beta, published 2026-10-02. Install it with pip install vfairness on Python 3.11 or newer; the early 0.0.1 placeholder that held this name before the release has been yanked. The API is not frozen until the stable 1.0.0 release. We welcome feedback and contributions via GitHub.

“Beta” here is the version number, not a verdict that the library is ready to rely on. Run again on 2026-10-01, scripts/release_gate.py reports BETA READY: all eight beta conditions are met and no known defect is open, while the full 1.0 gate still waits for a second examiner on part of the graded code. The earlier measurement is kept below as it was taken. Run on 2026-09-18, scripts/release_gate.py reports NOT BETA READY and NOT READY, four of five criteria failing in each. Two separate measurements sit behind that and are never added together: the 215-row capability census carries 0 open defects, and of the 420 code units graded so far on the wider surface, 95 are DEFECT OPEN. The standing figures and the criteria are on Quality & Hardening, and the release pipeline page records where the gate stands.

Roadmap to 1.0.0

The following features are planned for the stable 1.0.0 release:

Stable API

Finalized, documented API with backward compatibility guarantees.

Complete Test Coverage

Comprehensive unit and integration tests with >90% coverage. Measured at 81% line and branch combined as of 2026-09-11, commit 8559dda; the CI floor is 55% and the per-pull-request diff floor is 80%.

Production Documentation

Full API reference, tutorials, and deployment guides.

PyPI Publication

Done for the beta: 0.1.0 was published to PyPI on 2026-10-02. 1.0.0 will be the first stable release under the backwards-compatibility contract.

Releases

Unreleased

on main
  • 4 Fixed

Changes on main after the v0.1.0 tag. They are not in the 0.1.0 package on PyPI and ship with the next version. This section highlights the Unreleased block of CHANGELOG.md.

Hide

Fixed4

Pulse gap interval holds at zero counts (9 Oct)
The selection-rate gap carried a percentile bootstrap, and a group with nobody selected resamples only zeros: 0 of 22 against 114 of 1,601 read “5.8 to 8.4 points, confirmed”. The gap now carries the Newcombe hybrid score interval (-7.8 to 8.5 points there, not confirmed), and the confirmation uses its dual p-value, so it never contradicts the printed interval. Healthy counts barely move. How a gap is confirmed.
run_pulse ciMethod ciLevel ciNotComputedReason
Every Fail has a finding (9 Oct)
The disparate-impact finding was raised only for a gap of 5 points or more outside US employment, so a confirmed exclusion at a low base rate (0 of 318 aged 18 to 24, four-fifths ratio 0.00, gap 4.6 points) was graded Fail with no finding or recommendation behind it. A confirmed Fail now always raises a critical finding.
build_assurance_verdict
The worst pair is corrected for being chosen (9 Oct)
With an honest interval, the published clean control (nothing planted) read age as confirmed: 9 of 56 against 45 of 118, pair p 0.0034, which the bootstrap had hidden only through its 1/1000 resolution. The worst group is the most extreme of several comparisons with the reference, so its p is now Bonferroni-corrected for that choice before it can confirm. The clean control is quiet again, every planted mechanism previously confirmed still is, and three spurious confirmations on the synthetic pack are gone.
pValueSelection selectionComparisons
An empty grid cell is not measured (9 Oct)
The quarantined legacy pipeline's intersectional grid (unreachable, kept as reference) gave an empty cell a gap of 0.00 and applied no size floor. A cell below the floor now carries no gap, keeps its n and stays out of the worst cell, matching the platform heatmap fix. The live Pulse grid already excluded such cells.
pipeline._intersectional_grid

v0.1.0

2026-10-02
Latest
  • 8 Added
  • 4 Changed
  • 7 Fixed
  • 2 Security

The first public beta: a plain SemVer 0.x release carrying the Development Status :: 4 - Beta classifier, installable with a normal pip install vfairness on Python 3.11 or newer. It was published on 2026-10-02 from the v0.1.0 tag by the release pipeline, which built, verified and smoke-tested the artifact and then published it to PyPI. There is no TestPyPI leg: it was removed on 2026-08-28. The API is not frozen until 1.0.0. This section highlights the [0.1.0] block of CHANGELOG.md.

Hide

The most recent entries under ## [0.1.0] - 2026-10-02 in CHANGELOG.md, summarised (updated 1 Oct 2026; the file has every change in full). Within each group below, the newest entries come first.

Added8

New result fields
Results now say what they left out: the groups a selection-rate headline excluded as too small to read, and the rows removed for a missing prediction.
headline_excluded_groups n_rows_missing_prediction uncovered_groups groups_without_measured_calibration
New helpers
Shared answers to questions several modules asked separately: whether a model can predict right now, whether a value is a real finite number, and the current time in a store's own time zone.
prediction_fn_available finite_or_nan MetricsStore.window_now BoundRole
Output branding switch
New public set_branding() and branding_enabled() remove the validant.ai mark from generated SVGs and reports, honouring an explicit call, then the VFAIRNESS_BRANDING environment variable, then a branded default. A single chart can opt out with render_svg(name, {"branding": False}). An unbranded chart contains no occurrence of the brand string at all, and the switch is unconditional by design: nothing verifies a licence or calls home.
set_branding branding_enabled
SSRF egress guard (vfairness.net)
A new cross-cutting infrastructure sub-package. Wired into the LLM api_proxy first; coverage of the remaining outbound call sites is follow-up work. validate_endpoint resolves and vets the target, guarded_post is a drop-in for requests.post, PinnedIPAdapter pins the vetted IP against DNS rebinding, and a refusal raises SSRFError. Wired into the LLM api_proxy, so assessment traffic cannot be pointed at loopback, private, link-local or cloud-metadata targets. A local model server needs the explicit allow_loopback=True opt-in, which unlocks loopback only.
vfairness.net validate_endpoint guarded_post PinnedIPAdapter SSRFError
Groundedness axis (vfairness.validity)
A public sub-package for the validity / groundedness axis: the VG_* metric identifiers (faithfulness, groundedness, hallucination rate, citation accuracy, context precision and recall, answer correctness), a GroundednessScorer, a fail-closed judge ladder (GroundednessJudge / LlmGroundednessJudge) and aggregate_validity. Deliberately not registered in the capability manifest yet, so nothing can be sealed on it before its gold set lands.
vfairness.validity GroundednessScorer aggregate_validity
Statistical-robustness layer for the sealed spec-v2 gates
A gate now passes on affirmative confidence-interval evidence (one-sided equivalence / TOST), never on a point estimate. New *_with_ci variants cover disparate impact, FPR parity, negative predictive value, conditional demographic disparity, pricing disparity, the integrated calibration index and multicalibration. Custom-shape statistics get an interval from a new bootstrap_over_index helper that resamples row indices stratified by group. The interval width feeds the three-state verdict.
bootstrap_over_index disparate_impact_ratio_with_ci multicalibration_with_ci
Spec-v2 metric panel: independence, sufficiency, calibration, decision utility
New spine primaries and diagnostics: disparate_impact_ratio (the four-fifths selection-rate ratio), conditional_demographic_disparity (the CJEU objective-justification mirror), negative_predictive_value_difference, auroc_parity (a validity gate against levelling-down), grid-free integrated_calibration_index and multicalibration, the TRIPOD recalibration diagnostics calibration_in_the_large and calibration_slope, plus pricing_disparity, net_benefit_parity and conditional_adverse_impact. The error-rate metrics the catalogue referenced but the engine lacked (fpr_parity_difference, fnr_parity_difference, accuracy_parity_difference, worst_group_accuracy) also landed, closing the overclaims found in the knowledge-graph spine audit.
Blocking lint and type gates, mutation testing, governance
ruff check src tests and mypy src are both blocking on every library change, with the toolchain declared in the dev extra so CI, pre-commit and local runs agree. The type burndown reached zero errors and surfaced three real bugs (a dead subgroup-robustness audit in Pulse, a crash on list-valued sample_weight in FairRegressor.fit, and a no-op correlation proxy detector). Also added: mutation testing on the band logic, new property-based / determinism / deserialization-safety / reference-parity suites, an sdist allowlist, beta-programme materials, and the supply-chain governance files.

Changed4

The beta gate is met (1 Oct 2026)
scripts/release_gate.py reports BETA READY: all eight beta conditions met, including two new ones anyone can re-run. Every capability that an established library also computes gives the same answer (34 of 34), and no capability that measures invents a number on broken data (109 of 109). Second-examiner confirmation of every grade is now a condition of 1.0, not of the beta.
release_gate.py B6 B7
Return types and refusals that changed
No public symbol was removed or renamed. Some functions return richer types that still behave like the old ones (FeasibleThresholds is a dict, the proxy results are lists); some can now return None, meaning could not check, so test them with is None; some now raise instead of returning a misleading value. The full list is in CHANGELOG.md.
find_feasible_thresholds identify_proxy_variables find_proxy_chains is_satisfied require_checked to_json
Licence moved from MIT to Apache-2.0
The library ships under Apache-2.0 (copyright Glinz & Company GmbH), with a NOTICE file alongside the LICENSE.
Python floor raised to 3.11
requires-python is now >=3.11, following Scientific Python SPEC 0.

Fixed7

Six defects found by the broken-data check (1 Oct)
Missing predictions read as a “no”; a group that never queried scored the maximum bias; a model with no decision explained as certain; a single row called significant; a genuine null refused; single-attribute comparisons silently dropped. Each now refuses or measures correctly, and each is pinned by a test shown to fail without the fix.
subgroup_robustness_audit RAGBiasAnalyzer.full_analysis AnchorsExplainer EmergentBiasDetector TextFairnessAnalyzer intersectional_disparity_analysis
Every remaining public unit examined (30 Sep)
Each unit nobody had graded was run on input where its quantity cannot exist. Fixes include age banding that rounded 17.6 into the 18-24 band, renderers that drew a full bar for True and a measured-looking bar for NaN, regularisers that accepted a negative strength, and a task envelope that read "false" as success.
prepare_protected_attributes BaseRegularizer TaskResult.from_envelope compute_regression_effect_sizes
Fixes found by the second examination of grades (29 Sep)
A near-constant residual turned Cohen's d into -6.9e15; a NaN group label silently shrank a 0.50 gap to 0.05; a health score read 96.2 GREEN while its breaching records had no readable timestamp. All now refuse or measure correctly.
cohens_d permutation_test_equal_opportunity MetricsStore.compute_health_score MetricsStore.ingest_alert
The second-round audit and the core census (27 to 28 Sep)
193 further findings closed in one sweep, and every core capability examined: unmeasured values no longer appear as clean zeros, ticks or passes in charts, CI rows, dashboards and MCP tool output.
ModelFairnessGate DecodingTrustRunner FairnessDashboard
An undefined group rate is insufficient evidence, not 0.0 parity
predictive_parity_difference, fpr_parity_difference, fnr_parity_difference and equalized_odds_difference collapsed a NaN per-group rate (a group with no positive predictions, no negatives, or no positives) into a 0.0 “perfect parity” result, which read as PASS with a deceptively tight [0.0, 0.0] bootstrap CI. They now return NaN so the verdict routes to insufficient_evidence, mirroring the earlier NPV fix.
False PASS on unmeasurable spec-v2 gates
Four of the new sealed metrics collapsed an unmeasurable result to the numeric “perfectly fair” 0.0 instead of NaN, bypassing both the report's not-assessable guard and the bootstrap NaN filter, so a no-evidence case could seal as fair. multicalibration, negative_predictive_value_difference, auroc_parity and conditional_demographic_disparity now return NaN when there is no evidence, and the last of those renormalises its stratum weights instead of biasing toward “fair” in proportion to the un-assessable population.
Verdicts derive from the interval, not the point estimate
MetricResult gains a three-state verdict (fair / unfair / insufficient_evidence), and is_fair becomes an alias for verdict == 'fair'. A wide small-sample interval is no longer reported as a pass or a fail. predictive_parity_difference_with_ci closes the last sufficiency metric that shipped as a bare point estimate.

Security2

No implicit deserialization of untrusted models
The XAI sidecar refuses to deserialize a model payload unless the caller opts in explicitly (trust_input / VFAIRNESS_TRUST_MODEL_INPUT), closing a remote-code-execution vector. An AST guard test also stops the metric core from swallowing computation errors silently.
Defence in depth on agent- and artifact-supplied paths
The MCP load_dataframe tool rejects remote data_path URLs, so an agent-supplied path cannot become network egress; the Pulse artifact download pins the scheme to https and caps the response size; and the deserialization-safety scan now resolves aliased imports and covers marshal, pandas.read_pickle, jsonpickle and shelve.

v0.0.9

August 2026
Alpha
  • 5 Added
  • 3 Changed
  • 7 Fixed

A deep correctness-and-honesty hardening milestone. Five audit waves swept every subsystem for silently-wrong results, false fairness certificates and band inconsistencies; the rendering layer gained self-explaining, accessible SVGs and a single unified design language (Blanco); and the test/CI infrastructure was strengthened with golden-file value pins, render-smoke coverage across every template, and coverage measurement in CI. This was the last 0.0.x milestone before the 0.1.0 public beta.

Show

Added5

Self-explaining, accessible SVGs
Every rendered SVG now carries its own explanation automatically, wired at the render_svg choke point (so every adapter and .to_svg() inherits it): an auto-generated on-canvas explanation (concept → how to read it → the finding on this chart → recommended action) plus an accessible, machine-readable layer (role="img", <title>, a one-line <desc>, and a <metadata> JSON block). Severity is derived from the same data the badge uses, so the explanation can never contradict the badge. Curated content covers every template.
vfairness.rendering.explain CHART_META
Blanco — the one SVG design language
render_svg applies Blanco (“The Silent Gallery”) by default: a single post-render transform giving sharp corners, editorial type (Plus Jakarta Sans / JetBrains Mono), desaturated graphite chrome, and colour reserved for the four semantic tones. It never touches the templates or adapters themselves.
apply_skin list_skins SKINS
Golden-file regression tests
Exact-value pins for the core classification, regression and ranking metrics and the FairnessAnalyzer report, so a numeric regression fails loudly instead of drifting silently.
tests/test_golden_metrics.py
Render-smoke coverage for every template
Drives render_svg for every registered SVG template and asserts well-formed XML with no unrendered Jinja tokens.
tests/test_rendering.py
Task result envelope & coverage-in-CI
New TaskResult dataclass formalises the task-handler result envelope (schema_version, task_type, success, data/error, optional warnings) with a backward-compatible to_dict(). Coverage is now measured in CI (--cov=vfairness) behind a ratcheting --cov-fail-under gate.
vfairness.result.TaskResult

Changed3

SVG gallery is all-Blanco
The gallery renders every example in Blanco; the Original/Blanco skin switch and the parallel mirror were removed, along with the coloured category-icon boxes and the unused design-proposal SVGs.
CI runs the full test suite
The -m fairness marker matched zero tests (so CI silently ran nothing); the filter was removed from both GitHub and GitLab pipelines, and both now install the extras the suite exercises.
Task handlers use the result envelope
The CI/CD and Pulse handlers build their result via TaskResult.to_dict(). The success/data/error semantics are unchanged; the envelope gains additive schema_version and task_type keys.

Fixed7

Undefined metrics are not failures
A metric that returns NaN (undefined, e.g. an error-rate metric when a group has no positive labels, or R² when a group has constant y_true) is now surfaced in assessment.not_assessable_metrics and excluded from the fairness score, instead of being mis-scored as a FAIL. The classification report gains a not_assessable_metrics field.
Evaluation metrics & statistical honesty
Effect sizes now measure the gap the metric actually reports; equal opportunity returns NaN with a warning (not a silent “fair” 0.0) when a group has no positives; dropped small groups are named; fisher_exact_test uses the real exact test (no silent chi-square degrade); Bayesian credible intervals delegate to scipy (the homegrown Beta PPF had collapsed “95%” intervals to ~55% coverage); Wilson score intervals replace NaN bounds for large groups; multiple-testing correction runs real two-proportion z-tests instead of inverted pseudo p-values.
No more false fairness certificates
Degenerate data (0 samples or a single group) is marked NOT ASSESSABLE instead of certifying fairness_score 1.0; critical findings floor the overall risk band (1+ → MEDIUM, 3+ → HIGH) so three critical issues can no longer average out to a green badge; constant-probability ECE/MCE report the true error instead of a “perfect” 0.0.
Band consistency & explainer coherence
A single canonical vfairness._bands module drives every severity/threshold decision, so the badge, radar chart, detailed report and explanation text can no longer give contradictory verdicts for the same input; the divergent 0.2/0.4/0.6/0.8 risk scale was eliminated everywhere; NaN scores render a neutral N/A badge rather than a green MINIMAL.
In-, post- and pre-processing correctness
Threshold optimization and three of five reweighters crashed on every fit — both fixed; BetaCalibrator is now real Kull et al. 2017 beta calibration; GridSearch predicts with the classifier it reports; FairRegressor raises instead of silently fitting an unconstrained model for unsupported objectives; the adversarial debiasing loss trains its adversary in the correct direction; the outcome column is no longer scanned as its own proxy.
Rendering, XAI, Pulse & ops robustness
Hostile inputs (empty modules, None nests, stringified numbers, Infinity) render or raise a clear error naming the template instead of leaking ZeroDivision/Overflow; falsy-zero bugs (score 0, zero-tolerance thresholds) read at the correct severity; deep-model XAI routing selects Integrated Gradients (Phase-2 DeepExplainer had made every deep job fail); SHAP attributions, base value and prediction now come from one class; causal identification and refuter-crash disclosure fixed; monitoring/drift windows and health-score trends corrected.
Packaging, frozen API surface & test hardening
requirements.txt now includes the hard runtime deps requests and scikit-learn (a minimal install previously could not even import vfairness); the mcp extra is gated to Python ≥ 3.10 so the project locks again; frozen-surface exports (calibration_difference, r2_parity_difference, residual_bias) restored; ~800 new regression tests across tests/test_audit_wave1–5 pin every fix. Documentation examples corrected against the real APIs.

v0.0.8

March 2026
Alpha
  • 22 Added
  • 2 Fixed

Release consolidating all v0.0.x development work into a stable alpha milestone. Includes 43 historical discrimination patterns across US, EU, EU AI Act & Swiss jurisdictions, finalized APIs for the full pipeline, expanded test coverage, and polished documentation across all modules.

Show

Added22

DisparateImpactRemover (pre-processing transformer)
Feldman et al. (2015) geometric repair: aligns each numeric feature's distribution across protected groups toward a shared per-quantile-median target, removing disparate impact while preserving within-group ranking. Tunable repair_level (0 = no change, 1 = full repair) with a tie-robust empirical-CDF mapping. Registered under dispatch key disparate_impact_removal and wired into the platform's pre-processing step (with a configurable repair-strength control).
DisparateImpactRemover disparate_impact_removal
LabelMassager (pre-processing transformer)
Kamiran & Calders (2012) “massaging”: a logistic ranker scores each instance by proximity to the decision boundary, then the minimal set of borderline labels is flipped (promote deprived-group negatives, demote favored-group positives) to equalize group positive rates. Row-preserving; changes only the target. Exposes the additive get_massaged_labels() hook and a max_flip_fraction safety cap. Wired into the platform's pre-processing step.
LabelMassager label_massaging
Resampler (pre-processing transformer)
Kamiran & Calders (2012) sampling: balances protected-group (and group × label) representation by random over-sampling with replacement up to the largest cell, or under-sampling down to the smallest. Exposes the additive get_resampled_data() hook with configurable strategy and balance_by. The platform rebalances the training split only, leaving the held-out test split intact for honest evaluation.
Resampler resampling
Fairness Regularization (in-processing, platform wiring)
The Navigator's “Fairness Regularization” tile now runs a torch training loop that minimizes BCE + λ · GroupFairnessRegularizer. The fairness metric (demographic parity, equalized odds, equal opportunity, or FPR parity) is selected from the chosen fairness definition, and the penalty strength gives continuous control over the accuracy-fairness trade-off. Registered under dispatch key fairness_regularization; falls back to reweighting if torch is unavailable.
GroupFairnessRegularizer fairness_regularization
FairRepresentationTransformer (pre-processing transformer)
Adversarial fair-representation learning (Zemel et al. 2013 LFR; Louizos et al. 2016 VFAE; Madras et al. 2018 LAFTR): an autoencoder encodes the numeric features into a latent code trained with reconstruction loss plus a gradient-reversal adversary that cannot recover the protected group, yielding a group-invariant representation. transform returns the latent columns (rep_0..). Tunable representation_dim and lambda_fairness; multiple attributes handled via their intersection. Registered as fair_representation and wired into the pre-processing step + Pareto sweep. Requires torch.
FairRepresentationTransformer fair_representation
Intervention hardening: intersectional, secondary constraints, guardrails
Across the intervention suite: (1) DisparateImpactRemover, LabelMassager, Resampler, FairRepresentationTransformer and the fairness-regularization training loop now operate on the intersection of all protected attributes rather than only the first (e.g. they repair / rebalance / penalize race x gender jointly); (2) the regularizer accepts secondary fairness constraints (penalized jointly); (3) a degenerate-collapse guard detects and softens penalties that drive the model to a single class; (4) a training-row cap keeps the torch sweep responsive on large datasets; (5) pre-processing techniques (Disparate Impact Removal, Label Massaging, Resampling, Fair Representation) gained Pareto strength sweeps; and (6) results surface proxy-survival notes and small-sample / confidence warnings so a near-unchanged metric is explained rather than mistaken for a no-op.
get_massaged_labels get_resampled_data secondary_constraints
Historical discrimination patterns
43 patterns (up from 12) with multi-jurisdiction coverage across US, EU, and Swiss contexts.
Attribute × domain historical-pattern cross-walk
New attribute_historical_pattern(attribute, domain) helper resolves 24 documented (attribute class × regulated domain) precedents -- race / gender / age / national-origin / religion / disability / geographic proxy across hiring, lending, healthcare, justice, insurance and education. Pulse attaches the citation-backed envelope onto bias findings so the “Historical pattern” channel keys off a structured flag instead of regex-matching evidence text.
attribute_historical_pattern
EU AI Act patterns
11 patterns: 4 prohibited practices (Art. 5) and 7 high-risk systems (Annex III).
European discrimination patterns
12 patterns covering employment, housing, welfare, policing, and education.
Swiss-specific patterns
8 patterns including Ausweis-based discrimination, naturalisation bias, and cantonal disparities.
EU AI Act compliance detection
Automated risk classification with penalty exposure alerts for prohibited and high-risk AI systems.
CRITICAL HIGH MEDIUM
Penalty exposure alerts
Prohibited practices (€35M / 7% turnover) and high-risk systems (€15M / 3% turnover).
Expanded synthetic demo dataset
19 columns including EU, AI Act, and Swiss-specific features.
Stabilised API surface
Consistent interface across all 6 sub-packages (as of this release; the taxonomy is 15 top-level sub-packages as of 0.1.0).
Comprehensive documentation
Curated academic references for each jurisdiction.
Statistical validation module _statistics.py
Comprehensive uncertainty quantification: bootstrap CIs (percentile, BCa, basic), stratified bootstrap for fairness metrics, Bayesian credible intervals (proportion, difference, mean), multiple testing corrections (Bonferroni, Benjamini-Hochberg FDR), effect sizes (Cohen's d, risk ratio, odds ratio), and auto method selection based on sample size. Zero scipy dependency. (Superseded in v0.0.9: the homegrown scipy-free Beta PPF was found to collapse "95%" credible intervals to about 55% coverage, so it now delegates to scipy, which is a hard runtime dependency.)
bootstrap_ci stratified_bootstrap_ci bayesian_proportion_ci bayesian_difference_ci compute_metric_with_ci
Proportion tests and power analysis
Two-proportion z-test, Fisher's exact test (exact for n≤200, chi-squared fallback), Cohen's h effect size for binary outcomes, minimum detectable effect computation, and human-readable power warnings for small subgroups.
proportion_z_test fisher_exact_test cohens_h minimum_detectable_effect power_warning
Intersectional disparity SVG template
New ranked-bar chart SVG template (intersectional_disparity.svg) with ground truth overlay, severity-colored bars, prediction delta indicators, most/least advantaged comparison cards, and insight cards. 44th SVG template in the rendering module.
intersectional_disparity_to_svg()
Dual-lens intersectional analysis
identify_privileged_groups() and intersectional_disparity_analysis() now compute ground truth rates, false positive rates, and prediction deltas per subgroup. New outcome_polarity parameter supports both favorable (loan approval) and unfavorable (recidivism) outcome directions.
ground_truth_rate false_positive_rate prediction_delta outcome_polarity
Confidence intervals in structured findings
Intersectional structured findings now include Bayesian credible intervals for small subgroups, enriching compliance reporting with uncertainty quantification.
Improved SVG chart rendering
Enhanced Pareto frontier visuals, improved disparity heatmap layout, and refined calibration adapters for cleaner publication-ready output.

Fixed2

Validation SVG: raw dict/list strings no longer leak into rendered text
data_validation_to_svg() now safely handles nested dicts and lists, truncates overly long string representations, and only includes scalar values in metric cards. Previously, raw Python repr output (e.g., {'count': 5}) could appear in the SVG text.
Polarity-aware ratio in intersectional insights
Fixed the insight generation to correctly compute advantage ratios when outcome_polarity='positive_unfavorable'. Previously, the ratio calculation was inverted for unfavorable outcomes, making insights misleading.

v0.0.7

February 2026
Alpha
  • 18 Added
  • 7 Changed
  • 3 Fixed

Documentation quality overhaul and major feature release. Complete site redesign with gradient theme system, plus new Reporting & Dashboards, Experimentation & A/B Testing, and Workflow Integration modules. 43 SVG templates across 15 adapter modules, 114 gallery illustrations.

Show

Added18

Reporting module vfairness.operations.reporting
Privacy-preserving queries, Plotly progressive-disclosure views, multi-format NLG output, and interactive Dash/HTML dashboards.
MetricsStore FairnessDashboard ReportGenerator InteractiveDashboard
Experimentation module vfairness.operations.experimentation
A/B testing with intersectional analysis, SPRT early stopping, Pareto frontier and causal decomposition.
FairnessExperiment FairnessPowerAnalyzer ExperimentAnalysis
Monitoring expansion
Wavelet + KS multi-scale drift detection, adaptive thresholds, and alert prioritisation.
FairnessDriftDetector AdaptiveThresholdManager FairnessAlertPrioritizer
Monitoring SVG templates
4 new templates for production monitoring visualisation.
monitoring_dashboard drift_report alert_timeline temporal_analysis
SVG template redesign
All 32 core templates redesigned with consistent card-based layout system and explanation overlays.
SVG gallery page update
Reporting & Experimentation sections, new filter categories, and expanded code examples.
Workflow integration vfairness.operations.cicd
Three-level intersectional gate evaluation, PR-ready markdown reports, and under-powered group detection.
HierarchicalGateConfig FairnessReportCard SmallSampleWarning
W&B integration log_fairness_to_wandb()
Weights & Biases experiment tracking, mirroring the existing MLflow integration.
Auto-logging decorator @auto_log_fairness
Automatic fairness logging during model training with MLflow and W&B backend support.
Pre-commit hooks
Early validation of fairness documentation before commits.
vfairness-check-config vfairness-check-model-card
Formal pytest plugin
Registered via pytest11 entry point with marker, fixture, and terminal summary.
@pytest.mark.fairness fairness_gate
CI/CD configurations
GitHub Actions workflow, GitLab CI config, and PR template with fairness checklist.
fairness-checks.yml
Workflow SVG templates
3 new templates for workflow visualisation.
workflow_overview hierarchical_gate report_card
Getting Started guide expansion
Reporting, Experimentation, and Workflow Integration quick-start sections.
Notebooks
Interactive tutorials for new modules.
reporting_units_3.ipynb experimentation_unit_4.ipynb vfairness_8_workflow_integration.ipynb
8 new SVG templates (35 → 43)
General correlation matrix with mixed methods, causal decomposition, robustness testing, ranking fairness, data validation, auto discovery, regression fairness, and multi-tier reporting dashboard.
correlation_matrix causal_decomposition robustness_testing ranking_fairness data_validation auto_discovery regression_fairness reporting_dashboard
6 new rendering adapter modules
Dedicated adapter files for robustness, ranking, validation, discovery, regression, and reporting SVG templates.
adapters_robustness.py adapters_ranking.py adapters_validation.py adapters_discovery.py adapters_regression.py adapters_reporting.py
Full notebook coverage for all 43 SVG templates
Every SVG adapter function is now exercised in at least one notebook. SVG rendering demo notebook expanded from 35 to 43 templates with new Advanced Modules category.

Changed7

Public API expansion
200+ exports across all pipeline stages with backward-compatible sys.modules aliasing.
Rendering adapters
All adapters updated for consistent explanation parameter and template context.
Project dependencies pyproject.toml
New module dependencies with Plotly and Dash as optional extras.
SVG gallery
Expanded to 43 templates with 114 gallery illustrations including advanced modules for robustness, ranking, validation, discovery, regression, and reporting.
Documentation overhaul
New Workflow Integration nav link, expanded Business Guide with CI/CD gates and MLOps tracking sections, API Reference with 6 new component pages.
Hero stats updated
Getting Started page statistics refreshed: 60K+ lines of code, 1,211 functions, 217 classes, 43 SVG templates, 114 illustrations. (Counts as of v0.0.7, measured as raw definitions. What the Getting Started tiles count has changed since: they now report the whole callable surface, 463 functions, 392 classes and 691 methods across 204 modules, alongside 215 capability-registry rows naming 222 of those same objects, and 44 SVG templates, measured 2026-09-18. Those tiles are filled from data/library-stats.json at page load rather than typed in, so read them there rather than here.)
Business Guide glossary repositioned
Glossary section moved to its correct location before Appendices for better document flow.

Fixed3

Column label readability in correlation SVG templates
Rotated column labels in correlation_heatmap and correlation_matrix templates were painted over by grid cells. Fixed text-anchor from end to start, increased font size to 12px, and added 100px panel headroom.
Cohen's d lollipop chart alignment in regression fairness SVG
Effect size dots were displaced ∼200px left of the center axis due to mismatched coordinate origins. Recentered dot_x computation around the axis position and added smart label placement for negative values.
SVG template count references
Outdated template counts (30+, 35) across Getting Started, API Reference, and main index pages updated to 43.
Feature Release

Major feature and documentation release. New modules require optional dependencies: pip install vfairness[reporting], pip install vfairness[experimentation], or pip install vfairness[mlops].

v0.0.5-dev

February 2026
Alpha
  • 10 Added
  • 4 Changed

Post-processing expansion and SVG rendering engine. Added threshold optimisation, prediction reweighting, operations/monitoring modules for production deployment, and 28 purpose-built SVG templates for server-side report generation.

Show

Added10

Threshold optimisation
Multi-objective threshold tuning for group-aware decisions.
ThresholdOptimizer GroupThresholdOptimizer
Prediction reweighting
Prediction adjustment techniques for fairer outcomes.
PredictionReweighter RejectionOptionClassifier CalibratedEqualizer
Operations module
Production deployment tools for bias validation, gating, and monitoring.
DataBiasValidator ModelFairnessGate BiasMonitor
Drift detection & constraint specification
Alert system for fairness metric drift and declarative constraint types.
DriftAlert FairnessConstraintType
Testing utilities
pytest integration for fairness assertions.
FairnessTestSuite @fairness_test
Rendering sub-package
Core SVG engine, adapters, and 28 purpose-built templates for server-side report generation.
vfairness.rendering
Fairness report SVG adapters
Adapters for classification, regression, and ranking results.
Calibration SVG adapters
Reliability diagrams and disparity plots.
Feature engineering SVG adapters
Correlation matrices and proxy chain visualisation.
Training analysis SVG adapters
Loss curves and constraint satisfaction tracking.

Changed4

Post-processing restructuring
Reorganised into calibration, threshold_optimization, and reweighting sub-modules.
CI/CD module relocation
Moved from evaluation.integrations to dedicated operations.cicd module.
CalibrationAnalyzer enhancement
Pareto frontier and impossibility theorem diagnostics.
Rendering / visualisation separation
Plotly/Matplotlib for interactive use, SVG engine for static reports.

v0.0.3-dev

January – February 2026
Alpha
  • 14 Added
  • 5 Changed
  • 3 Performance
  • 3 Fixed

Full pipeline expansion. Post-processing calibration, feature engineering, in-processing training interventions, and enhanced visualisation. Expanded vfairness from detection-only to a full-pipeline fairness toolkit.

Show

Added14

FairExplAIner module
Human-readable explanations for all fairness metrics.
FairExplAIner
Calibration module
Group-aware probability calibration with 5 methods, intersectional calibration, and Pareto trade-off analysis.
GroupCalibrator IntersectionalCalibrator CalibrationAnalyzer
Feature engineering sub-package
Fairness-aware data transformation, 5 transformers, and proxy chain detection via mutual information.
vfairness.preprocessing.feature_engineering FeatureEngineeringAnalyzer
In-processing module
Complete training-time intervention toolkit with 12 loss functions, 5 constraint types, and 5 regularizers.
vfairness.in_processing ExponentiatedGradient FairClassifier
Trainable group calibrators
5 calibration methods for per-group adjustment during training.
TemperatureScaling PlattScaling TrainableGroupCalibrator
Enhanced visualisation suite
Interactive Plotly charts, publication-ready matplotlib outputs, reliability diagrams, and disparity dashboards.
Bayesian confidence intervals
Conjugate priors for small sample sizes (n < 30).
Auto-discovery of protected attributes
Automatic detection from dataset column names and metadata.
Geographic bias detection
Location-based disparity analysis.
Quick analysis function
Convenience function for rapid fairness analysis.
classification_fairness_report()
Multi-class classification support
Fairness metrics extended to multi-class settings.
Training analyser
Monitoring fairness metrics during model training.
FairnessTrainingAnalyzer
Documentation site
Comprehensive API reference, tutorials, and 6 Jupyter demo notebooks.
scikit-learn wrappers
Drop-in compatible classifiers and regressors.
FairClassifier FairRegressor

Changed5

Package reorganisation
6 top-level sub-packages: preprocessing, in_processing, post_processing, evaluation, operations, rendering.
Library scope expansion
From detection/measurement to full-pipeline fairness: detect, mitigate, calibrate, monitor.
Bootstrap confidence intervals
Bias-corrected and accelerated (BCa) method for improved accuracy.
MLflow integration
Automatic artifact logging and metric tracking.
FairnessAnalyzer refactoring
Better memory efficiency with large datasets.

Performance3

Bootstrap computation
40% faster through vectorized operations.
Intersectional analysis
60% reduced memory footprint.
Permutation tests
Parallel processing support.

Fixed3

Demographic parity edge case
Fixed calculation when one group has zero positive predictions.
Calibration numerical stability
Resolved issues with very small probabilities.
pytest assertion messages
Clearer failure diagnostics.
Scope Change

vfairness is no longer detection-only. With in-processing and post-processing modules, it now covers the full ML fairness pipeline.

v0.0.1-alpha

December 2025
Alpha
  • 16 Added
  • 3 Fixed

Initial development release. Core fairness metrics, statistical validation, bias detection module, and MLOps integration foundations.

Show

Added16

Core fairness analyser
Classification fairness analysis with demographic parity, equal opportunity, and equalized odds.
FairnessAnalyzer
Calibration & disparate impact metrics
Calibration difference, predictive parity, and 80% rule calculation.
Intersectional analysis
Multi-attribute fairness evaluation with automatic subgroup discovery.
Regression fairness metrics
Error rate disparity and prediction gap analysis.
Statistical validation
Bootstrap confidence intervals, permutation testing, and multiple testing corrections (Bonferroni, Benjamini-Hochberg).
Effect size calculations
Cohen's d, risk ratio, and odds ratio.
Bias detection module
Comprehensive data-level bias analysis with historical pattern detection and representation bias.
BiasDetector
Proxy variable identification
Correlation and mutual information-based detection.
MLflow integration
Native fairness logging for experiment tracking.
log_fairness_to_mlflow()
pytest plugin
Fairness assertions in CI/CD pipelines.
Ranking fairness metrics
NDKL, skew, and exposure disparity.
Training callbacks
Framework-specific monitoring hooks for Keras, PyTorch, and scikit-learn.
Basic visualisation
Matplotlib bar charts and heatmaps.
Report export
JSON, dictionary, and structured output formats.
Sample size warnings
Recommendations for adequate statistical power.
Unified API
Consistent interface across classification, regression, and ranking analysers.

Fixed3

NaN handling
Fixed incorrect handling of NaN values in sensitive attribute columns.
Memory leak fix
Resolved leak in iterative confidence interval computation.
Effect size calculations
Corrected calculations for imbalanced groups.
Initial Development Release

First public development release. Not recommended for production use. Install from source or GitHub.

What's Next

Some of the most interesting parts of vfairness are already in the code and maturing toward a full documentation page and a frozen API; others are planned. Here is where the work is heading.

In progress (already in the code)

  • Explainability & XAI: the SHAP family, the Lundberg fairness decomposition and DiCE counterfactuals (shipping in the code today; full documentation page landing this release)
  • Validity / groundedness axis: the VG-* metric family with a fail-closed LLM judge (interim, live)
  • Vision representation fairness: Skew / NDKL / bias-amplification math shipped, plus an optional FairFace demographic sidecar
  • Causal fairness graphs and fairness-aware model selection

Planned

  • A per-symbol API reference for every top-level export
  • An owned groundedness detector to succeed the interim LLM judge
  • Continuous monitoring dashboard
  • Enterprise compliance reports
Contributing

Found a bug or have a feature request? Please open an issue on GitHub. We welcome contributions from the community.