dataset-shift

Frouros: an open-source Python library for drift detection in machine learning systems.

Code for 'Automatic dataset shift identification to support root cause analysis of AI performance drift', MICCAI 2025

A flexible and powerful Python library for dataset shift analysis and characterization, providing supervised and unsupervised evaluation of temporal and multi-source data shifts, visualization tools, and statistical insights for data integrity and model performance monitoring

Code for "Distance Matters for Improving Performance Estimation Under Covariate Shift", ICCV Workshop on Uncertainty Quantification 2023, Roschewitz & Glocker.

Code for the paper "Where are we with calibration under dataset shift in image classification?"

statistical tests for drift detection and dataset shift

Calibration, seed stability, and external validation for diabetic retinopathy under dataset shift.

Genome-anchored calibration of a continuous phenotype across cohorts, with uncertainty-propagated probabilistic labels

Reliability-first cardiovascular AI with patient-disjoint HPO, selective prediction, calibration stress tests, and cross-dataset transportability auditing.

Reproducible disease-classification study: data audit, nested model comparison, confusion matrices, calibration, thresholds, and hospital transfer.

Analysis code and aggregate results for a three-database study (MIMIC-IV, eICU-CRD, ALOTT) of cross-institution transport loss in a sepsis-associated acute kidney injury prediction model

Code and results for "Stable Discrimination Can Hide Reliability Failures in AI Decision Support Under Distribution Shift and Changing Target Definitions". Computers 15(9), 560 (2026). DOI: 10.3390/computers15090560

WaX: explainable Wasserstein distances. Attributes a Wasserstein distance to instances, features and subspaces (Naumann, Kauffmann and Montavon, TPAMI 2026).

Cross-dataset chest X-ray reliability study of semantic shift, calibration, and high-confidence errors.

Label-shift vs concept-drift experiments and a lightweight logit-offset adapter for prevalence shift in binary classification.

A benchmark postmortem on synthetic shortcuts, near-duplicates, and distribution shift. 2nd place at Purple Hack 2026.

Beyond Random Tile Splits: Source-Held-Out Evaluation and Multi-Stream Fusion for Colorectal Histology. Held-out-source evaluation of color, traditional CV, CNN, and foundation-model (Phikon-v2, ConvNeXt-Tiny) pipelines on Kather-5k colorectal histology. Macro F1 drops from 0.98 to 0.82 when whole source images are held out.

Shift type, not magnitude, determines ML failure modes under deployment shift — a cross-domain audit protocol and benchmark