02 · Data
Four panels, one question.
MASTER, CANCER, CFTR and PAH. Each point is a variant; unwinding the helix lays them out by predicted probability, one lane per panel.
02 / 03
Health AI · Genomics · Team project
VARIGEN
Classifying genetic variants as pathogenic or benign across four gene panels, and keeping the decision threshold meaningful when the test set shifts.
Fig. 0 · Variant helix
Live
01
Problem
A variant is a single change in a gene. Most are harmless; a few cause disease. The task: give every variant one probability of being pathogenic, across four gene panels with very different class balances, scored by a jury on a hidden test set.
A probability is not a decision. Somewhere a threshold turns it into one, and that threshold quietly encodes the data it was tuned on.
Fig. 1 · From helix to decision
Counts from the final · dots illustrative
02.1 · Model
One probability per variant.
CatBoost beat XGBoost, ExtraTrees and a blended ensemble under grouped, nested cross-validation, so the same variant never sits in both a training and a validation fold.
02.2 · Threshold
Pick an operating point that holds.
Each panel's threshold maximises (F1 + MCC) / 2 under stressed class ratios. F1 alone can look great while MCC sits near zero, flagging almost everything.
02.3 · Shift
1 positive out of 4,419.
The test set was far more benign than training. The model responded correctly with lower probabilities, and the fixed thresholds became unreachable. Three of four panels flagged nothing.
02.4 · Fix
Transfer the rate, not the threshold.
A threshold's value depends on the distribution; the positive rate travels better. With the model frozen, each panel's optimal rate from out-of-fold predictions is applied as a quantile of the test probabilities. No test labels involved.
03 · Results — national final
4thTEKNOFEST Health AI, national final
4,419Test variants
4Gene panels
04
What carries over
- 01
A fixed threshold is a statement about the training distribution, not about the model.
- 02
Rate transfer assumes test prevalence resembles out-of-fold prevalence: a better guess, not a guarantee.
- 03
Report MCC next to F1. When classes are this imbalanced, F1 alone hides failure.