Key takeaway
Choose the class set and F1 reduction before comparing models. Micro pools decisions; macro gives each declared class equal weight after calculating class scores.
Read the averaging argument beside the metric name
Two receiving reports both say F1, yet choose different winning models. Before investigating the training data, inspect the class set and averaging rule. Scikit-learn defines micro F1 by pooling true positives, false positives and false negatives. Macro F1 first calculates each label’s F1 and then takes their unweighted mean. Those operations can rank the same predictions differently.
F1 for one class is 2TP/(2TP+FP+FN). Pooling those counts and averaging separate ratios are different calculations. The report should identify whether the task is single-label or multilabel, which classes are included, how predictions were produced and what happens when a class is absent. A metric name without those choices leaves the acceptance rule incomplete.
A complete hypothetical reversal on the same cases
Consider 100 invented, equally weighted single-label cases: 90 common cases and 10 rare cases. Model A predicts common for all 100. Model B correctly predicts 75 common and eight rare cases; it misclassifies 15 common cases as rare and two rare cases as common. Both models are evaluated against exactly the same supplied labels. These are teaching counts, not observed product performance.
For A, common-class F1 is 180/(180+10+0)=0.947368; rare-class F1 is zero because it misses all ten rare cases. For B, common-class F1 is 150/(150+2+15)=0.898204, and rare-class F1 is 16/(16+15+2)=0.484848. Macro averages each pair. Micro pools both classes: A has 90 correct decisions and B has 83, giving 0.90 and 0.83 in this all-classes single-label fixture.
| Synthetic model | Common F1 | Rare F1 | Macro F1 | Micro F1 |
|---|---|---|---|---|
| A: always common | 0.947368 | 0 | 0.473684 | 0.90 |
| B: 75 common and 8 rare correct | 0.898204 | 0.484848 | 0.691526 | 0.83 |
The winning column expresses a decision about classes
Micro favors A; macro favors B. The arithmetic is not a contradiction. Micro reflects pooled label decisions in the stipulated case mix, while macro gives the rare class the same weight as the common class after their scores are calculated. The author recomputed all four class scores and both reductions from the confusion counts locally. No threshold search or model fitting produced this table.
Neither reduction automatically captures the operating consequences of these errors. A rare class may be consequential, but its false alarms may also consume scarce review capacity. Keep the confusion matrix and per-class support beside the headline. If the decision needs explicit error costs or capacity limits, define those separately instead of treating macro F1 as an economic objective.
This is also different from applying survey weights to estimate a population quantity. No selection probabilities or population expansion weights are present here. The choice concerns how label-specific performance summaries are reduced over a fixed evaluation. Changing the macro/micro argument cannot repair an unrepresentative case selection or establish uncertainty.
Inspect defaults, absent classes and the prediction rule
Scikit-learn’s F1 interface defaults to binary averaging, whereas the cited TensorFlow F1Score interface defaults to average=None, returning class-level results. TensorFlow also documents that threshold=None converts the largest predicted value to the selected class; a specified threshold applies a different conversion. Do not assume two libraries computed the same decisions or summary from similarly named inputs.
The declared label set matters even when a class has no examples. Scikit-learn exposes a zero_division policy and notes that undefined class scores can be assigned values or excluded when the chosen setting is NaN. Record the choice and the absent-class count. Excluding an unobserved class may be appropriate for a defined report, but it narrows the claim and must not disappear from the evaluation coverage.
Freeze the report definition before the next comparison
For this fictional comparison, the completed result is a ranking reversal under two declared reductions. The next acceptance decision is to choose which class treatment serves the intended use, preserve both columns for inspection and retain the full confusion counts. A team should not switch reductions after seeing which one makes its preferred model win.
Save the label vocabulary and order, actual and predicted class for each eligible case, support per class, averaging argument, undefined-score policy, prediction conversion and software version. That compact record lets a receiving reviewer reproduce the metric without requesting unrelated source narratives. Keep any later action threshold, deployment test or data-use permission as its own decision.