Key takeaway
A smaller average absolute error can coexist with a larger squared-error loss. Keep the error cases, units and chosen loss beside the ranking.
A winner depends on the loss being compared
A buyer’s evaluation table puts the lowest error in green and calls that model best. Before accepting the result, identify the target, units, evaluated cases and exact loss. Mean absolute error and root mean squared error summarize the same residuals differently. A large individual miss has a stronger influence when errors are squared before averaging, so one winner need not lead both measures.
Scikit-learn documents separate mean_absolute_error and root_mean_squared_error functions, with sample-weight and multioutput options. Their presence does not decide which objective a workflow should use. The evaluator must specify the loss, weighting and aggregation before selecting a favorable result, then retain that definition in the package another reviewer receives.
Calculate the same four invented cases both ways
This hypothetical fixture concerns four equally weighted scalar predictions in invented quantity units. Each residual is prediction minus the reference value. Model A has residuals −4, 4, 4, −4; Model B has −1, 1, 1, 9. Assume the case alignment and reference values are correct only for this teaching calculation. These are not measured model outputs or evidence of a sellable dataset.
For A, the absolute-error total is 16 and squared-error total 64. Dividing each by 4 gives MAE 4 and mean squared error 16; the square root gives RMSE 4. For B, those totals are 12 and 84, giving MAE 3 and mean squared error 21; RMSE is √21, approximately 4.5826. B wins the MAE comparison while A wins RMSE. Neither calculation establishes the right commercial or operating decision.
| Synthetic model | Four residuals | MAE | RMSE | Observed toy ranking |
|---|---|---|---|---|
| A | −4, 4, 4, −4 | 4 units | 4 units | Lower RMSE |
| B | −1, 1, 1, 9 | 3 units | √21 ≈ 4.5826 units | Lower MAE |
Keep the large miss visible
The nine-unit residual in B is the reason its squared-error summary is less favorable even though three cases have small misses. Inspect that case with the domain owner. Is it a valid difficult observation, a reference error, a unit mismatch or an alignment defect? Do not delete it solely because removing it would improve the preferred model’s result. Corrections need an evidence trail and a new calculation.
MAE does not identify a universal typical-case guarantee, and RMSE does not mean every future error will be below that value. Both are summaries of the specified case set. Preserve signed residuals as well: two models with the same loss can overpredict and underpredict in different patterns. A deployment may care differently about those directions, which would need another explicitly justified loss or constraint.
State weights and output aggregation explicitly
The documented functions accept sample weights and define how multiple outputs are aggregated. The example uses neither unequal case weights nor multiple outputs. If an actual archive contains repeated readings from one incident, or several targets with different physical units, disclose the chosen unit of analysis and aggregation. Combining unlike target scales without a justified convention can let the larger numerical scale dominate a summary.
Give the receiving reviewer the exact reference/prediction pairing, eligible cases, exclusions, target units, function and parameters, case weights and output-level results. Do not assume a shared label such as RMSE proves two reports used the same reduction. A root taken after pooling squared errors and an average of separately rooted output errors can differ. The actual implementation needs to match the published method.
Choose a claim that survives a different display order
For the fictional fixture, the completed result is a disagreement between two valid numerical rankings, with all four residuals retained. The team has enough information to ask which error pattern matters and insufficient evidence to call either model operationally best. Freeze the intended objective and evaluation boundary before the next comparison, and use appropriate independent test evidence for a final performance claim.
This is a point-prediction loss question. Probability calibration asks whether predicted chances match observed fractions, and a decision threshold maps scores into actions with their own costs and constraints. A favorable MAE or RMSE does not answer either question. Retain the original error evidence and the predeclared objective so a recipient can assess the same claim, rather than inherit a winner selected by changing the metric.