MAE / MAPE use the median (point) prediction; pinball-d2, coverage and width use all available quantiles ("-" = model has no quantiles). All rows across the 8 CV folds are pooled.
operator_day_ahead is the ENTSO-E day-ahead load forecast for France (issued by RTE), shown for the 24h horizon only: its forecast for target time t is compared with the actual load at t. It has no quantiles or fit time, so it is absent from the pinball, coverage/width and Pareto figures.
| metric | chronos2_ctx4000 | hgb_mean | hgb_quantile_q7 | hgb_quantile_recal_q7 | kumo_forecast | kumo_tabular | tabicl_n8_full | operator_day_ahead |
|---|---|---|---|---|---|---|---|---|
| coverage_90__12h | 0.8519 | - | 0.7631 | 0.9102 | - | 0.9746 | 0.8456 | - |
| coverage_90__1h | 0.9549 | - | 0.8146 | 0.9047 | - | 0.9193 | 0.9283 | - |
| coverage_90__24h | 0.8804 | - | 0.7599 | 0.9048 | - | 0.9878 | 0.904 | - |
| coverage_90__average | 0.8957 | - | 0.7792 | 0.9066 | - | 0.9606 | 0.8927 | - |
| mae__12h | 911.9 | 938 | 952.7 | 1011 | 1973 | 754.3 | 1029 | - |
| mae__1h | 412.7 | 531.4 | 534 | 540.9 | 1161 | 276.9 | 384 | - |
| mae__24h | 1035 | 971.4 | 960.4 | 1011 | 2280 | 826.4 | 1036 | 1051 |
| mae__average | 786.6 | 813.6 | 815.7 | 854.2 | 1805 | 619.2 | 816.2 | - |
| mape__12h | 0.01892 | 0.01917 | 0.01948 | 0.02046 | 0.04098 | 0.0156 | 0.02057 | - |
| mape__1h | 0.008655 | 0.01107 | 0.01108 | 0.01119 | 0.02506 | 0.00582 | 0.007879 | - |
| mape__24h | 0.02148 | 0.01993 | 0.01971 | 0.0206 | 0.04723 | 0.01702 | 0.02117 | 0.02174 |
| mape__average | 0.01635 | 0.01672 | 0.01676 | 0.01742 | 0.03776 | 0.01281 | 0.01654 | - |
| width_90__12h | 3297 | - | 3325 | 4780 | - | 4647 | 4059 | - |
| width_90__1h | 2194 | - | 2167 | 2732 | - | 1571 | 1944 | - |
| width_90__24h | 4204 | - | 3288 | 4789 | - | 5964 | 5519 | - |
| width_90__average | 3232 | - | 2926 | 4100 | - | 4061 | 3841 | - |
| d2_pinball_score__12h__q_0.05 | 0.8324 | - | 0.8015 | 0.8165 | - | 0.8451 | 0.8061 | - |
| d2_pinball_score__12h__q_0.5 | 0.8866 | - | 0.8815 | 0.8743 | - | 0.9062 | 0.8721 | - |
| d2_pinball_score__12h__q_0.95 | 0.8865 | - | 0.8551 | 0.8666 | - | 0.8992 | 0.885 | - |
| d2_pinball_score__1h__q_0.05 | 0.9206 | - | 0.8913 | 0.8943 | - | 0.9427 | 0.9285 | - |
| d2_pinball_score__1h__q_0.5 | 0.9487 | - | 0.9336 | 0.9327 | - | 0.9655 | 0.9522 | - |
| d2_pinball_score__1h__q_0.95 | 0.9488 | - | 0.9173 | 0.9209 | - | 0.9596 | 0.9522 | - |
| d2_pinball_score__24h__q_0.05 | 0.8039 | - | 0.7956 | 0.8107 | - | 0.8036 | 0.7902 | - |
| d2_pinball_score__24h__q_0.5 | 0.8712 | - | 0.8805 | 0.8742 | - | 0.8972 | 0.8711 | - |
| d2_pinball_score__24h__q_0.95 | 0.8738 | - | 0.8553 | 0.8681 | - | 0.8756 | 0.8645 | - |
| d2_pinball_score__average__q_0.05 | 0.8523 | - | 0.8295 | 0.8405 | - | 0.8638 | 0.8416 | - |
| d2_pinball_score__average__q_0.5 | 0.9021 | - | 0.8985 | 0.8937 | - | 0.923 | 0.8985 | - |
| d2_pinball_score__average__q_0.95 | 0.903 | - | 0.8759 | 0.8852 | - | 0.9115 | 0.9006 | - |
True vs predicted load per horizon (separate pages, linked below to keep this page small). For quantile models the plot shows the median as a line and the 5%-95% interval as a shaded band. Each page stacks all models with tied axes: zooming one chart zooms the others, and models can be hidden.
Reliability / calibration diagrams
Pareto plots: average MAPE and pinball loss vs fit + predict time
Each row is a model, with one lane per horizon. MAE/MAPE are relative to hgb_mean; pinball/width are relative to hgb_quantile_q7.
Large markers: pooled mean difference over all test rows, with a 95% moving-block bootstrap interval (14-day blocks over daily differences; conditional on the fitted models, it ignores training randomness). Small dots: mean difference in each of the 8 CV folds (one dot per fold; hover shows the fold). Folds are not independent, so no t-interval across folds is shown. The dashed zero line means no difference; lower is better for
every panel (losses and width). A model whose interval excludes zero and whose
dots are all on one side is consistently different from the baseline; dots on
both sides of zero mean the sign depends on the period.
Each point is one model/horizon, using the mean coverage and mean interval width across the 8 folds. The vertical dashed line is the nominal 90% coverage. Points close to the line are well calibrated; lower is sharper.
Horizontal bars show the mean across the 8 CV folds, grouped by horizon (1h, 12h, 24h), without error bars. The table above pools all rows across folds, so its values can differ slightly, especially for the nonlinear pinball-d2 metrics.