MAE / MAPE use the median (point) prediction; pinball-d2, coverage and width use all available quantiles ("-" = model has no quantiles). All rows across the 8 CV folds are pooled.
operator_day_ahead is the ENTSO-E day-ahead load forecast for France (issued by RTE), shown for the 24h horizon only: its forecast for target time t is compared with the actual load at t. It has no quantiles or fit time, so it is absent from the pinball, coverage/width and Pareto figures.
| metric | chronos2_ctx4000 | hgb_mean | hgb_mean_fast | hgb_quantile_q7 | hgb_quantile_recal_q7 | kumo_forecast | kumo_tabular | tabicl_forecast_ctx1024_n4 | tabicl_n8_full | operator_day_ahead |
|---|---|---|---|---|---|---|---|---|---|---|
| coverage_90__12h | 0.8404 | - | - | 0.7511 | 0.9046 | - | 0.9744 | 0.8607 | 0.8153 | - |
| coverage_90__1h | 0.9551 | - | - | 0.8151 | 0.9052 | - | 0.9276 | 0.9215 | 0.9168 | - |
| coverage_90__24h | 0.8768 | - | - | 0.7333 | 0.8995 | - | 0.9794 | 0.8676 | 0.887 | - |
| coverage_90__average | 0.8908 | - | - | 0.7665 | 0.9031 | - | 0.9605 | 0.8833 | 0.873 | - |
| mae__12h | 967.3 | 1004 | 1016 | 1001 | 1082 | 1969 | 826.2 | 1327 | 1063 | - |
| mae__1h | 415.2 | 532.8 | 543.6 | 533.8 | 543.6 | 1165 | 276.4 | 382 | 386.3 | - |
| mae__24h | 1100 | 1079 | 1082 | 1042 | 1108 | 2286 | 962.7 | 1470 | 1090 | 1051 |
| mae__average | 827.4 | 872.2 | 880.5 | 859 | 911.1 | 1807 | 688.4 | 1059 | 846.6 | - |
| mape__12h | 0.0199 | 0.02042 | 0.02068 | 0.02033 | 0.02173 | 0.04083 | 0.01688 | 0.02679 | 0.0212 | - |
| mape__1h | 0.008668 | 0.01106 | 0.01128 | 0.01105 | 0.01123 | 0.02514 | 0.005799 | 0.007893 | 0.00793 | - |
| mape__24h | 0.02261 | 0.02203 | 0.02208 | 0.02122 | 0.02236 | 0.04734 | 0.01949 | 0.02953 | 0.022 | 0.02174 |
| mape__average | 0.01706 | 0.01784 | 0.01801 | 0.01753 | 0.01844 | 0.03777 | 0.01405 | 0.02141 | 0.01704 | - |
| width_90__12h | 3408 | - | - | 3416 | 5063 | - | 5055 | 5296 | 3899 | - |
| width_90__1h | 2201 | - | - | 2158 | 2735 | - | 1600 | 1775 | 1896 | - |
| width_90__24h | 4404 | - | - | 3387 | 5067 | - | 6286 | 6136 | 5341 | - |
| width_90__average | 3338 | - | - | 2987 | 4288 | - | 4313 | 4402 | 3712 | - |
| d2_pinball_score__12h__q_0.05 | 0.8226 | - | - | 0.7909 | 0.8016 | - | 0.8309 | 0.7545 | 0.8033 | - |
| d2_pinball_score__12h__q_0.5 | 0.8797 | - | - | 0.8755 | 0.8655 | - | 0.8972 | 0.835 | 0.8677 | - |
| d2_pinball_score__12h__q_0.95 | 0.8792 | - | - | 0.8462 | 0.8606 | - | 0.891 | 0.8421 | 0.8731 | - |
| d2_pinball_score__1h__q_0.05 | 0.9194 | - | - | 0.8898 | 0.8939 | - | 0.9415 | 0.9312 | 0.929 | - |
| d2_pinball_score__1h__q_0.5 | 0.9483 | - | - | 0.9336 | 0.9324 | - | 0.9656 | 0.9525 | 0.9519 | - |
| d2_pinball_score__1h__q_0.95 | 0.9492 | - | - | 0.9169 | 0.9198 | - | 0.9605 | 0.9523 | 0.9517 | - |
| d2_pinball_score__24h__q_0.05 | 0.7905 | - | - | 0.773 | 0.7952 | - | 0.7854 | 0.7174 | 0.7897 | - |
| d2_pinball_score__24h__q_0.5 | 0.8632 | - | - | 0.8703 | 0.8622 | - | 0.8802 | 0.8172 | 0.8644 | - |
| d2_pinball_score__24h__q_0.95 | 0.8704 | - | - | 0.8392 | 0.8578 | - | 0.8709 | 0.8233 | 0.8543 | - |
| d2_pinball_score__average__q_0.05 | 0.8442 | - | - | 0.8179 | 0.8302 | - | 0.8526 | 0.801 | 0.8406 | - |
| d2_pinball_score__average__q_0.5 | 0.8971 | - | - | 0.8931 | 0.8867 | - | 0.9144 | 0.8682 | 0.8947 | - |
| d2_pinball_score__average__q_0.95 | 0.8996 | - | - | 0.8674 | 0.8794 | - | 0.9075 | 0.8726 | 0.893 | - |
True vs predicted load per horizon (separate pages, linked below to keep this page small). For quantile models the plot shows the median as a line and the 5%-95% interval as a shaded band. Each page stacks all models with tied axes: zooming one chart zooms the others, and models can be hidden.
Reliability / calibration diagrams
Pareto plots: average MAPE and pinball loss vs fit + predict time
Each row is a model, with one lane per horizon. MAE/MAPE are relative to hgb_mean; pinball/width are relative to hgb_quantile_q7.
Large markers: pooled mean difference over all test rows, with a 95% moving-block bootstrap interval (14-day blocks over daily differences; conditional on the fitted models, it ignores training randomness). Small dots: mean difference in each of the 8 CV folds (one dot per fold; hover shows the fold). Folds are not independent, so no t-interval across folds is shown. The dashed zero line means no difference; lower is better for
every panel (losses and width). A model whose interval excludes zero and whose
dots are all on one side is consistently different from the baseline; dots on
both sides of zero mean the sign depends on the period.
Each point is one model/horizon, using the mean coverage and mean interval width across the 8 folds. The vertical dashed line is the nominal 90% coverage. Points close to the line are well calibrated; lower is sharper.
Horizontal bars show the mean across the 8 CV folds, grouped by horizon (1h, 12h, 24h), without error bars. The table above pools all rows across folds, so its values can differ slightly, especially for the nonlinear pinball-d2 metrics.