PAIRWISE TIME SERIES BENCHMARKfev-bench • 100 TasksScaled Quantile Loss (SQL)
TimesFM-3 vs t0-gamma Benchmark
Empirical head-to-head evaluation of TimesFM-3 (Google) versus t0-gamma (The Forecasting Co.) across 100 standardized time series forecasting tasks from AutoGluon fev-bench.
HEAD-TO-HEAD WINS?Direct pairwise task matchup wins where the model achieved lower Scaled Quantile Loss (SQL) across 9 probability levels (p10–p90).
TimesFM-346
:54t0-gamma
t0-gamma leads 54.0% of tasks
GLOBAL SQL WIN RATE?Average win rate (%) across all 41 competing models in the fev-bench suite.
TimesFM-378.5%
t0-gamma84.4%
Global benchmark win rate
SQL SKILL SCORE?Normalized percentage improvement in SQL relative to the Seasonal Naive reference baseline (0% = baseline, higher is better).
TimesFM-3+49.0%
t0-gamma+49.4%
Improvement vs Seasonal Naive
MEDIAN LATENCY?Median inference time per 100 series in seconds. Lower is faster.
TimesFM-33.41s
t0-gamma4.38s
Per 100 time series
ARCHITECTURAL SPECIFICATIONS
Model Architecture & Execution Profiles
Comparing underlying architectures, tokenization mechanics, and in-browser deployment feasibility.
Foundation Model
TimesFM-3
GoogleGoogle Research's flagship time-series foundation model pre-trained on over 100 billion real-world time points for zero-shot forecasting.
ArchitecturePatch-based Transformer Decoder
TokenizationContinuous Patches (32-step temporal patches)
Context Window512 to 2,048 steps standard context
In-Browser RuntimeServer / Cloud Only (PyTorch / JAX)
LicenseApache 2.0
Foundation Model
t0-gamma
The Forecasting Co.High-capacity closed-weights foundation model from The Forecasting Company achieving high skill scores on fev-bench.
ArchitecturePre-trained Temporal Transformer
TokenizationProprietary Continuous Embedding
Context WindowMulti-Horizon Dynamic Context
In-Browser RuntimeServer / Cloud API Only
LicenseProprietary / Closed-Weights
EMPIRICAL BENCHMARK BREAKDOWN
Task-by-Task Comparison Across 100 fev Tasks
Detailed breakdown of Scaled Quantile Loss (SQL) for each task. Lower loss indicates superior probabilistic calibration. Tasks featured in Carla's interactive Google Sheets catalog are highlighted with direct sheet links.
| Benchmark Task | Dataset Category | TimesFM-3 (SQL) | t0-gamma (SQL) | Advantage | Winner | Sheet |
|---|---|---|---|---|---|---|
| BizITobs Cloud Infrastructure Telemetrybizitobs_l2c_5T | Carla Demo | 0.3010 | 0.3993 | +24.6% | TimesFM-3 | |
| Boomlet Smart Meter Building Energyboomlet_1282 | Carla Demo | 0.3961 | 0.4013 | +1.3% | TimesFM-3 | |
| Boomlet Smart Meter Building Energyboomlet_1676 | Carla Demo | 0.5525 | 0.5529 | +0.1% | TimesFM-3 | |
| Boomlet Smart Meter Building Energyboomlet_619 | Carla Demo | 0.3105 | 0.3025 | +2.6% | t0-gamma | |
| Electricity Transformer TemperatureETT_15T | Carla Demo | 0.5433 | 0.5430 | +0.1% | t0-gamma | |
| Electricity Transformer TemperatureETT_1H | Carla Demo | 0.8302 | 0.8418 | +1.4% | TimesFM-3 | |
| Global Energy Forecasting Competition (GEFCom 2014)proenfo_gfc14 | Carla Demo | 0.2688 | 0.3068 | +12.4% | TimesFM-3 | |
| Global Sovereign Carbon Dioxide (CO2) Emissionsworld_co2_emissions | Carla Demo | 2.7799 | 2.5551 | +8.1% | t0-gamma | |
| Hospital Emergency Admissionshospital_admissions_1D | Carla Demo | 0.5552 | 0.5537 | +0.3% | t0-gamma | |
| Hospital Emergency Admissionshospital_admissions_1W | Carla Demo | 0.5814 | 0.5784 | +0.5% | t0-gamma | |
| Jena Climate & Weather Telemetryjena_weather_1H | Carla Demo | 0.3454 | 0.3568 | +3.2% | TimesFM-3 | |
| Monash High-Density Time Series BenchmarkM_DENSE_1D | Carla Demo | 0.6523 | 0.6604 | +1.2% | TimesFM-3 | |
| Nord Pool Day-Ahead Electricity Pricesepf_np | Carla Demo | 0.6986 | 0.6146 | +12.0% | t0-gamma | |
| Rohlik E-Grocery Daily Fulfillment Ordersrohlik_orders_1D | Carla Demo | 0.9120 | 0.9046 | +0.8% | t0-gamma | |
| Rossmann Store Daily & Weekly Salesrossmann_1D | Carla Demo | 0.2487 | 0.2601 | +4.4% | TimesFM-3 | |
| Rossmann Store Daily & Weekly Salesrossmann_1W | Carla Demo | 0.2653 | 0.2655 | +0.1% | TimesFM-3 | |
| Solar Photovoltaic Power Output with Weathersolar_with_weather_1H | Carla Demo | 0.6524 | 0.6616 | +1.4% | TimesFM-3 | |
| UCI Air Quality Chemical Gas Sensor Telemetryuci_air_quality_1H | Carla Demo | 0.7532 | 0.8228 | +8.5% | TimesFM-3 | |
| US Macroeconomic Consumption & Disposable Incomeus_consumption_1Y | Carla Demo | 3.6859 | 3.1735 | +13.9% | t0-gamma | |
| australian tourismaustralian_tourism | fev-bench | 0.7906 | 0.7145 | +9.6% | t0-gamma | — |
| bizitobs l2c 1Hbizitobs_l2c_1H | fev-bench | 0.2820 | 0.2963 | +4.8% | TimesFM-3 | — |
| boomlet 1062boomlet_1062 | fev-bench | 0.5453 | 0.5689 | +4.1% | TimesFM-3 | — |
| boomlet 1209boomlet_1209 | fev-bench | 0.6454 | 0.6853 | +5.8% | TimesFM-3 | — |
| boomlet 1225boomlet_1225 | fev-bench | 0.1838 | 0.1899 | +3.2% | TimesFM-3 | — |
| boomlet 1230boomlet_1230 | fev-bench | 1.1599 | 1.1845 | +2.1% | TimesFM-3 | — |
| boomlet 1487boomlet_1487 | fev-bench | 0.4117 | 0.4144 | +0.7% | TimesFM-3 | — |
| boomlet 1631boomlet_1631 | fev-bench | 0.5741 | 0.5754 | +0.2% | TimesFM-3 | — |
| boomlet 1855boomlet_1855 | fev-bench | 0.4506 | 0.4490 | +0.4% | t0-gamma | — |
| boomlet 1975boomlet_1975 | fev-bench | 0.1255 | 0.1124 | +10.4% | t0-gamma | — |
| boomlet 2187boomlet_2187 | fev-bench | 0.7139 | 0.7142 | +0.0% | TimesFM-3 | — |
| boomlet 285boomlet_285 | fev-bench | 0.2973 | 0.2678 | +9.9% | t0-gamma | — |
| boomlet 772boomlet_772 | fev-bench | 0.2766 | 0.2882 | +4.0% | TimesFM-3 | — |
| boomlet 963boomlet_963 | fev-bench | 0.6899 | 0.6959 | +0.9% | TimesFM-3 | — |
| ecdc iliecdc_ili | fev-bench | 2.1446 | 1.9604 | +8.6% | t0-gamma | — |
| entsoe 15Tentsoe_15T | fev-bench | 0.4910 | 0.4648 | +5.3% | t0-gamma | — |
| entsoe 1Hentsoe_1H | fev-bench | 0.3154 | 0.3956 | +20.3% | TimesFM-3 | — |
| entsoe 30Tentsoe_30T | fev-bench | 0.5312 | 0.4510 | +15.1% | t0-gamma | — |
| epf beepf_be | fev-bench | 0.4822 | 0.5123 | +5.9% | TimesFM-3 | — |
| epf deepf_de | fev-bench | 0.4886 | 0.4571 | +6.4% | t0-gamma | — |
| epf frepf_fr | fev-bench | 0.3297 | 0.3563 | +7.5% | TimesFM-3 | — |
| epf pjmepf_pjm | fev-bench | 0.4110 | 0.3946 | +4.0% | t0-gamma | — |
| ercot 1Dercot_1D | fev-bench | 0.7795 | 0.8552 | +8.9% | TimesFM-3 | — |
| ercot 1Hercot_1H | fev-bench | 1.0002 | 0.9708 | +2.9% | t0-gamma | — |
| ercot 1Mercot_1M | fev-bench | 0.7642 | 0.7537 | +1.4% | t0-gamma | — |
| ercot 1Wercot_1W | fev-bench | 0.9106 | 0.9010 | +1.1% | t0-gamma | — |
| ETT 1DETT_1D | fev-bench | 1.1268 | 1.1067 | +1.8% | t0-gamma | — |
| ETT 1WETT_1W | fev-bench | 2.4342 | 2.2598 | +7.2% | t0-gamma | — |
| favorita stores 1Dfavorita_stores_1D | fev-bench | 0.9071 | 0.9212 | +1.5% | TimesFM-3 | — |
| favorita stores 1Mfavorita_stores_1M | fev-bench | 1.9233 | 1.8086 | +6.0% | t0-gamma | — |
| favorita stores 1Wfavorita_stores_1W | fev-bench | 2.2531 | 1.9834 | +12.0% | t0-gamma | — |
| favorita transactions 1Dfavorita_transactions_1D | fev-bench | 0.8021 | 0.8745 | +8.3% | TimesFM-3 | — |
| favorita transactions 1Mfavorita_transactions_1M | fev-bench | 1.0161 | 1.0316 | +1.5% | TimesFM-3 | — |
| favorita transactions 1Wfavorita_transactions_1W | fev-bench | 1.2066 | 1.3602 | +11.3% | TimesFM-3 | — |
| fred md 2025/ceefred_md_2025/cee | fev-bench | 3.3309 | 3.3866 | +1.6% | TimesFM-3 | — |
| fred md 2025/macrofred_md_2025/macro | fev-bench | 5.1089 | 5.1259 | +0.3% | TimesFM-3 | — |
| fred qd 2025/ceefred_qd_2025/cee | fev-bench | 1.6971 | 1.7060 | +0.5% | TimesFM-3 | — |
| fred qd 2025/macrofred_qd_2025/macro | fev-bench | 3.3802 | 3.2557 | +3.7% | t0-gamma | — |
| gvargvar | fev-bench | 0.5684 | 0.5604 | +1.4% | t0-gamma | — |
| hermeshermes | fev-bench | 0.5987 | 0.6209 | +3.6% | TimesFM-3 | — |
| hierarchical sales 1Dhierarchical_sales_1D | fev-bench | 0.5487 | 0.5488 | +0.0% | TimesFM-3 | — |
| hierarchical sales 1Whierarchical_sales_1W | fev-bench | 0.6151 | 0.6135 | +0.3% | t0-gamma | — |
| hospitalhospital | fev-bench | 0.6825 | 0.6686 | +2.0% | t0-gamma | — |
| jena weather 10Tjena_weather_10T | fev-bench | 0.3381 | 0.3521 | +4.0% | TimesFM-3 | — |
| jena weather 1Djena_weather_1D | fev-bench | 1.1433 | 1.0758 | +5.9% | t0-gamma | — |
| kdd cup 2022 10Tkdd_cup_2022_10T | fev-bench | 0.4251 | 0.4169 | +1.9% | t0-gamma | — |
| kdd cup 2022 1Dkdd_cup_2022_1D | fev-bench | 0.6994 | 0.6983 | +0.2% | t0-gamma | — |
| kdd cup 2022 30Tkdd_cup_2022_30T | fev-bench | 0.4261 | 0.4236 | +0.6% | t0-gamma | — |
| LOOP SEATTLE 1DLOOP_SEATTLE_1D | fev-bench | 0.7587 | 0.7490 | +1.3% | t0-gamma | — |
| LOOP SEATTLE 1HLOOP_SEATTLE_1H | fev-bench | 0.5619 | 0.5996 | +6.3% | TimesFM-3 | — |
| LOOP SEATTLE 5TLOOP_SEATTLE_5T | fev-bench | 0.5515 | 0.5284 | +4.2% | t0-gamma | — |
| M DENSE 1HM_DENSE_1H | fev-bench | 0.5079 | 0.5582 | +9.0% | TimesFM-3 | — |
| m5 1Dm5_1D | fev-bench | 0.7080 | 0.7024 | +0.8% | t0-gamma | — |
| m5 1Mm5_1M | fev-bench | 0.9553 | 0.9364 | +2.0% | t0-gamma | — |
| m5 1Wm5_1W | fev-bench | 0.8779 | 0.8797 | +0.2% | TimesFM-3 | — |
| proenfo gfc12proenfo_gfc12 | fev-bench | 0.5502 | 0.5971 | +7.9% | TimesFM-3 | — |
| proenfo gfc17proenfo_gfc17 | fev-bench | 0.4021 | 0.3900 | +3.0% | t0-gamma | — |
| redset 15Tredset_15T | fev-bench | 0.7035 | 0.6298 | +10.5% | t0-gamma | — |
| redset 1Hredset_1H | fev-bench | 1.3330 | 1.2525 | +6.0% | t0-gamma | — |
| redset 5Tredset_5T | fev-bench | 0.6469 | 0.6442 | +0.4% | t0-gamma | — |
| restaurantrestaurant | fev-bench | 0.7001 | 0.6712 | +4.1% | t0-gamma | — |
| rohlik orders 1Wrohlik_orders_1W | fev-bench | 1.3486 | 1.2614 | +6.5% | t0-gamma | — |
| rohlik sales 1Drohlik_sales_1D | fev-bench | 0.8257 | 0.8336 | +0.9% | TimesFM-3 | — |
| rohlik sales 1Wrohlik_sales_1W | fev-bench | 1.1134 | 1.0795 | +3.0% | t0-gamma | — |
| solar 1Dsolar_1D | fev-bench | 0.6037 | 0.5982 | +0.9% | t0-gamma | — |
| solar 1Wsolar_1W | fev-bench | 0.9428 | 0.9144 | +3.0% | t0-gamma | — |
| solar with weather 15Tsolar_with_weather_15T | fev-bench | 0.6889 | 0.7156 | +3.7% | TimesFM-3 | — |
| SZ TAXI 15TSZ_TAXI_15T | fev-bench | 0.3924 | 0.3928 | +0.1% | TimesFM-3 | — |
| SZ TAXI 1HSZ_TAXI_1H | fev-bench | 0.4009 | 0.3864 | +3.6% | t0-gamma | — |
| uci air quality 1Duci_air_quality_1D | fev-bench | 1.0301 | 1.0810 | +4.7% | TimesFM-3 | — |
| uk covid nation 1D/cumulativeuk_covid_nation_1D/cumulative | fev-bench | 6.4477 | 5.9877 | +7.1% | t0-gamma | — |
| uk covid nation 1D/newuk_covid_nation_1D/new | fev-bench | 1.9857 | 1.9766 | +0.5% | t0-gamma | — |
| uk covid nation 1W/cumulativeuk_covid_nation_1W/cumulative | fev-bench | 3.9370 | 2.5286 | +35.8% | t0-gamma | — |
| uk covid nation 1W/newuk_covid_nation_1W/new | fev-bench | 4.2343 | 4.4654 | +5.2% | TimesFM-3 | — |
| uk covid utla 1D/newuk_covid_utla_1D/new | fev-bench | 3.5892 | 3.3798 | +5.8% | t0-gamma | — |
| uk covid utla 1W/cumulativeuk_covid_utla_1W/cumulative | fev-bench | 15.3350 | 15.8831 | +3.5% | TimesFM-3 | — |
| us consumption 1Mus_consumption_1M | fev-bench | 1.4469 | 1.4049 | +2.9% | t0-gamma | — |
| us consumption 1Qus_consumption_1Q | fev-bench | 1.7103 | 1.5001 | +12.3% | t0-gamma | — |
| walmartwalmart | fev-bench | 0.6877 | 0.6672 | +3.0% | t0-gamma | — |
| world life expectancyworld_life_expectancy | fev-bench | 1.1368 | 1.0146 | +10.7% | t0-gamma | — |
| world tourismworld_tourism | fev-bench | 2.6767 | 2.8831 | +7.2% | TimesFM-3 | — |
FREQUENTLY ASKED QUESTIONS
TimesFM-3 vs t0-gamma FAQ
How does TimesFM-3 compare to t0-gamma in forecasting accuracy?
Across the 100 standardized tasks in fev-bench, t0-gamma achieved lower Scaled Quantile Loss (SQL) on 54 tasks, recording a head-to-head win rate of 54.0%.
What is the latency difference between TimesFM-3 and t0-gamma?
TimesFM-3 records a median end-to-end latency of 3.41s per 100 time series, compared to 4.38s for t0-gamma.
Can TimesFM-3 or t0-gamma run locally in the browser?
Both models are designed for Python backend servers and require remote GPU infrastructure or cloud APIs.