Hourly Ensemble Forecast CRPS: Compute and Benchmark

Written by: Olivier Lam, Physical AI Team, Jua.ai AG | Last updated: June 27, 2026

Key Takeaways for Energy Traders

• The closed-form ensemble CRPS equation gives fast, production-grade scoring of hourly wind and solar forecasts without Monte-Carlo integration.
• CRPSS against climatology quantifies real trading value. A 0.15 CRPSS at 24 h can translate to roughly €1.5 M annual savings on a 1 GW wind portfolio.
• Careful lead-time slicing, missing-data masking, and the fair-CRPS correction for small ensembles keep benchmarks unbiased and actionable.
• EPT-2e delivers lower CRPS than the ECMWF ENS mean on energy-relevant variables while updating 4× daily at up to 5 km resolution.
• See how EPT-2e performs on your data and benchmark CRPS head-to-head against your current ensemble provider on the Jua platform.

Step 1: Load Ensemble Members and Observations

Start by ingesting your ensemble array with shape (lead_times, members) and a matching observation vector of shape (lead_times,). Both must share the same hourly timestamp index. With the Jua Python SDK, ensemble members for 100 m wind or surface solar radiation download directly:

import numpy as np import pandas as pd # ens: np.ndarray, shape (T, M) — T hourly steps, M members # obs: np.ndarray, shape (T,) — station observations ens = np.load("ept2e_wind100m.npy") # replace with SDK call obs = pd.read_csv("station_obs.csv", index_col=0, parse_dates=True)["wind_speed"].values 

Align timestamps before any scoring step. Mask missing observations instead of zero-filling them, because a zero-filled gap suppresses CRPS artificially and biases lead-time slices.

Step 2: Apply the Closed-Form Ensemble CRPS Equation

The closed-form ensemble CRPS decomposes into two terms: the mean absolute error of ensemble members against the observation and a spread penalty that rewards ensemble diversity:

CRPS(x, y) = (1/M) Σi=1M |xi − y| − (1/2M²) Σi=1M Σj=1M |xi − xj|

The first term penalises bias. The second rewards spread. A perfectly reliable ensemble, whose spread matches its error, minimises both simultaneously. In NumPy:

def crps_ensemble(ens: np.ndarray, obs: np.ndarray) -> np.ndarray: """ ens : (T, M) ensemble members obs : (T,) observations returns: (T,) CRPS per time step """ M = ens.shape[1] mae_term = np.mean(np.abs(ens - obs[:, None]), axis=1) spread_term = np.mean( np.abs(ens[:, :, None] - ens[:, None, :]), axis=(1, 2), ) / (2 * M) return mae_term - spread_term 

This vectorised form avoids Python loops and scales to continental grids. For M = 10 members, EPT-2e’s published ensemble depth, and T = 8,760 hourly steps, runtime on a modern CPU stays under one second.

Step 3: Aggregate Hourly CRPS and Slice by Lead Time

Next, aggregate raw per-step CRPS into lead-time bins. A forecast issued at initialization time t0 has lead time τ = t − t0. Slice by τ to produce a CRPS profile across the forecast horizon:

def crps_by_leadtime(crps_scores: np.ndarray, lead_hours: np.ndarray) -> pd.Series: df = pd.DataFrame({"crps": crps_scores, "lead_h": lead_hours}) return df.groupby("lead_h")["crps"].mean() 

For day-ahead trading, focus on lead times from 12 to 36 h. For intraday trading, focus on 1 to 12 h. EPT-2e updates 4x/day, so each initialization cycle produces a fresh lead-time profile. Pool across cycles before aggregating to reduce sampling noise.

Step 4: Compute the CRPSS Skill Score Against Climatology

CRPSS, the Continuous Ranked Probability Skill Score, measures improvement over a reference forecast, typically climatology:

CRPSS = 1 − CRPSmodel / CRPSref

CRPSS = 1 is a perfect forecast. CRPSS = 0 matches climatology. CRPSS < 0 is worse than climatology. Build the climatological reference by computing CRPS against the empirical distribution of historical observations at each hour-of-year:

def crpss(crps_model: np.ndarray, crps_clim: np.ndarray) -> np.ndarray: return 1.0 - crps_model / crps_clim 

Report CRPSS by lead-time bin. A CRPSS of 0.15 at 24 h lead time means the ensemble is 15% more skillful than climatology at that horizon. That edge matters in day-ahead power markets, where a 1 GW wind portfolio that gains four percentage points of forecast accuracy saves approximately €1.5 M per year.

Step 5: Interpret CRPS for Wind and Solar Trading

CRPS uses the same units as the forecast variable, such as m/s for 100 m wind or W/m² for surface solar radiation (SSRD). Lower CRPS means tighter, better-calibrated probabilistic forecasts, which directly reduces imbalance costs and improves hedging precision.

For wind, a CRPS reduction of 0.3 m/s at 24 h lead time on a 100 m wind forecast translates to measurably lower generation uncertainty at hub height. That variable drives turbine dispatch. For solar, SSRD CRPS reductions at lead times of 6 to 18 h matter most for intraday balancing, where cloud-cover uncertainty dominates.

EPT-2e significantly surpasses the ECMWF ENS mean for energy-relevant variables. EPT-2e updates 4x/day and Jua’s models can natively forecast up to 5 km resolution, which delivers ensemble CRPS profiles at the spatial granularity that asset-level trading requires. The EPT-1.5 technical report provides further details on the EPT family.

Jua is a foundation model and agent company. Jua for Energy is the first applied product, similar to the relationship Anthropic has to Claude Code. EPT-2e is a variant of the Earth Physics Transformer (EPT), a general spatiotemporal physics foundation model, not a purpose-built weather model. Athena, Jua’s AI agent instrumented with the Jua for Energy tool surface, can run a full CRPS backtest across years of hindcast data in approximately five minutes from a natural-language query.

Run a live comparison on the Jua platform and see CRPS benchmarks for your region and variable against EPT-2e and ECMWF ENS.

Step 6: Avoid Common Pitfalls with Ensemble Size

Even with a production-ready CRPS implementation and access to high-quality ensemble forecasts like EPT-2e, several implementation mistakes can distort your benchmarks. The most critical pitfall is ensemble size bias. The closed-form CRPS estimator is biased for small M.

The spread term (1/2M²) Σ|xi − xj| underestimates true ensemble spread when M is small, which causes CRPS to appear artificially high. Apply the fair CRPS correction, multiplying the spread term by M/(M−1), when M < 20:

spread_term_fair = spread_term * M / (M - 1) crps_fair = mae_term - spread_term_fair 

Beyond ensemble size, three data-handling mistakes can bias CRPS benchmarks. First, mixing ensemble members from different initialization times inflates apparent spread because you combine forecasts with different information states. Always slice by initialization cycle before scoring. Second, comparing raw CRPS values across variables with different units is meaningless because CRPS uses the forecast variable’s units. Normalize by climatological standard deviation or use CRPSS for cross-variable comparisons. Third, aggregating CRPS across lead times without weighting by sample size produces misleading headline numbers when data coverage is uneven, because sparse data at long lead times can dominate the mean.

Single Copy-Paste Python Snippet for Hourly CRPS

The function below accepts an hourly ensemble array and returns mean CRPS per lead time, with the fair-CRPS correction applied:

import numpy as np import pandas as pd def hourly_crps_by_leadtime( ens: np.ndarray, obs: np.ndarray, lead_hours: np.ndarray, fair: bool = True, ) -> pd.Series: """Compute mean CRPS per lead-time hour for a finite ensemble. Parameters ---------- ens : np.ndarray, shape (T, M) — ensemble members obs : np.ndarray, shape (T,) — observations (same units) lead_hours : np.ndarray, shape (T,) — integer lead time in hours fair : bool — apply fair-CRPS small-ensemble correction Returns ------- pd.Series indexed by lead hour, values = mean CRPS """ T, M = ens.shape mask = ~np.isnan(obs) ens, obs, lead_hours = ens[mask], obs[mask], lead_hours[mask] mae_term = np.mean(np.abs(ens - obs[:, None]), axis=1) spread_term = np.mean( np.abs(ens[:, :, None] - ens[:, None, :]), axis=(1, 2), ) / (2 * M) if fair and M > 1: spread_term = spread_term * M / (M - 1) crps = mae_term - spread_term return ( pd.DataFrame({"crps": crps, "lead_h": lead_hours}) .groupby("lead_h")["crps"] .mean() .rename("CRPS") ) 

Example CRPS and CRPSS Values by Lead Time

The table below presents illustrative CRPS and CRPSS values for EPT-2e on 100 m wind (m/s) and surface solar radiation (W/m²), with CRPSS computed against a climatological reference. All benchmark figures are drawn from arXiv:2507.09703 and arXiv:2410.15076. EPT-2e shows strong performance versus the ECMWF ENS mean for energy-relevant variables.

Note: Figures are representative of EPT-2e performance as reported in arXiv:2507.09703. Exact values vary by region, season, and station network. Run your own benchmark on the Jua platform for site-specific numbers.

See EPT-2e CRPS Benchmarks on Your Own Region

The Jua platform puts 25+ models, including EPT-2e and ECMWF ENS, on a single benchmarking surface. You select any region, any variable, any time window, and the platform returns a head-to-head CRPS comparison in seconds. Athena can extend that into a full backtest across years of hindcast data in approximately five minutes.

Schedule a benchmarking session to run a live CRPS comparison on your portfolio’s region and variables.

Frequently Asked Questions

Why CRPS Beats RMSE for Ensemble Forecasts

CRPS (Continuous Ranked Probability Score) is a proper scoring rule that evaluates the full predictive distribution of an ensemble forecast against a single observation. RMSE measures only the error of a point estimate, typically the ensemble mean, and discards all information about spread and calibration. CRPS rewards an ensemble that is both accurate and well-spread, so a forecast that places high probability mass near the observed value scores better than one that is accurate on average but overconfident or underconfident. For energy trading, where the tails of the wind or solar distribution drive imbalance costs, CRPS is the operationally relevant metric. A 10% CRPS reduction at 24 h lead time on 100 m wind translates directly to tighter generation uncertainty bounds and lower hedging costs.

Minimum Ensemble Size for Reliable Hourly CRPS

The closed-form CRPS estimator is unbiased in expectation for any M ≥ 2, but variance is high for small ensembles. In practice, M ≥ 10 is sufficient for stable hourly CRPS estimates when aggregated across a full season of lead-time bins. For M < 20, apply the fair-CRPS correction, multiplying the spread term by M/(M−1), to remove the small-sample bias. EPT-2e ships with 10 members as its published ensemble depth, which is sufficient for production CRPS scoring when the fair correction is applied and scores are aggregated across multiple initialization cycles.

Typical CRPSS Targets for Day-Ahead Wind and Solar

CRPSS is interpreted relative to a climatological reference. CRPSS = 0 means the ensemble adds no skill over climatology, and CRPSS = 1 is a perfect forecast. For day-ahead, 24 h lead time, 100 m wind forecasts over European markets, a well-calibrated operational ensemble typically achieves CRPSS in the range of 0.25 to 0.40 against a seasonal climatology. For surface solar radiation at the same lead time, CRPSS values of 0.25 to 0.35 are typical. Values below 0.10 at any lead time within the 0 to 72 h window indicate a poorly calibrated ensemble or a mismatch between the ensemble’s spread and its actual error. Both cases should trigger model review before the forecast is used in trading decisions.

How EPT-2e Compares to ECMWF ENS on CRPS

As documented in the EPT-2 technical report (arXiv:2507.09703), EPT-2e outperforms ECMWF ENS on the energy-relevant variables that drive trading decisions. EPT-2e achieves this with 10 members against the ENS’s 50, which means the per-member information content is substantially higher. For energy firms that currently benchmark against ECMWF ENS as their probabilistic reference, EPT-2e represents a direct, like-for-like improvement on the metric that drives imbalance and hedging cost calculations.

Using CRPS on Power Output Forecasts

The closed-form ensemble CRPS equation applies directly to power output forecasts, not just raw meteorological variables. Power output forecasts in MW or GW can be scored directly against metered generation observations using the same NumPy function described in this guide. The key requirement is that ensemble members and observations share the same unit and the same spatial aggregation, for example both representing total wind generation for a control zone, not a mix of hub-height wind speed and aggregated MW. CRPSS against a climatological power-output distribution is the recommended skill metric for reporting to trading desks, because it normalizes across seasons and regions.

Conclusion: From CRPS Theory to Trading-Ready Benchmarks

Computing hourly ensemble forecast CRPS in production requires four components. You need the closed-form ensemble equation applied per time step, proper handling of missing observations, lead-time slicing by initialization cycle, and CRPSS normalization against a climatological reference. The six steps above, from array ingestion through energy-trading interpretation, provide a complete, copy-paste-ready implementation in NumPy and pandas.

The benchmark anchor is concrete. EPT-2e delivers the CRPS improvements over ECMWF ENS documented earlier, with 4x daily updates and up to 5 km native resolution, which matches asset-level energy trading requirements.

Jua is a foundation model and agent company. Jua for Energy is the first applied product, used by Axpo, TotalEnergies, Statkraft, EnBW, EDF, and Hydro-Québec. The Jua platform exposes EPT-2e alongside 25+ models, including ECMWF ENS, through a single API and Python SDK, with hindcast data available for multi-year CRPS backtests. Athena, Jua’s AI agent instrumented with the Jua for Energy tool surface, turns a natural-language backtest request into a full CRPS report in approximately five minutes.

Book a demo to see EPT-2e CRPS benchmarks on your region and variables, head-to-head against ECMWF ENS and your current ensemble provider.