Summary
The CRPS is the most widely used proper scoring rule for continuous probabilistic forecasts. Implement both ensemble-based and closed-form (distributional) variants.
Functions to Implement
| Function |
Description |
crps_ensemble(obs, fct_samples) |
CRPS from ensemble/sample forecasts |
crps_normal(obs, μ, σ) |
Closed-form CRPS for normal distribution |
Algorithm
The ensemble CRPS decomposes as:
$$ \text{CRPS}(F, y) = \mathbb{E}|X - y| - \frac{1}{2}\mathbb{E}|X - X'| $$
where $X, X' \sim F$ are independent draws. The NRG (energy) representation avoids sorting:
crps = mean(abs.(fct .- obs)) - 0.5 * mean(abs.(fct .- fct'))
For the normal distribution, the closed-form uses erf and the standard normal PDF/CDF.
References
- Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. JASA, 102(477), 359-378.
- Jordan, A., Krüger, F., & Lerch, S. (2019). Evaluating probabilistic forecasts with scoringRules. JSS, 90(12).
Acceptance Criteria
Summary
The CRPS is the most widely used proper scoring rule for continuous probabilistic forecasts. Implement both ensemble-based and closed-form (distributional) variants.
Functions to Implement
crps_ensemble(obs, fct_samples)crps_normal(obs, μ, σ)Algorithm
The ensemble CRPS decomposes as:
where$X, X' \sim F$ are independent draws. The NRG (energy) representation avoids sorting:
For the normal distribution, the closed-form uses
erfand the standard normal PDF/CDF.References
Acceptance Criteria
crps_ensemblematches scoringrules (Python) on reference vectorscrps_normalmatches analytical formula