Skip to content

Implement CRPS (Continuous Ranked Probability Score) #1

@jc-macdonald

Description

@jc-macdonald

Summary

The CRPS is the most widely used proper scoring rule for continuous probabilistic forecasts. Implement both ensemble-based and closed-form (distributional) variants.

Functions to Implement

Function Description
crps_ensemble(obs, fct_samples) CRPS from ensemble/sample forecasts
crps_normal(obs, μ, σ) Closed-form CRPS for normal distribution

Algorithm

The ensemble CRPS decomposes as:

$$ \text{CRPS}(F, y) = \mathbb{E}|X - y| - \frac{1}{2}\mathbb{E}|X - X'| $$

where $X, X' \sim F$ are independent draws. The NRG (energy) representation avoids sorting:

crps = mean(abs.(fct .- obs)) - 0.5 * mean(abs.(fct .- fct'))

For the normal distribution, the closed-form uses erf and the standard normal PDF/CDF.

References

  • Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. JASA, 102(477), 359-378.
  • Jordan, A., Krüger, F., & Lerch, S. (2019). Evaluating probabilistic forecasts with scoringRules. JSS, 90(12).

Acceptance Criteria

  • crps_ensemble matches scoringrules (Python) on reference vectors
  • crps_normal matches analytical formula
  • Handles edge cases: single ensemble member, zero variance
  • Docstrings with mathematical formulas

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions