Skip to content

Add ScoringRules.jl as scoring backend #32

@jc-macdonald

Description

@jc-macdonald

Summary

The scoring module is being developed as a standalone package at jcm-sci/ScoringRules.jl since proper scoring rules have broad utility beyond model criticism.

TradeStudy.jl should depend on ScoringRules.jl and wire it into the AbstractScorer dispatch system from the Protocols module (#25).

Proposed Design

struct ProperScoringRuleScorer <: AbstractScorer
    metric::Symbol  # :crps, :energy, :wis, :brier, etc.
end

function score(s::ProperScoringRuleScorer, obs, pred)
    ScoringRules.score(s.metric, obs, pred)
end

Depends On

Acceptance Criteria

  • ScoringRules added to [deps] in Project.toml
  • ProperScoringRuleScorer exported and documented
  • Integration test using CRPS on synthetic data

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions