Skip to content

Implement Brier Score and Log Score #4

@jc-macdonald

Description

@jc-macdonald

Summary

Standard proper scoring rules for probabilistic classification / binary events.

Functions

Function Description
brier_score(obs::Bool, prob::Real) Brier score for binary event
brier_score(obs::AbstractVector{Bool}, probs::AbstractVector) Vectorized
log_score(obs::Bool, prob::Real) Logarithmic score
log_score(obs::AbstractVector{Bool}, probs::AbstractVector) Vectorized

Algorithm

$$ \text{BS} = (p - o)^2 \quad\quad \text{LS} = -o\log(p) - (1-o)\log(1-p) $$

Acceptance Criteria

  • Perfect forecast ($p=1, o=1$) scores 0 (Brier) and 0 (Log)
  • Worst case ($p=0, o=1$) scores 1 (Brier) and Inf (Log, clamped)
  • Vectorized version matches element-wise mean

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions