Skip to content

Add an information about human metrics #195

@kordc

Description

@kordc

As in #72
human baseline. Monitor metrics other than the loss that are human-interpretable and checkable (e.g. accuracy). Whenever possible evaluate your own (human) accuracy and compare to it. Alternatively, annotate the test data twice and for each example treat one annotation as prediction and the second as ground truth.

For this we can only warn user to use metrics different than just loss.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions