Skip to content

Inconsistent SPICE scores #56

@brandbir

Description

@brandbir

It was found that in pycocoevalcap/spice/spice.py, line 29 is sorting the img_ids prior the generation of SPICE metrics. The result metrics are then extracted and mapped to the corresponding image ids in pycocoevalcap/eval.py by the setImgToEvalImgs() function. However, the latter function is using an unsorted img_ids list. This incompatibility between the sorted and unsorted lists makes the SPICE metrics being mapped to incorrect image ids. This does not have any effect on the overall SPICE metric as it is only a problem of misalignment between the computed SPICE metrics and their corresponding image ids. For consistency, no sorting is needed for imgIds in pycocoevalcap/spice/spice.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions