-
Notifications
You must be signed in to change notification settings - Fork 2
Attribution of instances closed by two algorithms #37
Description
Arising from Issue #13.
When comparing algorithms on "Instances Closed" plots, it is possible that some closed instances will not be attributed to any algorithm.
For example: an optimal method claims a best known bound equal to C, but it does not claim any plan. Another solver claims a best known solution with cost equal to C, but it does not claim any lower-bound. The instance is closed, yet according to the current filtering method, it does not appear in this plot. The problem arises because we only consider an instance closed if a submission reports a solution cost and a matching lower bound value.
How should we handle/attribute instances closed by a pair of algorithms working together? Suggest that we need an "Other" category in the corresponding plots.