Indicating available / applicable arms

[leandrolanzieri]

Hello! Thanks for this nice project!

I'm currently integrating mabwiser into my project. I have the situation that in given contexts certain arms are not applicable and shouldn't be considered in the prediction. This doesn't mean that I want to _drop_existing_arm, just ignore it. I'd like to keep the models for each arm.

Is there a way this could be achieved with the current API?

Thanks!

[skadio]

@leandrolanzieri thank you for the positive comments!

If I am following correctly, you should be able to achieve that with a "customized mab".

See the example here: https://github.com/fidelity/mabwiser/blob/master/examples/customized_mab.py

The idea is to inherit the policy you want, and then overwrite the predict() function, based on your decision criteria (e.g., skip some arms in your description).

In the example, we let MABWiser to calculate/manage everything under the hood, we then leverage predict_expectations() to decide the best regret, instead of the classical greedy argmax.

Hope this helps,
Serdar

[leandrolanzieri]

Thanks for the quick reply @skadio!

So, if I understood correctly then one would have to inherit from a specific policy and re-implement predict. This is needed because, as stated in the docs "The definition of the best depends on the specified learning policy.".

As I'm currently trying to implement the code independently of the chosen policy, I was looking for a policy-agnostic approach, that would allow to temporarily mark arms as "not available" or similar. Currently I have something in this direction:

expectations = policy.predict_expectations(contexts=_context)

while len(expectations) > 0:
    best_action = argmax(expectations)
    if best_action in available_actions:
        selected_action = best_action
        break
    else:
        del expectations[best_action]
else:
    # No available action
    raise RuntimeError("No applicable actions available after evaluating expectations.")

The problem is that there's no guarantee that the policy would choose argmax(expectations) as the proposed arm.

[skadio]

This seems in the right direction to me (just a minor code comment; be careful with deletion inside the loop)

The other thing, and I am sure you've considered, is to call the same custom method in each policy inheritance, or at least the policies you are most interested in. It's a bit of boiler plate code but it should be ok to have it for each policy.