[Question] Do we need to set attention implementation to eager in config and in rotary setup component testing method?

[TensorCruncher]

Question

Qwen3 adds self.cfg.attn_implementation = "eager" to the config.

It further adds the following code to the rotary setup component testing method:

if hasattr(hf_model, "config") and hasattr(hf_model.config, "_attn_implementation"):
    hf_model.config._attn_implementation = "eager"

if hasattr(hf_model, "model") and hasattr(hf_model.model, "layers"):
    for layer in hf_model.model.layers:
        if hasattr(layer, "self_attn") and hasattr(layer.self_attn, "config"):
            layer.self_attn.config._attn_implementation = "eager"

Is this something we should replicate elsewhere? Isn't eager the default attention implementation? So the code block exists just to ensure the bridge component and HF instance both execute eager attention?

[jlarson4]

You'll need to replicate this for any model that does its own attention math, to ensure we can test that attention math as part of the benchmarks. We do need to force eager attention, transformers default attention is sdpa. You are correct in your assessment, this code block exists to make sure HF and the bridge are both executing eager attention.

These lines in the setup_component_testing block force eager on the separate HF reference model used for component comparison in the benchmarks. The per-layer loop is needed because HF attention layers read _attn_implementation from their own per-layer config at forward time, so setting only the top-level config doesn't reach the already-built layers. All of this allows us to compare our attention bridge components to HF's attention components.

22 of our adapters have this pattern. The remaining adapters that do not have this pattern are missing it because they defer to HF's attention. Since the attention is the same, no need to set it up and compare it in our benchmarks, that's just comparing A == A.

In the specific case of the HunYuan adapter you're working on presently, you probably need to do some work on the attention to expose hooks, similar to Qwen3, so you will also need this eager solution.

One note: this boilerplate is copy-pasted across many adapters (doing a search for hf_model.config._attn_implementation = "eager" turns up 23 matches), so it'd be a good candidate to refactor into a shared base helper at some point.

[TensorCruncher]

Thank you for the clarification. For now I'll just add the necessary lines to my adapter and tests. I'll hold off on the refactor till I gain a better understanding of the codebase.

If anyone else would like to take it up, go ahead!