Unexpected key(s) in state_dict: "text_model.embeddings.position_ids".

[anthonyyuan]

validation_pipeline = I2VPipeline.build_pipeline(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/work/third/stable-diffusion-good/sd2/PIA/animatediff/pipelines/i2v_pipeline.py", line 228, in build_pipeline
text_encoder.load_state_dict(text_encoder_checkpoint)
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
Unexpected key(s) in state_dict: "text_model.embeddings.position_ids".

[LeoXing1996]

@anthonyyuan Thank you for your interest in this project.
This error appears to be caused by a mismatched transformers version. We recommend using transformers==4.25.1 for PIA. You can check your transformers version by pip list | grep transformers or conda list transformers.

[johndpope]

make sure in the Lora drop down - you're not incorrectly selecting a model or just leave blank.

[JY-Joy]

Same issue when using kohya_ss repo, simply remove the unexpected key from state dictionary, with 'del text_encoder_checkpoint["text_model.embeddings.position_ids"]' in your case, works for me

[Seedmanc]

How to do that? Where do I enter the command? Del is a windows command.

[somnbody]

How to do that? Where do I enter the command? Del is a windows command.

Hei ! I also was having this issue. For me, the following steps worked:

Find model_util.py file and open in code editor.
For me, this was in C:\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\library
as I was specifically having this issue with Lora Training in Comfy. But you will find it in a similar place.

CTRL+F find in model_util.py:

def convert_ldm_clip_checkpoint_v1(checkpoint):

Just below that section replace the following:

# support checkpoint without position_ids (invalid checkpoint)
if "text_model.embeddings.position_ids" not in text_model_dict:
    text_model_dict["text_model.embeddings.position_ids"] = torch.arange(77).unsqueeze(0)  # 77 is the max length of the text

with

# remove position_ids for newer transformer, which causes error :(
if "text_model.embeddings.position_ids" in text_model_dict:
    text_model_dict.pop("text_model.embeddings.position_ids")

Good luck!

[emosGambler]

How to do that? Where do I enter the command? Del is a windows command.

Hei ! I also was having this issue. For me, the following steps worked:

Find model_util.py file and open in code editor. For me, this was in C:\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\library as I was specifically having this issue with Lora Training in Comfy. But you will find it in a similar place.

CTRL+F find in model_util.py:

def convert_ldm_clip_checkpoint_v1(checkpoint):

Just below that section replace the following:

# support checkpoint without position_ids (invalid checkpoint)
if "text_model.embeddings.position_ids" not in text_model_dict:
    text_model_dict["text_model.embeddings.position_ids"] = torch.arange(77).unsqueeze(0)  # 77 is the max length of the text

with

# remove position_ids for newer transformer, which causes error :(
if "text_model.embeddings.position_ids" in text_model_dict:
    text_model_dict.pop("text_model.embeddings.position_ids")

Good luck!

Dude, worked like a charm. CHEERS