Question regarding WISE implementation on Multimodality data

[boilerchun]

Thanks for the work!

I have a question regarding the implementation of WISE on multimodality dataset.

If I understand correctly, the current implementation of WISE on multimodality dataset seems directly use the locality data (which used for testing/evaluation) during the training. Wouldn't this result in leaking the test information?

Also, as the comment points out, the loc_prompt is not equivalent to locality prompt. However, the implementation seems they are equivalent.

EasyEdit/easyeditor/models/wise/utils.py

Line 241 in a642707

# TODO: locality_prompt(Unrelated Knowledge) is not equal to loc_prompt(Subject phrase)!

[sunnychenxiwang]

You can also use multimodal locality data for training.

[zxlzr]

hi, do you have any further questions?