Open
Conversation
This was referenced Jun 14, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This extension works with the SD1.5 models but fails with SDXL. In this code update, I checked the source code and identified that SDXL at this line: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/feee37d75f1b168768014e4634dcb156ee649c05/modules/sd_hijack_clip.py#L349.
The current logic only checks
shared.sd_model.cond_stage_model.wrapped, while the hijacked SDXLFrozenCLIPEmbedderForSDXLWithCustomWordsis located inshared.sd_model.cond_stage_model.embedders[0].wrapped.I lack the technical expertise to confirm differences in the tokenizer. However, SD1.5 and SDXL appear to produce the same ID, source code also suggests the vocab maps to Unicode, making any model suitable.
In this pull request, I refactored the logic so
VanillaClipandOpenClipno longer depend on instance checks. They now verify the required attribute. I passedembedders[index].wrappedto the Clip classes, so the first matching attribute is used for tokenization. Since SDXL and SD1.5 produce the same tokenization, I didn't include logic to identify the best "embedder" (which should beFrozenCLIPEmbedderForSDXLWithCustomWords), as they likely yield the same result.Lastly, token colors were barely visible in dark theme. I updated them to use the theme color instead of hardcoded RGB.