Requests and Bugs

[icpryde]

Hello, first I want to say amazing job with this and utilizing coreML this is so awesome! I was hoping I could mention a few things on a wish list that would really turn this into my main diffusion to use.

First on a few of the existing UI stuff:

• The prompt boxes are quite small, it would be nice if we could resize them or have them larger so we can see the full prompt in the window, also no way to currently scroll in the prompt as well making it hard to navigate.

• Maybe a spot somewhere that shows the current model you are using, I know you can see it in settings but sometimes after walking away I forget which model i'm using so just would be nice to see.

• you currently can upscale but only with the initial prompt, it would be nice to be able to upscale AFTER the prompts are done and you've seen them so you can choose which ones you want to upscale.

• Since you already have the RealESRGAN model built in with the upscaler, would be awesome if you could implement the face restoration feature as well

• It would be nice instead of just seeing a loading screen while it generates the images it would be nice to actually see the noise doing its thing while it creates the image as it slowly removes the noise

Features to add:

• Weights to the prompts is something that would be awesome to have

• It would be great if there was a way with the models to add VAE as a lot of the time this is requires along with the model to have it working properly

• Embeddings would be a nice feature as well to add

• different samplers/schedulers like k_euler or k_euler_a, K_dpm_2 and a bunch of others if possible

• a way to change the width/height as well to the images when prompting

Possible Bugs:

• img2img doesn't seem to work with dreambooth models this would be nice if this could work

Then of course, it would be AMAZING if inpainting can be implemented as well but I think the above features would be more useful until that can be done.

Amazing work! I hope this keeps getting developed because its already so promising!

[icpryde]

Also I saw this post from Apple Github on how to implement the noise

apple/ml-stable-diffusion#85 (comment)

[TheMurusTeam]

About your requests:
I've just released build 5 on TestFlight.

• Prompt now resizes with window
• added a label with current model and compute units
• you could already upscale after generating an image, even in previous builds. Click the magnifier button then 'upscale'
• to implement face restoration I need to find a coreml model for this purpose. Maybe gfpgan? However I will add support for custom upscale models so one can choose which one to use
• The code for displaying noisy steps is already there, but it's disabled because it reduce generation speed drastically,

About features to add:
I'm afraid all these features must be added to the framework itself before I can implement them on the app. It currently only supports 2 schedulers.

About img2img bug:
what do you mean? What happens with dreambooth models? App crash, or the GUI for img2img is hidden? Or what? Which models are you trying? Did you convert them by yourself? If so, please note that you need to convert also VAEEncoder.mlmodelc if you want img2img. If the model lacks VAEEncoder.mlmodelc, the img2img GUI appears disabled. See this PR for more info: apple/ml-stable-diffusion#73

Thanks for your comments :)