how many pictures needed?

[humworkb]

hi, thanks for your fantastic work.
I got a question. how many pictures needed for train a model to translate class A to class B?

[ZouShilong1024]

Hello! For training CycleDiff, a larger dataset typically results in higher translation quality. We recommend using at least 3,000 images per category to achieve better performance.

[humworkb]

@ZouShilong1024 thanks for your reply. Recently, I was trying to train vae for class A, but I got some problems:
1.the training is slow about 2200step/36h
2.disc_loss is always 0.00000, logits_real from-0.7 to -0.68, logits_fake from -0.70 to -0.77, lr always 0.00000

what's training requirement of hardware? or can you give me some advice for training? Thanks a lot!

[ZouShilong1024]

Thanks for your feedback and detailed report.
We conducted our training on a single A100 GPU, and also tested it on RTX 3090 and RTX 4090 — all worked without issues.
For your first question, the training speed on an A100 is about 1.10 it/s.
For the second question, the training log you described looks normal, and we suggest training for a longer time to observe better results. You can also check the generated images in the checkpoint directory during training to monitor progress.
Hope this helps!

Reference training log:

[humworkb]

@ZouShilong1024 thanks again!
As for your speed, maybe it takes 17 hours for 70000 steps to train a vae?
How long to train the whole model(2 vae+2 ldm +1ldm_cycle)?
BTW, how long to inference a picture?

[ZouShilong1024]

Thanks for your question.
① The VAE can be trained for fewer iterations (around 30,000–40,000 steps) if you are using pretrained weights.
② The whole model training takes about 3–4 days in total — where VAE + LDM training requires around 2–3 days, and the cycle translator training takes about 1 day. The cycle translator relies on a pretrained diffusion model for initialization, which makes its training relatively time-consuming. This is also one of the potential directions for future optimization.
③ For inference speed, as reported in our paper, CycleDiff runs at approximately 4.9 seconds per iteration with a batch size of 1.
Hope this helps!

[jsjnrxx2018]

The asymmetric dataset used has approximately 500 images in dataset A and approximately 450 images in dataset B. The model, trained for 140,000 steps, performs poorly when transferring from A to B, but performs exceptionally well when transferring from B to A. What could be the reason for this?

[ZouShilong1024]

The asymmetric dataset used has approximately 500 images in dataset A and approximately 450 images in dataset B. The model, trained for 140,000 steps, performs poorly when transferring from A to B, but performs exceptionally well when transferring from B to A. What could be the reason for this?

To help understand why A→B performs poorly while B→A works well, could you share a bit more about the training setup—specifically whether this is a cross-modal translation task and the exact number of images in domain A and domain B for training? Thanks!

[jsjnrxx2018]

不是跨模态翻译任务，是RGB<->RGB的翻译任务，在 186k→187k 之间发生了数值崩溃，模型已经炸掉了。目前是每一万步保存一个模型，采用model-15.pt，model-16.pt，model-17.pt进行测试，发现翻译后的图像并不能与原图像的结构保持一致，如下图所示。

[jsjnrxx2018]

白天图像951张，夜晚图像798张，数据集中图像的像素大小并不统一，数据集中的图片是未配对的