Artificats/Patterns in custom training on SEN2NAIP dataset

[CristianCerasuoloo]

Description

I am seeking insight regarding significant artifacts appearing during ESRGAN training on the SEN2NAIP dataset. These appear to be convolutional or checkerboard artifacts and are visible across the entire image, particularly in the low-frequency regions.

Training Details

The issue persists across different training stages:

PSNR-oriented pre-training: Artifacts are present even when training with only content loss (no Discriminator).

Full GAN training: The problem remains visible until early stopping.

Attempted Mitigations

To address this (an issue already present in the original repository), I have implemented several features from the original ESRGAN paper, though they have not yet resolved the problem:

Smaller initialization: Adjusted weights initialization to improve stability.

Relativistic Average Loss (RaGAN): Implemented for the GAN phase.

Perpetual Pre-training: Added the g_pretrain_steps: -1 feature to allow continuous generator pre-training with content loss, as suggested in the original literature.

This justifies some new keys available in the attached configurations.

Configurations

I have attached the configuration files used:

Pre-training config: Generator-only with content loss.

Full training config: Resumed training with combined losses.

And some visual examples:
Please zoom into the low-frequency parts of the attached W&B examples to better visualize the pattern.

pretraining.yaml
discriminator.yaml

[simon-donike]

Hi @CristianCerasuoloo , thanks a lot for opening the detailed issue. I'll try to reproduce and report back in the next days!

[CristianCerasuoloo]

Many thanks @simon-donike, if some details are missing let me know and I'll report it!

[simon-donike]

I'm trying to train the ESRGAN on SEN2NAIP currently, but can't get the models to converge at all. It would be amazing if you could open a PR for the "RaGAN" and the "Perpetual Pre-training", it would be great if you could make that contribution! Ideally into the new branch ESRGAN which I just created, where I'd like to try to fix the ESRGAN behavior.

[CristianCerasuoloo]

Yes I could try but I had convergence after many iterations also without RaGAN. So I don’t think that’s the problem in you le case

[CristianCerasuoloo]

PR #106 (comment) generated. As you requested I implemented "RaGAN" and the "Perpetual Pre-training". I also made other minor changes in the codebase like "Before activation" evaluation of perceptual loss, "smaller inizialization" and more importantly i fixed a bug caused by the 2 schedulers where "LambdaLR" scheduler would continue stepping even after warmup completion but using initial LR and not the one set by the subsequent scheduler.