Model degeneration issue?

[astradzhao]

Hello,

I have tried training my own version of DiffuLLaMA but unfortunately I have ran into model degeneration when attempting to generate text outputs. When generating answers using the scoring function for datasets like HSwag, Wino, SIQA, etc. I can get somewhat comparable performance to that of the paper (~40-50% acc for each dataset), but I am unable to produce any reasonable text output for other tasks like Gsm8k. (The model only generates a bunch of new line and comma tokens)

I have trained on Llama-7b on 1 split of SlimPajama with these hyperparameters:
batch-size 16
gradient-accumulate-every 8
max-train-steps 1000
learning-rate 1.5e-5
seq-length 2048

I was wondering if I need to change any of these hyperparameters or perhaps am required to do training on more than just 1 split in order to produce meaningful text outputs? but I also had to change some stuff in training script for training to work on my end, so there may be some errors in my training script that there were not in the original.

Would just love some guidance here, as me and my lab are trying to do some followup work on this paper.

Thank you!

[astradzhao]

For example for this prompt: "Question: Darrell and Allen's ages are in the ratio of 7:11. If their total age now is 162, calculate Allen's age 10 years from now.
Let's solve this step by step:"

I get following output:
"

,
but

.

,

1

is

,

.

,

,

',

,

1,
the

\

, and 000

and
and ,

00,0,00s., and,
0
but are,
,0,

."

[summmeer]

This is a common issue for not-well-trained diffusion language models.
Suggestions for model degeneration:

1. Try to train more steps with more data.
2. Try different decoding algorithms. Here we have "random mask" strategy and "low confidence remask" strategy. "random mask" shows better for unconditional generation while the latter one shows better for condition generation.

[tara-tan]

2. Try different decoding algorithms. Here we have "random mask" strategy and "low confidence remask" strategy. "random mask" shows better for unconditional generation while the latter one shows better for condition generation.

Hi, where can I find the setting for this? also can a model trained on one be used for another (if the algorithm choice does effect training)?

[summmeer]

Hi @tara-tan Training is the same. This is just decoding-only algorithm. You can find in:

DiffuLLaMA/LLaMA-Factory/src/llamafactory/train/ddm/trainer.py

Line 357 in c17e897

def generate_samples(model, diff_args, tokenizer, inputs, eval=False, gsm=False):

DiffuLLaMA/LLaMA-Factory/src/llamafactory/train/ddm/trainer.py

Line 467 in c17e897

def generate_samples_v2(model, diff_args, tokenizer, inputs, eval=False):