Question Regarding the Llada Parallel decoding in GSM8K

[littlestone111]

Hi,
Thanks for this great work. I am having an issue understanding how the parallel decoding is enabled for LLADA.

In the eval_gsm8k.sh, this is the suggested command for dual cache+parallel.

dual cache+parallel
accelerate launch eval_llada.py --tasks ${task} --num_fewshot ${num_fewshot}
--confirm_run_unsafe_code --model llada_dist
--model_args model_path=${model_path},gen_length=${length},steps=${length},block_length=${block_length},use_cache=True,dual_cache=True,threshold=0.9,show_speed=True

In eval.md,

accelerate launch eval_llada.py --tasks ${task} --num_fewshot ${num_fewshot}
--confirm_run_unsafe_code --model llada_dist
--model_args model_path='GSAI-ML/LLaDA-8B-Instruct',gen_length=${length},steps=${steps},block_length=${block_length},use_cache=True,dual_cache=True,threshold=0.9,show_speed=True

To the best of my knowledge, the steps should enable parallel decoding. However, when I do steps=${steps} for Llada, the generation makes no sense. Do you have any suggestions in this matter? Thank you so much!

[hills-code]

Are you using the latest branch of our code? I can generate reasonable results when steps=${steps}. Here is the code.

cd llada
export HF_ALLOW_CODE_EVAL=1
export HF_DATASETS_TRUST_REMOTE_CODE=true

task=gsm8k
length=256
block_length=32
num_fewshot=5
steps=$((length / block_length))
factor=1.0
model_path='GSAI-ML/LLaDA-8B-Instruct'

accelerate launch eval_llada.py --tasks ${task} --num_fewshot ${num_fewshot} --confirm_run_unsafe_code --model llada_dist --model_args model_path=${model_path},gen_length=${length},steps=${length},block_length=${block_length},use_cache=True,dual_cache=True,threshold=0.9,show_speed=True

[littlestone111]

Yes. When I use steps=${length} (the bash command you shared), I can produce reasonable results. I am confused about the difference here.