How to do c-RLFT?

[whosaytree]

As the essay said, the fine-tuning proccess consists two parts: c-SFT and c-RLFT. However, it seems like the train process in code is only c-SFT. I don't know how to do c-RLFT. I'll be very grateful if someone can give me some help, thanks!

[imoneoi]

C-RLFT is weighted C-SFT. You can use the "weight" field in the training data format to set up weights for different conditions.