2 Questions during the process

[Sehong-Oh]

Hi, I am interested in your research and trying to implement your work.

1. how many high quality questions and low quality questions did you get in step 2 with a K-QA dataset?
   In my case, a few questions are considered as a good question, which means that I did not have enough good questions for DPO training even though I did not modify your code.

2. I wonder if I could train llama3 8b? You mentioned that 40gb is needed to train but I am not able to run it with my A100 40gb.

Thank you for your research and hope your reply.

[3244we]

1. My training set of the k-qa has 101 samples, and each sample has a combination of 10 good samples with 20 bad samples. So there are a total of about 20,000 samples at most, but in some cases I actually get less than 10,000 samples, but this is enough for training because it's very easy to overfitting.

2. I truncated the length of k-qa to 256 during training, so it can be trained. Larger lengths require higher GPU memory.

[Sehong-Oh]

Thank you for your reply. Unfortunately, I was not able to run "train.py" even though I truncated the max length as 256.
I'll try again from the beginning and let you know if If something moves forward.

Also, did you happen to train the model with DPO(w/o reference model)?

[3244we]

Thank you for your reply. Unfortunately, I was unable to run "train.py" even though I truncated the maximum length to 256. I'll try again from the beginning and let you know if something moves forward.

Also, did you happen to train the model with DPO(w/o reference model)?

Actually, you need to make the total length of the original question, plus the prompt, plus good and bad sample pairs, less than 256. I haven't tried it, but I think the results should be weird without the REWARD model.

[Sehong-Oh]

@3244we Hi, Thank you for your reply.

I have one more question. Did you fully fine-tune it, or did you use LoRA instead?
Even though we have A100 80GB(40x2) now, we are unable to do full fine-tuning llama3 8b model.

[3244we]

@3244we Hi, Thank you for your reply.

I have one more question. Did you fully fine-tune it, or did you use LoRA instead? Even though we have A100 80GB(40x2) now, we are unable to do full fine-tuning llama3 8b model.

Of course, it is fully fine-tuned.