We have a validator now, yet still I got the error
The job failed due to a file format error in the training file. Invalid file format. Example 6174 message contains an invalid token: <|fim_suffix|>.
That was just "<|fim_suffix|>" in the text, IDK where it came from but certainly the validator should be able to catch that. And perhaps other tokens like this.
We have a validator now, yet still I got the error
That was just "<|fim_suffix|>" in the text, IDK where it came from but certainly the validator should be able to catch that. And perhaps other tokens like this.