求一个RedPajama-Data-1T-Sample中的book_sample.jsonl数据集处理的脚本，做微调任务 #56

Open

opened

on Jun 24, 2024

类似与tools/alpaca_tokenizer.py的脚本，处理book_sample.jsonl的数据

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests