SummaCoz

Paper Link

You can access the dataset on huggingface datasets nkwbtb/SummaCoz

from datasets import load_dataset
summacoz = load_dataset("nkwbtb/SummaCoz")

train.py: training script
testing.py: testing script for Balanced Accuracy on test sets
template.py: prompt template and functions for parsing the LLM output
merge_adapter.py: merge LoRA adapter to the original model
llama2_gen.py: generate with llama2
instruct_data.py: dataset utility for training, merge the generated reasoning back to the dataset
config.py: model and hyperparameters setting
check_overlap.py: check the overlap between different datasets, remove the duplications for training

Name		Name	Last commit message	Last commit date
Latest commit History 2,006 Commits
benchmark		benchmark
data		data
.DS_Store		.DS_Store
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
check_overlap.py		check_overlap.py
claim_tokens.png		claim_tokens.png
config.py		config.py
cot_tokens.png		cot_tokens.png
document_tokens.png		document_tokens.png
falcon_instruct_test.ipynb		falcon_instruct_test.ipynb
instruct_data.py		instruct_data.py
llama2_gen.py		llama2_gen.py
llama2_instruct_test.ipynb		llama2_instruct_test.ipynb
merge_adapter.py		merge_adapter.py
sft_testing.py		sft_testing.py
sft_training.py		sft_training.py
t5_testing.py		t5_testing.py
t5_trainer.py		t5_trainer.py
template.py		template.py
testing.py		testing.py
token_stats.py		token_stats.py
train.py		train.py
utils.py		utils.py
xsum_annot.ipynb		xsum_annot.ipynb