Set up python environment (requirements will be generated)
-
Set up basic byt5 / t5 hugging face models (done in byt5.py)
-
Run STS-b eval (done in the evaluate_stsb.py) the simplest eval
-
Get the sentEval repo set up and added to python path (the repo is already nested inside this repo, just cd and pip install)
cd SentEval
pip install -e .
pip install nltk scikit-learn
- Download the datasets for SentEval using:
cd data/downstream
bash get_transfer_data.bash
- Run the sentEval_tests.py - see the args for more info on running them