We should clean up our dependencies, lots are unused, for the ones we want to keep they should be installed as optional dependencies.
dependencies = [
"datasets>=4.0.0",
"fast-langdetect>=1.0.0",
"huggingface-hub>=0.34.4",
"ipython>=9.5.0",
"joblib>=1.5.2",
"jupyter>=1.1.1",
"langchain>=0.3.27",
"langchain-community>=0.3.29",
"langchain-openai>=0.3.32",
"langchain-together>=0.3.1",
"matplotlib>=3.10.6",
"pandas>=2.3.2",
"rich>=14.1.0",
"scikit-learn>=1.7.2",
"seaborn>=0.13.2",
"tqdm>=4.67.1",
]
We should clean up our dependencies, lots are unused, for the ones we want to keep they should be installed as optional dependencies.