feat: simplify tool

[timurcarstensen]

This PR removes a lot of stuff that was previously automated, namely:

• disk-caching hf hub download calls
• pre-downloading datasets for all tasks the user wants to eval

This allows us to remove a bunch of code and place more responsibility on the user. They must now ensure the presence of the dataset in HF_HOME if they want to eval a task no in task-groups.yaml.

We are now also able to remove a bunch of dependencies, like numpy, torch, lighteval and lm-eval which makes the tool significantly leaner and reduces import times

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Cache Flores datasets per language config

The Flores task groups declare only dataset: facebook/flores at group level and no per-task subset, so _pre_download_datasets_from_specs ends up calling load_dataset without a config. For facebook/flores that means either a failure (config required) or only the default/first config is cached, leaving the language-specific configs needed by tasks such as flores200:bul_Cyrl-eng_Latn unavailable offline. Running these task groups with skip_checks=False will therefore halt during pre-download or force compute nodes to hit the network. Please specify the language-specific subsets for these tasks so each translation pair is cached ahead of time.

Useful? React with 👍 / 👎.