Added Autoresearch task

[ParamThakkar123]

Changes

• Added autoresearch/ directory with complete autoresearch framework
• Included gui.py for desktop GUI training interface with VRAM and LR controls
• Added prepare.py for data preparation, tokenizer training, and dataloader
• Created train.py with model architecture, training loop, and export functionality
• Added program.md with comprehensive documentation for autonomous experimentation
• Included task.yaml with resource configuration and setup dependencies

Features

• Autonomous LLM research framework with fixed time budgets
• GUI interface for manual experimentation with visual controls
• Support for custom model architectures and hyperparameters
• Automatic model export and evaluation (bits per byte metric)
• Data preparation pipeline with BPE tokenizer training
• Comprehensive logging and results tracking in TSV format

Parameters

• depth: Model depth (layers)
• device_batch_size: Batch size per device
• total_batch_size: Total batch size across devices
• Various learning rates: matrix_lr, embedding_lr, unembedding_lr
• weight_decay, window_pattern for training configuration

How to Test

1. In TransformerLab, select the 'autoresearch-task'
2. Configure model and training parameters as desired
3. Run the task (note: requires significant GPU resources - RTX3090 recommended)
4. Monitor training progress through logs
5. After completion, check exported models and results in the output directory

Note: This task performs autonomous model training experiments. Ensure adequate VRAM and time for the 5-minute training budget.

[greninja]

I am stuck in this error for now:

Error: Command '['/home/shadab/.transformerlab/miniforge3/bin/conda', 'run', '--prefix', '/home/shadab/.transformerlab/envs/transformerlab', 'uv', 'pip', 'install', '--python', '/home/shadab/.transformerlab/local_provider/local_provider_runs/orgs/6091344b-f07a-4120-a019-c761f3cc30c2/195d9cd3-cf3e-43a6-b8ca-6d79ccdee8ce/workspace/venv/bin/python', '.[nvidia]']' timed out after 900 seconds

will take a look at it again later, but did you face this when importing the task in the transformerlab app? For what its worth, the task.yaml looks ok for this one.

not requesting changes for now since I haven't faced any issue or error with the code till now.