AutoSP Setup Guide

Quick start guide to clone and set up the AutoSP repository.

Prerequisites

• CUDA 12.8 compatible GPU (recommended)
• Conda installed
• Git

Installation

1. Clone the Repository

git clone https://github.com/spikerheado1234/AutoSP.git
cd AutoSP

2. Create Conda Environment

conda create --prefix ./autosp_env python=3.10 -y
conda activate ./autosp_env

3. Install PyTorch

pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

4. Install Dependencies

pip install \
  transformers==4.50.3 \
  tokenizers==0.15.2 \
  huggingface-hub==0.25.1 \
  safetensors==0.4.5 \
  datasets \
  accelerate \
  scipy \
  tqdm \
  pyyaml

5. Install AutoSP

pip install --no-build-isolation -e .

6. Copy partitioners.py to Torch Installation

Copy the custom need_to_copy_paste/partitioners.py from the repository to your conda environment's torch installation.

This enables custom partitioning support for the benchmarking scripts.

Benchmarking

See bench_dc_ulysses/ directory for benchmarking scripts (update sample.slurm with the correct env and paths):

cd bench_dc_ulysses
source sample.slurm
bash run_ulysses.sh {seq_len} {compile|eager|deepcompile} {num_layers}

Troubleshooting

• Build isolation errors: Use pip install --no-build-isolation -e .
• CUDA issues: Verify CUDA 12.8 compatibility with nvidia-smi
• Out of memory: Adjust batch size or sequence length parameters