Skip to content

SETUP_GUIDE.md

Latest commit

 

History

History
242 lines (176 loc) · 5.63 KB

SETUP_GUIDE.md

File metadata and controls

242 lines (176 loc) · 5.63 KB

CW2: DistilBERT Exploration - Setup Guide

Student: Martynas Prascevicius Student ID: 001263199 Course: COMP1818 Artificial Intelligence Applications

Quick Start

1. Install Requirements

cd /Users/m2000uk/Desktop/coding/AI/CW2

# Use parent directory's virtual environment (recommended)
# OR install packages globally
pip3 install torch transformers datasets scikit-learn numpy pandas matplotlib seaborn tqdm

2. Verify Setup

# Test enhanced model
python3 src/enhanced_model.py

# List all experiments
python3 src/experiment_runner.py --list

3. Run Your First Experiment

# Run baseline (reproduces CW1 result: 91.16%)
python3 src/experiment_runner.py --experiment baseline_default

This will:

  • Download IMDB dataset automatically
  • Train DistilBERT for 3 epochs (~30-45 minutes on Mac M4)
  • Save results to results/baseline_default.json

Running Experiments

Option 1: Single Experiment

python3 src/experiment_runner.py --experiment lr_3e5

Option 2: Entire Phase

# Run all Phase 2 experiments (learning rates)
python3 src/experiment_runner.py --phase 2

Option 3: All Experiments (WARNING: ~20 hours!)

python3 src/experiment_runner.py --all

Analyzing Results

View Summary

python3 src/results_analyzer.py

Shows:

  • Total experiments completed
  • Best overall accuracy
  • Best configuration details
  • Results by phase

Compare All Experiments

python3 src/results_analyzer.py --compare-all

Analyze Specific Phase

python3 src/results_analyzer.py --phase 2

Compare Specific Hyperparameter

python3 src/results_analyzer.py --hyperparameter learning_rate

Export for LaTeX

python3 src/results_analyzer.py --export-latex

Creates experiments/results_table.tex ready for your report!

Recommended Experiment Sequence

Week 1: Baseline + Learning Rates (Phases 1-2)

python3 src/experiment_runner.py --phase 1  # Baseline: ~45 min
python3 src/experiment_runner.py --phase 2  # Learning rates: ~3 hours

Goal: Find optimal learning rate (likely 2e-5 or 3e-5)

Week 2: Batch Sizes + Duration (Phases 3-4)

python3 src/experiment_runner.py --phase 3  # Batch sizes: ~2 hours
python3 src/experiment_runner.py --phase 4  # Epochs: ~2 hours

Goal: Balance speed and accuracy

Week 3: Sequence Length + Schedulers (Phases 5-6)

python3 src/experiment_runner.py --phase 5  # Max length: ~2 hours
python3 src/experiment_runner.py --phase 6  # Schedulers: ~2 hours

Goal: Optimize training schedule

Week 4: Advanced Fine-tuning (Phases 7-11)

python3 src/experiment_runner.py --phase 7   # Regularization
python3 src/experiment_runner.py --phase 8   # Layer freezing
python3 src/experiment_runner.py --phase 10  # Pooling strategies

Goal: Explore advanced techniques

Week 5: Best Configuration + Report Writing

# Update optimized_v1 in experiment_configs.py with best hyperparameters
# Then run it
python3 src/experiment_runner.py --experiment optimized_v1

# Export for LaTeX
python3 src/results_analyzer.py --export-latex

Project Structure Reference

CW2/
├── src/
│   ├── experiment_configs.py      # All experiment definitions
│   ├── enhanced_model.py          # DistilBERT with advanced features
│   ├── experiment_runner.py       # Automated training
│   └── results_analyzer.py        # Results comparison
│
├── results/                       # JSON files (one per experiment)
├── figures/                       # Visualizations (to be generated)
├── experiments/                   # Logs and LaTeX exports
├── literature/
│   └── references_distilbert.bib  # 10 academic papers
│
├── EXPERIMENT_PLAN.md             # Detailed plan (12 phases)
├── SETUP_GUIDE.md                 # This file
└── README.md                      # Project overview

Expected Results

Baseline (Phase 1)

  • baseline_default: 91.16% ± 0.02% (reproduces CW1)

Learning Rates (Phase 2)

  • lr_1e5: ~90.5% (conservative, stable)
  • lr_2e5: ~91.2% (baseline)
  • lr_3e5: ~91.5% (expected best)
  • lr_5e5: ~90.0% (too aggressive)

Batch Sizes (Phase 3)

  • batch_8: ~91.0% (slower, noisier)
  • batch_16: ~91.2% (baseline)
  • batch_32: ~91.4% (faster, smoother)

Final Goal

  • Achieve >92% accuracy with optimized configuration
  • Document all findings with academic references
  • Write comprehensive 4-page LaTeX report

Tips for Success

  1. Start with Phase 1-2 to understand the baseline and find optimal LR
  2. Run experiments overnight on Mac M4 (each takes 30-60 min)
  3. Track results using results_analyzer.py after each phase
  4. Document everything - note which configs work and why
  5. Cite papers - Use references_distilbert.bib for your report
  6. Generate figures - All visualizations programmatically (no screenshots!)

Troubleshooting

"Module not found" errors

# Install missing packages
pip3 install torch transformers datasets scikit-learn numpy pandas

Out of memory on Mac M4

# Reduce batch size in experiment_configs.py
batch_size: 8  # instead of 16

MPS (GPU) errors

# Fall back to CPU in experiment_configs.py
device: 'cpu'  # instead of 'mps'

Student Information

Name: Martynas Prascevicius Student ID: 001263199 Course: COMP1818 Artificial Intelligence Applications University: University of Greenwich (2025-26) Deadline: Nov 19, 2025, 5pm (Grace: Nov 21, 2025, 5pm)