Add a Llama-like 7B training script for data ablations

[epwalsh]

This is a basic Llama-like 7B model with some changes to optimize for throughput, including:

• SWA on 3 out of every 4 layers, just like Olmo 3.
• FP8 linear layers with tensor-wise scaling.
• Flash attention 3.

Here's a bash script to launch a quick benchmarking run:

#!/usr/bin/env bash

cmd=${1:-launch}
cluster=${2:-ai2/augusta}
run_name=${3:-speed-test-for-data-ablations}

python src/scripts/train/Llama-like-7B.py "$cmd" "$run_name" "$cluster" \
  --launch.num_nodes=1 \
  --launch.num_gpus=8 \
  --launch.priority=high \
  --launch.workspace=ai2/OLMo-pretraining-stability \
  --trainer.callbacks.wandb.enabled=false \
  --trainer.callbacks.lm_evaluator.enabled=false \
  --trainer.callbacks.downstream_evaluator.enabled=false \
  --trainer.no_checkpoints \
  --trainer.no_evals \
  --trainer.hard_stop='{unit: steps, value: 100}'

See https://beaker.org/ex/01KCM6Y89ZAD3WF4M058XEFE8Y.

[tyler-romero]

LGTM