Skip to content

README.md

Latest commit

 

History

History
77 lines (64 loc) · 1.51 KB

README.md

File metadata and controls

77 lines (64 loc) · 1.51 KB

RetentionEngine

A simple adapter implementation to transform pretrained Transformer-family models into the Titans architecture.

Experiments Setup

Exp 1: Comparison with Base Models [3B]

Baseline

  • Titans 4B (Gemma3 4B IT)
  • Atlas 4B (Gemma3 4B IT)
  • Llama3.2 3B Instruct
  • Qwen3 4B
  • Gemma3 4B IT
  • RAG + Llama3.2 3B Instruct
  • RAG + Qwen3 4B
  • RAG + Gemma3 4B IT

Suggested

  • Cronos-L 3B (Llama3.2 3B Instruct)
  • Cronos-Q 4B (Qwen3 4B)
  • Cronos-G 4B (Gemma3 4B IT)

Exp 2: Comparison with Base Models [8B]

Baseline

  • Titans 9B (Gemma2 9B IT)
  • Atlas 9B (Gemma2 9B IT)
  • Llama3.1 8B Instruct
  • Qwen3 8B
  • Gemma3 9B IT
  • RAG + Llama3.1 8B Instruct
  • RAG + Qwen3 8B
  • RAG + Gemma3 9B IT

Suggested

  • Cronos-L 8B (Llama3.1 8B Instruct)
  • Cronos-Q 8B (Qwen3 8B)
  • Cronos-G 9B (Gemma3 9B IT)

Exp 3: Comparison with Recurrent Models [8B]

Baseline

  • Titans 9B (Gemma2 9B IT)
  • Atlas 9B (Gemma2 9B IT)
  • Mamba2 8B
  • RWKV6 7B

Suggested

  • Cronos-L 3B (Llama3.2 3B Instruct)
  • Cronos-Q 4B (Qwen3 4B)
  • Cronos-G 4B (Gemma3 4B IT)
  • Cronos-L 8B (Llama3.1 8B Instruct)
  • Cronos-Q 8B (Qwen3 8B)
  • Cronos-G 9B (Gemma3 9B IT)

Build

git clone https://github.com/retentionlabs/RetentionEngine.git
cd RetentionEngine
uv pip sync

Usage

  • See usage.ipynb for usage examples.

Deploy

rm -rf dist/
rm -rf build/
rm -rf *.egg-info

python -m pip install --upgrade pip
pip install flit
python -m flit build

echo "Uploading..."
python -m flit publish

echo "Deployment complete!"