A comprehensive Python library for managing and versioning LLM prompts, with built-in A/B testing, metric tracking, and performance monitoring capabilities.

✨ Features • 🚀 Quick Start • 📖 Documentation • 🎯 Examples

---

🎯 Why Prompt Versioner?

Prompt Versioner provides enterprise-grade prompt management with:

• 🔄 Version Control: Full versioning of prompts
• 📊 Performance Tracking: Metrics and regression detection
• 🧪 A/B Testing: Built-in statistical framework
• ⚡ Real-time Monitoring: Alerts and dashboards
• 👥 Team Collaboration: Annotations and reviews
• 🎨 Modern UI: Responsive web dashboard

---

✨ Features

🔧 Core Functionality Automatic MAJOR/MINOR/PATCH versioning Metrics tracking: tokens, latency, quality, cost Multi-model performance comparison Export & share prompt history Optional Git integration	🧪 Advanced Testing & Monitoring A/B Testing framework Model performance benchmarking Automated regression detection Live metrics dashboards Custom alert thresholds
👥 Collaboration & Management Team annotations and feedback Version comparison & visual diff Search & filtering by metadata, performance, and tags	🎨 Modern Web Interface Interactive dashboard with responsive layout Dark/Light themes Tab navigation for Prompts, Testing, Comparison, Alerts Real-time updates

---

📦 Installation

Prerequisites

• Python 3.11+
• Poetry (recommended) or pip
• Git (optional)

Using PyPI

pip install prompt-versioner

---

🚀 Quick Start

from prompt_versioner import PromptVersioner, VersionBump

pv = PromptVersioner(project_name="my-first-project", enable_git=False)
pv.save_version(
    name="assistant-prompt",
    system_prompt="You are a helpful assistant.",
    user_prompt="Please answer the following question: {question}",
    bump_type=VersionBump.MAJOR
)

print("✅ Created first prompt version 1.0.0!")

---

🎨 Web Dashboard

Core Features

📋 Prompts Management

• Version history with visual diff
• Semantic search
• Bulk operations
• Live preview

📊 Metrics & Analytics

• Token usage, latency, cost analysis
• Quality score tracking
• Multi-model performance comparison with automatic "best model" detection

Model Comparison: Test the same prompt across different models (GPT-4, Claude, Gemini, etc.) and see aggregated metrics with automatic identification of the fastest, cheapest, and highest-quality model.

🧪 A/B Testing

• Split testing
• Real-time results

🔍 Version Comparison

• Side-by-side visual diff
• Metadata & performance delta
• Smart annotations

⚠️ Smart Alerts

• Automatic detection of regressions
• Cost and error monitoring
• Custom metrics & thresholds

---

💻 CLI Interface

# Initialization & Setup
pv init                              # Initialize prompt versioner

# Prompt Operations
pv list                              # List all tracked prompts
pv versions <prompt>                 # List versions of a prompt
pv show <prompt> <version>           # Show version details

# Model Pricing & Cost Estimation
pv models                            # List all models with pricing
pv models --sort-by input            # Sort by input price
pv models --filter gpt               # Filter specific models
pv estimate-cost <model> <in> <out>  # Estimate cost for usage
pv compare-costs <in> <out>          # Compare costs across models

# Version Comparison
pv diff <prompt> <v1> <v2>           # Show diff between versions
pv compare <prompt> <v1> <v2>        # Compare with metrics

# Management
pv delete <prompt> <version>         # Delete a version
pv rollback <prompt> <version>       # Rollback to version
pv clear-db                          # Clear database

# Dashboard
pv dashboard --port 5000             # Launch web dashboard

---

📖 Examples

Examples in examples/ are fully functional:

File	Description
basic_usage.py	Getting started
version_management.py	Advanced version control
metrics_tracking.py	Metrics logging
multi_models.py	Multi-model comparison
ab_testing.py	Statistical testing
performance_monitoring.py	Automated monitoring
summarization_example.py	Real-world summarization
run_dashboard.py	Launch dashboard
clear_db.py	Reset database

---

🌟 Contributing

See CONTRIBUTING.md for contribution guidelines.

---

📄 License

MIT License - LICENSE

---

📞 Support

• Documentation: https://pepes97.github.io/prompt-versioner/
• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

Build by Sveva Pepe, NLP Engineer
_{⭐ Star this project if it helps you build better AI applications!}