| title | Self-Hosted PromptLayer |
|---|---|
| description | Deploy PromptLayer in your own infrastructure for complete data control and compliance |
| icon | server |
PromptLayer can be deployed entirely within your own infrastructure, giving you complete control over your data while maintaining all the powerful features of our cloud platform. Our self-hosted solution inherits the same SOC 2, HIPAA, and GDPR compliance standards as our cloud offering.
Our self-hosted architecture consists of fully dockerized services designed for scalability and reliability:
The web interface for accessing PromptLayer's dashboard, analytics, and management features. Connects directly to the Backend API Service for all operations.
The core Python Flask application that handles all API requests, authentication, and business logic. This service orchestrates communication between all other components and serves as the primary entry point for both the frontend and SDK integrations.
The primary relational database storing all metadata, configurations, user data, and system state. We use PostgreSQL 15 for its robust performance, reliability, and advanced features.
High-performance storage for request/response data, logs, and large payloads. Supports both Amazon S3 and Google Cloud Storage, allowing you to use your existing cloud storage infrastructure.
In-memory data store using Valkey 8.1.0 (Redis-compatible) for:
- Job queue management
- Caching frequently accessed data
- Session management
- Real-time data processing
- APScheduler: Handles scheduled tasks, periodic jobs, and cron-like operations
- Celery Background Workers: Distributed task queue for asynchronous processing, data pipelines, and heavy computations
- Redis Queue Background Workers: Lightweight job processing for real-time operations and quick tasks
Isolated Docker container environment for safely executing code blocks in evaluations and the workflow builder. Provides sandboxed execution with resource limits and security controls.
For a production deployment, you'll need:
- 5+ backend nodes for core services (exact number depends on scale)
- 2 Redis/Valkey instances
- 1 PostgreSQL instance (with recommended replication for production)
- Object storage (S3 or GCS bucket)
- PostgreSQL: Version 15
- Redis: Valkey 8.1.0 (Redis-compatible)
- Object Storage: Amazon S3, Google Cloud Storage
- Container Runtime: Docker, Kubernetes
We provide pre-built Docker images with Helm charts for:
Optimized for Amazon Web Services with EKS, RDS, and S3 integration Designed for Google Cloud Platform with GKE, Cloud SQL, and GCS While we officially support AWS and GCP, we can work with other cloud providers based on your requirements. Contact our team for custom deployment options.-
Kubernetes with Helm (Recommended for production)
- Full orchestration and scaling capabilities
- Built-in health checks and auto-recovery
- Horizontal pod autoscaling support
-
Docker Compose (Development/testing)
- Quick setup for evaluation
- Suitable for single-node deployments
-
Custom Installation
- Available with enterprise support
- Tailored to your specific infrastructure
- Built-in Authentication: Default user management system with secure password policies
- Single Sign-On (SSO): Integration via Auth0 supporting:
- SAML 2.0
- OAuth 2.0 / OpenID Connect
- Active Directory / LDAP
Our self-hosted solution maintains the same security standards as our cloud platform:
- SOC 2 Type II compliant architecture
- HIPAA ready configurations
- GDPR compliant data handling
- Encryption at rest and in transit
- API key management with role-based access control
- Audit logging and compliance reporting
We expose parameters for automatic scaling based on:
- CPU utilization
- Memory usage
- Queue depth
- Request rate
Our team provides consultation to help configure auto-scaling rules optimized for your usage patterns.
- Multi-node backend deployment
- Redis replication with automatic failover
- PostgreSQL streaming replication
- Load balancing across service instances
We recommend Datadog for comprehensive monitoring, providing:
- Real-time metrics and dashboards
- Log aggregation and analysis
- APM tracing
- Custom alerts and notifications
Alternative monitoring solutions can be integrated based on your existing infrastructure.
We follow industry best practices for updates:
- Versioned Docker images with detailed release notes
- Rolling updates via Kubernetes
- Automated database migrations with rollback capabilities
- GitOps-compatible deployment workflows
- Export from Cloud: We provide full data export from PromptLayer Cloud for migration to self-hosted
- Import tools: Automated scripts for importing existing data
- Zero-downtime migration: Support for gradual migration strategies
- Automated backup schedules for PostgreSQL and object storage
- Point-in-time recovery capabilities
- Disaster recovery playbooks
- Data retention policies configurable to your requirements
Our team offers additional services to ensure successful deployment:
- Installation Support: Expert assistance with initial setup
- Architecture Review: Optimization recommendations for your use case
- Custom Integration: Tailored solutions for unique requirements
- Training: Comprehensive onboarding for your team
Ready to deploy PromptLayer in your infrastructure? Our enterprise team is here to help.
Contact our team to discuss your self-hosted deployment requirements and get a customized solution.