Skip to content

Custom Profiles

Jump to bottom
Nick edited this page Mar 10, 2026 · 2 revisions

Custom Aggressiveness Profiles

PATAS supports custom aggressiveness profiles for rule promotion, allowing you to fine-tune thresholds based on your specific requirements.

Overview

Aggressiveness profiles control when rules are promoted from shadow to active status based on:

  • Minimum Precision: Minimum precision required for promotion
  • Maximum Coverage: Maximum traffic coverage allowed
  • Minimum Sample Size: Minimum number of hits required
  • Maximum Ham Hits: Maximum false positives allowed

Predefined Profiles

PATAS includes three predefined profiles:

Conservative

  • Min Precision: 0.95 (95%)
  • Max Coverage: 0.05 (5%)
  • Min Sample Size: 100
  • Max Ham Hits: 5

Best for: High-stakes environments where false positives are costly.

Balanced (Default)

  • Min Precision: 0.90 (90%)
  • Max Coverage: 0.10 (10%)
  • Min Sample Size: 50
  • Max Ham Hits: 10

Best for: Most production environments.

Aggressive

  • Min Precision: 0.85 (85%)
  • Max Coverage: 0.20 (20%)
  • Min Sample Size: 30
  • Max Ham Hits: 20

Best for: Environments where catching spam is more important than avoiding false positives.

Custom Profiles

Define custom profiles in your configuration file (config.yaml):

# Aggressiveness Profile
aggressiveness_profile: moderate  # Use custom profile

# Custom Aggressiveness Profiles
custom_profiles:
  ultra_conservative:
    min_precision: 0.98
    max_coverage: 0.02
    min_sample_size: 200
    max_ham_hits: 2
  
  moderate:
    min_precision: 0.92
    max_coverage: 0.08
    min_sample_size: 75
    max_ham_hits: 8
  
  experimental:
    min_precision: 0.80
    max_coverage: 0.30
    min_sample_size: 20
    max_ham_hits: 30

Profile Parameters

  • min_precision (0.0-1.0): Minimum precision required for promotion
  • max_coverage (0.0-1.0): Maximum fraction of traffic that can be matched
  • min_sample_size (≥1): Minimum number of rule hits required
  • max_ham_hits (≥0 or null): Maximum false positives allowed (null = no limit)

Configuration Methods

YAML Configuration File

# config.yaml
aggressiveness_profile: moderate
custom_profiles:
  moderate:
    min_precision: 0.92
    max_coverage: 0.08
    min_sample_size: 75
    max_ham_hits: 8

Environment Variables

# Use predefined profile
AGGRESSIVENESS_PROFILE=conservative

# Custom profiles must be defined in config.yaml

Using Custom Profiles

In Code

from app.v2_promotion import PromotionService
from app.database import AsyncSessionLocal

async with AsyncSessionLocal() as db:
    # Use custom profile by name
    service = PromotionService(db, profile_name="moderate")
    
    # Or use predefined profile
    service = PromotionService(db, profile_name="conservative")

In API

Custom profiles are automatically used when specified in configuration. The API respects the configured profile.

In CLI

# Profile is read from configuration
patas promote-rules

Profile Selection Guidelines

When to Use Conservative

  • High-value user communications
  • Legal/compliance requirements
  • Low tolerance for false positives
  • Small user base where false positives are noticeable

When to Use Balanced

  • General production use
  • Moderate user base
  • Standard spam detection requirements
  • Default choice for most deployments

When to Use Aggressive

  • High-volume spam attacks
  • Large user base
  • Spam detection is critical
  • False positives are acceptable

When to Create Custom Profiles

  • Specific precision/recall requirements
  • Regulatory compliance needs
  • A/B testing different thresholds
  • Gradual rollout strategies

Validation

Custom profiles are validated on startup:

  • Precision: Must be between 0.0 and 1.0
  • Coverage: Must be between 0.0 and 1.0
  • Sample Size: Must be ≥ 1
  • Ham Hits: Must be ≥ 0 or null

Invalid profiles will cause startup errors with clear messages.

Best Practices

  1. Start Conservative: Begin with conservative thresholds and adjust based on results
  2. Monitor Metrics: Track precision, recall, and false positive rates
  3. Gradual Changes: Adjust thresholds incrementally
  4. Document Rationale: Document why specific thresholds were chosen
  5. Test First: Test new profiles in shadow mode before production

Troubleshooting

Profile Not Found

If you see "Unknown profile" errors:

  1. Verify profile name matches configuration
  2. Check for typos in profile name
  3. Ensure custom profiles are defined in config.yaml

Invalid Thresholds

If startup fails with threshold errors:

  1. Check threshold ranges (precision/coverage: 0.0-1.0)
  2. Verify sample size is ≥ 1
  3. Check ham hits is ≥ 0 or null

Related Documentation

Clone this wiki locally