Skip to content

LLM Usage

Jump to bottom
Nick edited this page Nov 19, 2025 · 4 revisions

LLM Usage in PATAS

Purpose: This document explains how Large Language Models (LLMs) are used in PATAS, what data they process, and how their output is validated and constrained.

LLM Role: Pattern Discovery, Not Classification

LLMs play a crucial role in significantly improving pattern analysis accuracy through deep semantic understanding.

What LLMs Do

  1. Semantic Pattern Discovery: LLMs analyze aggregated spam signals to identify semantic patterns that go beyond simple keyword matching. They understand context, meaning, and variations in spam messages.

  2. Deep Semantic Analysis: Unlike rule-based methods that only match exact keywords or URLs, LLMs understand:

    • Semantic variations (e.g., "earn money" → "make cash" → "get income")
    • Contextual patterns (e.g., recognizing commercial spam intent across different phrasings)
    • Paraphrased content (e.g., LLM-generated spam variations)
    • Cross-language patterns (understanding spam intent across languages)

  3. Intelligent SQL Rule Generation: LLMs don't just generate SQL queries—they:

    • Understand the semantic meaning of discovered patterns
    • Propose SQL rules that catch semantic variations, not just exact matches
    • Generate rules that are both precise (low false positives) and comprehensive (catch variations)
    • Consider context and intent when creating rules

  4. Pattern Quality Assessment: LLMs evaluate the quality and safety of discovered patterns before they become rules, identifying potential false positive risks.

Deployment Options

On-Premise LLM (Recommended for production):

  • LLM runs inside your infrastructure
  • All processing happens offline during pattern mining
  • No data leaves your infrastructure

External LLM (e.g., OpenAI):

  • Requires sending aggregated data to external service
  • Data is encrypted and PII-redacted before sending
  • See "Data Privacy & LLM" section below

LLM Usage Flow

1. Pattern Mining Pipeline
   ↓
2. Aggregate spam signals (URLs, keywords, examples)
   ↓
3. Send aggregated summary to LLM (NOT individual messages)
   ↓
4. LLM performs deep semantic analysis:
   - Understands semantic patterns and variations
   - Identifies contextual spam intent
   - Generates pattern hypotheses with semantic understanding
   - Proposes SQL rules that catch semantic variations
   ↓
5. LLM Rule Validation (Optional):
   - Validates rule adequacy before production
   - Checks for potential false ban risks
   - Assesses semantic correctness
   ↓
6. Offline evaluation: test SQL rules on historical data
   ↓
7. Quality filtering: tier classification (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)
   ↓
8. Safety profiles: Conservative / Balanced / Aggressive
   ↓
9. Only then: rules can be used in production (with safety constraints)

Data Privacy & LLM

Sensitive Information Protection

Before sending data to LLM, PATAS encrypts sensitive information:

  • PII Redaction: Personal Identifiable Information (emails, phone numbers, credit cards) is automatically redacted or masked before sending to LLM
  • Data Encryption: Sensitive data is encrypted using configurable encryption methods
  • Minimal Data Exposure: Only aggregated signals and anonymized examples are sent to LLM
  • Configurable Privacy Levels: Different privacy modes control what data is sent and how it's protected

What Data LLM Sees

During Pattern Mining:

  • Aggregated signals only: Top URLs, top keywords, sample spam messages (limited to 10 examples)
  • Anonymized examples: User identifiers removed, PII redacted
  • NOT individual user messages in real-time
  • NOT full message history
  • NOT user identifiers (sender IDs, user names, etc.)

Example of aggregated data sent to LLM (after encryption/redaction):

{
  "total_spam": 1000,
  "total_ham": 200,
  "url_patterns": ["https://spam-site.com", "https://scam-link.net"],
  "keyword_patterns": ["earn money", "get rich", "work from home"],
  "spam_examples": [
    {"text": "Earn money fast! Click here...", "pii_redacted": true},
    {"text": "Get rich quick! Join now...", "pii_redacted": true}
  ]
}

Privacy Guarantees

  1. On-Prem Deployment: PATAS runs on your infrastructure by default
  2. Configurable Provider: LLM provider is configured by operator (internal endpoint or external)
  3. Strict Privacy Mode: Can disable external LLM providers entirely
  4. Sensitive Data Encryption: PII and sensitive information encrypted before LLM processing

See Privacy-and-Data-Protection for details.

LLM Configuration

Provider Configuration

# In app/config.py
llm_provider: str = "openai"  # or "none", "disabled", "local"
llm_model: str = "gpt-4o-mini"

Options:

  • "openai": Use OpenAI API (requires API key, sends data externally)
  • "local": Use local/on-prem LLM endpoint (configure endpoint URL, no external calls)
  • "none" or "disabled": Disable LLM entirely (pattern mining uses rule-based methods only)

Privacy Mode

In STRICT privacy mode:

  • External LLM providers are disabled by default
  • Only internal/on-prem LLM endpoints are allowed
  • Message texts are not stored in logs
  • Sensitive information is encrypted before any processing

See Privacy-and-Data-Protection for privacy mode details.

LLM Prompt Safety

The LLM prompt explicitly instructs:

  1. Narrow Patterns: Focus on explicit commercial/abusive spam, not broad categories
  2. No Sensitive Attributes: Never base rules on politics, religion, race, ethnicity, gender, sexual orientation
  3. Safe SQL Only: Generate only SELECT queries with whitelisted columns
  4. No Match-Everything: Rules must not match >80% of messages
  5. Interpretable: Patterns must be human-readable and explainable

See app/v2_llm_engine.py for the full prompt.

LLM Output Validation

All LLM-generated rules go through multiple validation layers:

1. SQL Safety Validation

  • ✅ Only SELECT queries
  • ✅ Only whitelisted tables/columns
  • ✅ No UPDATE/DELETE/INSERT/ALTER
  • ✅ No subqueries, no semicolons
  • ✅ No "match everything" patterns

See app/v2_sql_safety.py for implementation.

2. Coverage Check

  • ✅ Rules matching >80% of messages are rejected
  • ✅ Prevents over-broad rules from LLM

3. LLM Rule Validation (Before Production)

Before rules are deployed to production, LLM can validate them for adequacy and false ban risks:

  • ✅ Adequacy Check: LLM validates that the rule correctly identifies the intended spam pattern
  • ✅ False Ban Risk Assessment: LLM analyzes whether the rule could lead to false bans of legitimate users
  • ✅ Semantic Correctness: LLM verifies that the rule's semantic understanding matches the pattern intent
  • ✅ Edge Case Analysis: LLM identifies potential edge cases where the rule might fail
  • ✅ Token-efficient: ~200-400 tokens per validation
  • ✅ Graceful degradation: Skips if LLM unavailable (rules still go through other validation layers)

Rules flagged as "high risk" for false bans are rejected; "medium risk" rules are logged but allowed to proceed with manual review.

This validation step provides an additional safety layer before rules enter production, leveraging LLM's semantic understanding to catch potential issues that deterministic checks might miss.

See SQL_LLM_VALIDATION.md for detailed documentation.

4. Offline Evaluation

  • ✅ Rules tested on historical data
  • ✅ Precision, recall, ham hit rate calculated
  • ✅ Rules classified into tiers (SAFE_AUTO / REVIEW_ONLY / FEATURE_ONLY)

5. Safety Profiles

  • ✅ Only SAFE_AUTO rules in Conservative profile
  • ✅ Balanced/Aggressive profiles require manual review
  • ✅ Safety guardrails prevent unsafe auto-promotion

Summary

LLM Role in PATAS:

  • ✅ Deep semantic analysis (understands context, meaning, variations)
  • ✅ Significantly improves accuracy (catches semantic variations that rule-based methods miss)
  • ✅ Pattern discovery and rule generation (not real-time classification)
  • ✅ Aggregated data only (not individual messages, with PII encryption/redaction)
  • ✅ Configurable provider (internal or external, operator's choice)
  • ✅ Multi-layer validation (safety checks, LLM rule validation, evaluation, tiering)
  • ✅ Rule validation before production (adequacy and false ban risk assessment)

Privacy & Security:

  • ✅ On-prem deployment by default
  • ✅ LLM provider configurable by operator
  • ✅ Strict privacy mode available
  • ✅ Sensitive information encrypted before LLM processing
  • ✅ PII redaction and anonymization

Long-Term LLM Strategy

For information about the long-term direction for LLM integration in PATAS, including the potential development of a domain-specific PATAS-LLM for on-premise deployment, see the PATAS LLM Roadmap.

The roadmap describes:

  • Why PATAS is currently LLM-agnostic and works with external providers
  • Design constraints for a future in-house model (on-premise friendly, domain-specific, dense architecture)
  • Phased training strategy (evaluation harness → dataset curation → pre-training → fine-tuning → RL/DPO → productization)
  • Evaluation-first mindset and metrics
  • Tokenization considerations for abuse/spam detection

Last Updated: 2025-11-18

Clone this wiki locally