Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions extensions/policy_routing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# routellm-policy-routing

Class-aware cache + routing policy mapping for routellm.

Translates **c12n** classification labels into a routellm
**CachePolicy** (skip / lookup / store / TTL / namespace / match
strategy) per the cache policy table from showcase scenario 3
(**T-0104**).

## Status

Paper-stable. Wire-up to routellm core's cache hook ships when
**US-0102** (cache-policy-hooks) lands.

## Install

```sh
pip install -e extensions/policy_routing
```

## Use

```python
from policy_routing import classify_to_policy, CacheAction
from policy_routing.integrations import build_cache_hook

# Direct mapping:
policy = classify_to_policy(request, classification_result)
if policy.action == CacheAction.SKIP:
...

# Or as a routellm cache hook (US-0102):
hook = build_cache_hook(classifier=my_c12n_classifier)
controller = Controller(..., cache_policy=hook) # planned API
```

## Cache policy table

| c12n signal | action | ttl | match | namespace |
| ----------------------- | ----------------- | ------ | -------- | -------------- |
| `PII: EMAIL/PHONE/SSN` | skip | 0 | exact | tenant |
| `Jailbreak: high` | skip | 0 | exact | (n/a) |
| `Toxicity: high` | skip | 0 | exact | (n/a) |
| `CodeContent: <lang>` | lookup_and_store | 7d | exact | workspace |
| `Domain: legal` | lookup_and_store | 15m | both | workspace |
| `Domain: math|cs` | lookup_and_store | 7d | both | workspace |
| easy + low-cost | skip | 0 | exact | (n/a) |
| hard + high-cost | lookup_and_store | 7d | semantic | workspace |
| `OutputFormat: code/json/yaml` | lookup_and_store | 7d | exact | workspace |
| (else) | lookup_and_store | 24h | both | workspace |

Precedence top-to-bottom (safety wins).

## Spec links

- US-0102 — `docs/stories/US-0102-cache-policy-hooks.md`
- Scenario 3 — `~/.ops/.tlc/tracks/tools-showcase-scenarios/scenarios/3-class-aware-llm-gateway.md`
- T-0193 — workspace_id namespace default
- T-0179 — wsm intra-tenant cache leak audit

## Test

```sh
cd extensions/policy_routing
pytest
```
29 changes: 29 additions & 0 deletions extensions/policy_routing/policy_routing/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
"""routellm.extensions.policy_routing.

Class-aware cache + routing policy mapping. Translates c12n
ClassificationResult labels into a routellm CachePolicy per the
showcase scenario 3 spec table.

Public API:
CachePolicy — dataclass mirroring routellm US-0102 spec
CacheAction — enum of legal action values
MatchStrategy — enum of legal match values
classify_to_policy — pure function: (request, classification) -> CachePolicy
DEFAULT_NAMESPACE_KEY — sentinel meaning "use workspace_id default"
"""

from policy_routing.policy import (
DEFAULT_NAMESPACE_KEY,
CacheAction,
CachePolicy,
MatchStrategy,
classify_to_policy,
)

__all__ = [
"CacheAction",
"CachePolicy",
"DEFAULT_NAMESPACE_KEY",
"MatchStrategy",
"classify_to_policy",
]
65 changes: 65 additions & 0 deletions extensions/policy_routing/policy_routing/integrations.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
"""Wire-up helpers for routellm's cache hook (US-0102).

Until US-0102 lands in routellm core, this module ships a thin
adapter that stages the integration: a builder returns a callable
matching the planned hook signature, so consumers can wire it now
and flip the import path once core ships.

Planned routellm contract (US-0102, line 32):

cache_policy: Callable[[Request, ClassificationResult], CachePolicy]

The classifier is injected as a separate callable so the hook itself
stays pure — easier to unit-test and swap.
"""

from __future__ import annotations

from typing import Any, Callable, Optional

from policy_routing.policy import (
DEFAULT_POLICY,
CachePolicy,
classify_to_policy,
)

# Type aliases for readability. `Any` because c12n bindings + routellm
# Request types may not be importable yet at adoption time.
Request = Any
ClassificationResult = Any
Classifier = Callable[[Request], ClassificationResult]
CacheHook = Callable[[Request, ClassificationResult], CachePolicy]


def build_cache_hook(
classifier: Optional[Classifier] = None,
) -> CacheHook:
"""Return a cache_policy hook for routellm's controller.

If `classifier` is provided AND the caller invokes the hook with
`classification=None`, the hook will run the classifier on the
request first. Otherwise the caller is expected to pass an
already-classified result (the common case — classification
middleware sits upstream of the cache hook).

Returns a callable matching US-0102 contract.
"""

def hook(
request: Request,
classification: ClassificationResult = None,
) -> CachePolicy:
Comment on lines +26 to +51
if classification is None and classifier is not None:
classification = classifier(request)
if classification is None:
return DEFAULT_POLICY
return classify_to_policy(request, classification)

return hook


__all__ = [
"CacheHook",
"Classifier",
"build_cache_hook",
]
Loading