routing.mdx

title	Smart Routing
description	Automatically select the optimal LLM for each query based on task requirements

DeepMyst's smart routing capabilities intelligently direct each query to the most appropriate LLM based on the query's characteristics, required capabilities, and your optimization preferences.

Why Smart Routing Matters

Different LLMs excel at different tasks - some are better at creative writing, others at code generation, and others at mathematical reasoning. DeepMyst's router analyzes your queries and directs them to the most suitable model for each specific task, optimizing for performance, cost, or speed based on your preferences.

How Smart Routing Works

DeepMyst employs a sophisticated multi-step process for intelligent query routing:

1. Query Analysis: Each incoming query is analyzed to determine its:

   • Category (code, math, creative writing, etc.)

   • Complexity level (easy, medium, hard)

   • Required capabilities (reasoning, data analysis, etc.)

2. Model Evaluation: The system evaluates available models based on:

   • Performance benchmarks for the specific query category

   • Token cost considerations

   • Response latency requirements

   • Capability support

3. Decision Algorithm: A weighted scoring system considers:

   • Performance (weighted higher for complex queries)

   • Cost (weighted according to your budget preferences)

   • Latency (weighted based on speed requirements)

   • Capability match (ensuring all required capabilities are available)

4. Scoring and Selection: Each eligible model receives a composite score based on the weighted factors, and the highest-scoring model is selected.

5. Optional Fallbacks: If preferred models are unavailable, intelligent fallback options ensure reliability.

Using Smart Routing

DeepMyst offers several ways to leverage smart routing in your applications:

1. Automatic Model Selection

Use the DeepMyst-auto model identifier to let DeepMyst's router automatically select the optimal model:

from openai import OpenAI

client = OpenAI(
    api_key="your-deepmyst-api-key",
    base_url="https://api.deepmyst.com/v1"
)

# Let DeepMyst automatically select the best model
response = client.chat.completions.create(
    model="DeepMyst-auto",  # Automatic model selection
    messages=[
        {"role": "user", "content": "Create a Python function to analyze sales data"}
    ]
)

print(response.choices[0].message.content)

2. Model Group Routing

Route to the best model within a specific provider or model family:

# Select the best OpenAI model
response = client.chat.completions.create(
    model="DeepMyst-auto-openai",
    messages=[
        {"role": "user", "content": "Explain how transformers work in machine learning"}
    ]
)

# Select the best Claude model
response = client.chat.completions.create(
    model="DeepMyst-auto-anthropic",
    messages=[
        {"role": "user", "content": "Write a professional email requesting a meeting"}
    ]
)

Routing Categories

DeepMyst's router classifies queries into the following categories, each with specialized benchmarks:

Category	Description	Example Query
math	Mathematical calculations and proofs	"Solve for x: 3x^2 + 2x - 5 = 0"
code	Programming and software development	"Write a Python function to find prime numbers"
qa	General question answering	"Who won the World Cup in 2018?"
reasoning	Logical reasoning and deductions	"If all As are Bs, and some Bs are Cs, what can we say about As and Cs?"
creative_writing	Stories, poetry, and creative content	"Write a short story about a time traveler"
summarization	Condensing information	"Summarize the key points of this research paper"
instruction_following	Following complex instructions	"Create a step-by-step guide to baking a cake"
multilingual	Translation and cross-lingual tasks	"Translate this paragraph to Spanish"
knowledge_retrieval	Factual information across domains	"Explain how photosynthesis works"
tool_usage	Using tools and APIs	"Create a function that calls a weather API"
conversation	Multi-turn dialogue	"Continue this conversation naturally"

Detected Capabilities

The router identifies these key capabilities required by queries:

Capability	Description
real-time	Access to current information
function-calling	Ability to call functions or tools
search	Information retrieval capabilities
code-generation	Programming and code creation
data-analysis	Working with data and statistics
mathematical	Complex mathematical operations
translation	Language translation
reasoning	Logical reasoning and inference
creative-content	Creative and original content generation
summarization	Condensing and extracting key information
instruction-following	Following detailed or multi-step instructions

Best Practices

To get the most from DeepMyst's routing capabilities:

1. Use DeepMyst-auto for general queries: Let the router intelligently select the best model

2. Add priority flags for specific needs: Use -performance, -cost, or -speed when a particular factor is critical

3. Combine with optimization: Use DeepMyst-auto-optimize to both select the optimal model and reduce token usage

4. Adjust preferences based on data: Fine-tune routing preferences based on observed performance and cost outcomes